A u-shaped coding-decoding network-based uveitis lesion segmentation method

Abstract

The present application relates to the technical field of image segmentation, and discloses a u-shaped coding and decoding network-based uveitis lesion segmentation method, first, a multi-branch feature fusion module is constructed as a basic unit, and is stacked to form a multi-scale feature enhancement encoder. Secondly, a frequency domain cross-layer feature fusion module is designed in the jump connection, boundary information is extracted by Haar wavelet decomposition, and features are screened and fused by combining spatial and channel attention. A hybrid Mamba interactive bottleneck module is arranged at the top of the encoder, multi-level deep features are aligned and aggregated, global context is obtained through visual Mamba, and mutual correction of global and local features is carried out by using a joint attention map. In the decoding stage, an uncertainty perception recursive gating module is introduced, the Shannon entropy of the output of the prediction head is taken as an uncertainty map, the initial gate generated by the gating network is dynamically modulated, and the adaptive control of the introduction intensity of shallow details is realized.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to a method for segmenting uveitis lesions based on a U-shaped encoding and decoding network. Background Technology

[0002] Uveitis is an inflammatory disease of the eye that affects the iris, ciliary body, and choroid. Childhood uveitis accounts for about 5%–10% of all uveitis cases, with an incidence rate of about 4.3 / 100,000 and a prevalence of about 27.9 / 100,000. Although the overall incidence is lower than that in adults, it is characterized by its strong insidiousness, high risk of blindness, and possible association with systemic diseases. The posterior segment is involved in about 40% of cases, which can significantly increase the risk of visual impairment during the course of the disease. Retinal vasculitis, one of the severe manifestations of the posterior segment, is seen in 79.6% of idiopathic childhood uveitis.

[0003] In clinical assessment, fluorescein fundus angiography (FFA) is often considered the "gold standard" for evaluating posterior segment inflammation. It can sensitively visualize early uveitis abnormalities such as retinal vascular leakage, non-perfusion, neovascularization, and macular edema, providing a basis for judging inflammatory activity and monitoring treatment efficacy. However, low cooperation in children leads to poor image quality, subjective interpretation of images, and poor consistency, making the need for objective and quantitative analysis of uveitis lesions more urgent. In terms of segmentation and labeling of uveitis lesions, retinal vascular leakage (RVL) is defined as an irregular hyperfluorescent area surrounding retinal vessels; macular edema leakage (MEL) is usually located within one optic disc diameter in the center of the macula. Based on this, the macular edema leakage area is defined as an irregular, speckled, stellate, or ring-shaped hyperfluorescent area on FFA within a circular area with a radius of 1.5 mm centered on the fovea.

[0004] In the field of medical image processing, existing technologies have primarily evolved from traditional image processing to deep learning-driven automated analysis. Traditional methods typically rely on manually designed features combined with classic algorithms to achieve target extraction and segmentation, such as edge detection, thresholding, and region growing. However, these methods are limited in their feature representation capabilities and are sensitive to imaging noise and individual differences, making them difficult to reliably adapt to the complex structure of medical image data.

[0005] In recent years, deep learning methods have gradually become the mainstream technology for medical image analysis. Among them, models represented by convolutional neural networks (CNNs) can automatically extract local texture and morphological features through end-to-end learning, achieving significant progress in medical image classification, detection, and segmentation tasks. At the same time, novel architectures for modeling longer-range dependencies and efficient computation have also attracted attention. For example, the Mamba-like architecture based on the state-space model concept can enhance the model's ability to represent global or large-scale information while maintaining high efficiency, making it suitable for medical image segmentation scenarios with strong context dependencies.

[0006] Therefore, with the continuous advancement of related technologies, the processing and interpretation of medical images is gradually shifting from "human judgment relying on experience" to "efficient and consistent automated analysis." Based on these capabilities, medical imaging workflows are expected to receive more stable support in clinical processes such as screening, assessment, and follow-up, further promoting the standardization and intelligent development of clinical decision-making, and providing broader scope for the early identification and refined management of related diseases.

[0007] However, existing deep learning-based segmentation methods still have some drawbacks, including:

[0008] Since retinal vascular leakage (RVL) is often diffusely or patchily distributed along blood vessels, while macular leakage (MEL) is usually small in area and relatively concentrated in location, existing single-pathway or single-receptor field networks cannot simultaneously ensure complete coverage of large-scale lesions and stable localization of small-scale lesions, resulting in insufficient capacity to cover multiple scale lesions.

[0009] Existing U-shaped networks typically splice or add shallow and deep upsampled features directly in the spatial domain. For leakage areas with blurred boundaries, this coarse cross-layer fusion method can easily introduce aliasing noise from deep upsampling into the shallow structure, resulting in unclear boundary details and even weakening the role of shallow high-resolution features.

[0010] Existing technologies are insufficient in modeling long-range dependencies. When relying solely on convolutional operations, the network tends to focus more on local textures and underutilizes the global spatial relationships between lesions and the vascular tree, optic disc, and central macular region, thereby increasing the probability of confusion between lesions and similar high / low fluorescence interference regions.

[0011] Existing technologies lack sufficient decoding stability for high uncertainty regions and small target regions. In uveitis FFA images, the proportion of leaking foreground regions is usually small, especially macular effusion, which is even rarer. During the deep decoding stage, the network is prone to misclassifying blurred areas as background or large-scale retinal vascular effusion, resulting in insufficient recall of a small number of lesions.

[0012] The training constraints of existing technologies do not match the pixel distribution. If the loss function is not designed to account for the differences in pixel frequency among different categories in the training set, the model is more likely to favor the background and foreground regions that account for a large proportion, such as retinal vascular leakage, during optimization, which is not conducive to improving the learning intensity and training stability of categories that account for a small proportion.

[0013] In summary, existing lesion segmentation methods for uveitis FFA images suffer from inaccurate segmentation of lesion regions due to difficulties in accommodating multi-scale lesions, accurately extracting diffuse boundaries, insufficient utilization of long-range context, and unstable decoding of high-uncertainty regions. Summary of the Invention

[0014] Therefore, the technical problem to be solved by the present invention is to overcome the problems of inaccurate segmentation caused by the inability to take into account multi-scale lesions, the difficulty in accurately extracting diffuse boundaries, the insufficient utilization of long-range context, and the instability of decoding high uncertainty regions in the prior art.

[0015] To address the aforementioned technical problems, this invention provides a method for segmenting uveitis lesions based on a U-shaped encoding / decoding network, comprising: The FFA image to be segmented is input into a U-shaped encoder-decoder network. In the downsampling encoding path, it passes through N cascaded multi-branch feature fusion modules to obtain N encoded feature maps at different scales. In the upsampling decoding path, it passes through N-1 cascaded decoding modules to obtain the decoded feature map output by the first decoding module. This map is then input into the segmentation prediction head to obtain the segmentation probability map of the uveitis lesion. The segmentation result of the uveitis lesion is obtained based on the segmentation probability map. In this process, the downsampling encoding path and the upsampling decoding path pass features through a skip connection; the Nth encoded feature map is input into the bottleneck layer to obtain the bottleneck layer feature map; the bottleneck layer feature map serves as the starting deep feature of the upsampling decoding path. The multi-branch feature fusion module outputs an encoded feature map, which includes: passing the input feature map through two concatenated multi-branch fusion units with identical structures and independent parameters, and obtaining the output of the second multi-branch fusion unit as the encoded feature map; the multi-branch fusion unit includes: The input feature map is subjected to channel upscaling by passing it through a 1×1 convolution, group normalization, and activation function concatenated along the forward propagation direction to obtain an upscaled feature map. The upgraded feature map is passed through the first and second branches in parallel. In the first branch, it is passed through depthwise separable convolution, point convolution, group normalization and activation function concatenated along the forward propagation direction to obtain the context feature map. In the second branch, it is passed through 3×3 convolution, group normalization and activation function concatenated along the forward propagation direction to obtain the local detail feature map. After element-wise addition of the context feature map and the local detail feature map, channel recalibration is performed through the SE attention mechanism, followed by 1×1 convolution to obtain the convolutional feature map; The convolutional feature map is residually connected to the input feature map and used as the output of the multi-branch fusion unit.

[0016] Compared with the prior art, the above-described technical solution of the present invention has the following advantages: The uveitis lesion segmentation method based on a U-shaped encoding / decoding network described in this invention extracts encoded feature maps using a multi-branch feature fusion module in the downsampling encoding path. After obtaining the upscaling feature map, the multi-branch feature fusion module uses a parallel first branch with a 5×5 large kernel depthwise separable convolution to provide a wide geometric window to cover large areas of fused lesions, establish macroscopic topological consistency, and extract contextual feature maps. The parallel second branch uses a 3×3 convolution to capture high-frequency features of point-like and star-shaped micro-leakage and extract local detail feature maps. Then, based on channel recalibration and residual connections, encoded feature maps are obtained, improving feature expression efficiency, balancing accurate localization of micro-lesions with complete coverage of large-area lesions, alleviating the problem of insufficient segmentation of lesions with large size differences, and ensuring subsequent segmentation accuracy.

[0017] This invention introduces a frequency domain cross-layer feature fusion module on the skip connection path between the downsampling encoding path and the upsampling decoding path. During the cross-layer fusion process, a frequency domain attention mechanism is introduced. The high-frequency energy response obtained by Haar wavelet decomposition is used to enhance the expression of the boundary region and reduce the influence of low-frequency background interference and upsampling aliasing noise on the shallow structure, so as to ensure accurate identification of lesion boundaries and improve segmentation accuracy.

[0018] To overcome the inherent limitations of pure CNN architectures in long-range dependency modeling, this invention incorporates a hybrid Mamba interaction bottleneck module at the bottleneck layer to address visually similar anatomical artifacts. Through visual Mamba blocks, a 2D selective scanning mechanism is used to serialize images, providing the network with an anatomical coordinate system covering the entire image. This allows the network to not only see local fluorescent patches but also, by combining the global vascular tree and optic disc topology, eliminate interference terms that do not conform to the physical distribution of leakage at a geometric level. Furthermore, a global spatiotemporal continuity mapping is established in four directions, outputting a global feature map with a global receptive field. Simultaneously, a recurrent feature interaction unit bridges the semantic gap between the local aggregated feature map and the global feature map, obtaining a hybrid feature map.

[0019] This invention embeds an uncertainty-aware recursive gating U-RGM module in the corresponding stage design of the decoder. It uses Shannon information entropy as a metric for measuring the network's cognitive risk. When the network predicts minute lesions, the disorder in the prediction probability distribution causes a surge in local information entropy. U-RGM utilizes this surge signal to adaptively amplify the skip connection gating weights in that region, forcing the model to awaken the minute lesion features masked by downsampling from the shallow encoder layer. Furthermore, through an anchored class-weighted DiceFocal composite loss based on training set pixel frequencies, the loss constraint is made more consistent with the pixel distribution differences between retinal vascular leakage (RVL) and macular edema leakage (MEL), which helps improve the model optimization direction and convergence stability under class imbalance conditions. Attached Figure Description

[0020] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein: Figure 1 This is a flowchart illustrating the uveitis lesion segmentation method based on a U-shaped encoding / decoding network of the present invention. Figure 2 This is a schematic diagram of the multi-branch feature fusion module; Figure 3 This is a schematic diagram of the frequency domain cross-layer feature fusion module; Figure 4 This is a schematic diagram of the uncertainty-aware recursive gating module; Figure 5 This is a schematic diagram of the structure of the hybrid Mamba interaction bottleneck module; Figure 6 and Figure 7 This is a comparison chart of the segmentation results of this embodiment and various existing methods on the FFA dataset; Figure 6 (a) is the original FFA image; Figure 6 (b) is a schematic diagram of the gold standard; Figure 6 (c) is a segmentation diagram of the UNet model; Figure 6 (d) is a segmentation diagram of the AttUNet model; Figure 6 (e) is a segmentation diagram of the HRNet model; Figure 6 (f) is a segmentation diagram of the CPFNet model; Figure 6 (g) is a schematic diagram of the segmentation of the Missformer model; Figure 6 (h) is a schematic diagram of the segmentation of the VisionFM-BaseViT model; Figure 7 (i) is a schematic diagram of the segmentation of the TransUnet model; Figure 7 (j) is a schematic diagram of the segmentation of the H2Former model; Figure 7(k) is a segmentation diagram of the LM-Net model; Figure 7 (l) is a segmentation diagram of the GH-UNet model; Figure 7 (m) is a schematic diagram of the HGM model segmentation; Figure 7 (n) is a segmentation diagram of the FMI-Net model provided by the present invention. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0022] This invention designs a method for uveitis lesion segmentation based on frequency-domain cross-layer fusion and hybrid Mamba interaction. Based on the classic U-shaped encoder-decoder architecture, it incorporates a multi-branch fusion block (MBFB), a frequency cross-layer fusion module (FCFM), a global-local hybrid Mamba interaction bottleneck module (HMIB), and an uncertainty-aware recursive gated module (U-RGM). This method effectively overcomes the core problems of traditional U-shaped encoder-decoder networks, such as multi-scale lesion capture, loss of high-frequency edge information, insufficient long-range dependency modeling, and high uncertainty in complex boundary decisions.

[0023] Reference Figure 1 The diagram shows a flowchart of the uveitis lesion segmentation method based on a U-shaped encoding / decoding network of the present invention; the specific process includes: The FFA image to be segmented is input into a U-shaped encoder-decoder network. In the downsampling encoding path, it passes through N cascaded multi-branch feature fusion modules to obtain N encoded feature maps at different scales. In the upsampling decoding path, it passes through N-1 cascaded decoding modules to obtain the decoded feature map output by the first decoding module. This map is then input into the segmentation prediction head to obtain the segmentation probability map of the uveitis lesion. The segmentation result of the uveitis lesion is obtained based on the segmentation probability map. In this process, the downsampling encoding path and the upsampling decoding path pass features through a skip connection; the Nth encoded feature map is input into the bottleneck layer to obtain the bottleneck layer feature map; the bottleneck layer feature map serves as the starting deep feature of the upsampling decoding path. The multi-branch feature fusion module outputs an encoded feature map, which involves passing the input feature map through two concatenated multi-branch fusion units with identical structures and independent parameters, and obtaining the output of the second multi-branch fusion unit as the encoded feature map; the multi-branch fusion unit includes: The input feature map is subjected to channel upsizing by a 1×1 convolution, group normalization, and activation function concatenated along the forward propagation direction to obtain an upsizing feature map. The upgraded feature map is passed through the first and second branches in parallel. In the first branch, it is passed through depthwise separable convolution, point convolution, group normalization and activation function concatenated along the forward propagation direction to obtain the context feature map. In the second branch, it is passed through 3×3 convolution, group normalization and activation function concatenated along the forward propagation direction to obtain the local detail feature map. After adding the context feature map and the local detail feature map element by element, the channel is recalibrated through the SE attention mechanism, and then a 1×1 convolution is performed to obtain the convolutional feature map. The convolutional feature map is residually connected to the input feature map and used as the output of the multi-branch fusion unit.

[0024] The uveitis lesion segmentation method based on a U-shaped encoding / decoding network described in this invention extracts encoded feature maps using a multi-branch feature fusion module in the downsampling encoding path. After obtaining the upscaling feature map, the multi-branch feature fusion module uses a first branch and a second branch in parallel. The first branch uses depthwise separable convolution to achieve large receptive field context extraction, capturing a wider range of spatial dependencies with lower computational cost. The second branch achieves local detail texture extraction, focusing on extracting high-frequency local structural priors. After simultaneously extracting the context feature map and local detail feature map of the large receptive field, the encoded feature map is obtained based on channel recalibration and residual connections, improving feature expression efficiency and ensuring subsequent segmentation accuracy.

[0025] Specifically, retinal vascular leakage lesions in FFA images are widely distributed along retinal vessels, requiring a macroscopic geometric receptive field to capture their dispersed topological relationships. In contrast, macular edema leakage lesions are confined to the fovea region of the macula, with a smaller area; an excessively large receptive field would introduce too much background noise, causing feature annihilation of these rare, small targets during convolutional pooling. To address the multi-scale characteristics of retinal vascular leakage and macular edema leakage, the encoder of this model consists of stacked multi-level, multi-branch feature fusion modules (MBFB). The encoder is composed of five cascaded MBFB modules. The input of each MBFB module is the output feature of the previous MBFB module, and the output is the encoder feature at the current scale. The MBFB module simultaneously extracts contextual information from the large receptive field and local detail information through two parallel convolutional branches, and combines channel recalibration and residual connections to improve feature representation efficiency.

[0026] Reference Figure 2 The diagram shown is a structural schematic of the multi-branch feature fusion module; in the MBFB module, the input features... , Indicates the first The number of channels at each level, The height and width of the feature are represented respectively; first, a convolution with a kernel size of 1×1 is applied. Group normalization and ReLU activation function Perform channel dimensionality upscaling, mapping to a high-dimensional feature space, to obtain , is represented as: To achieve feature extraction from heterogeneous receptive fields, The residuals are fed into two parallel branches, left and right, respectively: in the left branch, for A depthwise separable convolution with a kernel size of 5×5 and padding of 2 is used. (Depthwise Convolution) enables context extraction with a large receptive field, capturing broader spatial dependencies at a lower computational cost, followed by a 1×1 point convolution. Pointwise Convolution, group normalization, and ReLU activation function are used to output features. , is represented as: In the right branch, for Using standard 3×3 convolution Then, after group normalization and ReLU activation, local detail texture extraction is achieved, focusing on extracting high-frequency local structural priors and outputting features. , is represented as: Two-branch output characteristics and The features are obtained by adding elements one by one along the spatial dimension. .feature Through the SE attention mechanism, firstly, global adaptive average pooling is applied. Spatial information is compressed into channel descriptors, and then channel weight vectors are generated through two layers of bottleneck MLP and a sigmoid function. , This enables adaptive recalibration of important feature channels, resulting in... ;in, This indicates global adaptive average pooling. This represents a multilayer perceptron (MLP). This represents the Sigmoid function. This represents the Hadamard product. Finally, the features... The residual projection is achieved through a 1×1 convolution and then compared with the input. Perform residual connections to obtain the output characteristics of the MBFB module. , is represented as: The encoder performs four stages of progressive downsampling to output hierarchical features at different scales. , , , and bottleneck layer features .

[0027] Because retinal vascular leakage and macular edema leakage vary greatly in morphology and size, ranging from tiny pinpoint leaks and stellate microaneurysms to diffuse edema areas formed by large-area barrier disruption, traditional single-receptor-field segmentation methods struggle to simultaneously achieve precise localization of small lesions and complete coverage of large lesions. This invention's encoder abandons a single convolution path and designs parallel heterogeneous receptive fields. In the MBFB module, the first branch uses a 5×5 deep separable convolution with a large kernel, providing a wide geometric window to cover large, fused lesions and establish macroscopic topological consistency; the second branch uses a 3×3 convolution to capture high-frequency features of pinpoint and stellate microleaks. This encoder design significantly improves the model's performance in medical image segmentation tasks.

[0028] In this embodiment of the invention, feature transfer between the downsampling encoding path and the upsampling decoding path is achieved through a skip connection, including: the encoded feature map output by the i-th multi-branch feature fusion module is passed through the i-th frequency domain cross-layer feature fusion module to obtain the i-th cross-layer fused feature map, which is then input into the decoding module as the optimized encoded feature map; wherein, 1≤i≤N-1, the N-1th to the 1st frequency domain cross-layer feature fusion modules are calculated sequentially from deep to shallow layers, and the i-th frequency domain cross-layer feature fusion module includes: The coded feature map output by the i-th multi-branch feature fusion module and the i-th deep upsampled feature map are used as inputs; The i-th encoded feature map is added element-wise to the i-th deep upsampled feature map to obtain the aggregated feature map; the aggregated feature map is subjected to a two-dimensional discrete Haar wavelet transform to obtain the core energy map, and then subjected to channel dimension compression, bilinear interpolation upsampling, 3×3 convolution and activation function to obtain the spatial attention map; the aggregated feature map is subjected to global pooling, multilayer perceptron and activation function to obtain the channel attention map; the spatial attention map and the channel attention map are subjected to mean modulation to obtain the joint attention map; After concatenating the i-th encoded feature map with the i-th deep upsampled feature map along the channel dimension, the initial fused feature map is obtained by performing a 1×1 convolution, group normalization, and activation function. After reconstructing the initial fused feature map spatially and channel-wise using the joint attention map, it is then residually connected with the i-th coding feature map to obtain the i-th cross-layer fused feature map, which is used as the i-th optimized coding feature map. Wherein, when 1≤i≤N-2, the i-th deep upsampled feature map is the feature map obtained by upsampling the cross-layer fusion feature map output by the (i+1)-th frequency domain cross-layer feature fusion module to the same spatial resolution as the i-th coding feature map; when i=N-1, the i-th deep upsampled feature map is the feature map obtained by upsampling the coding feature map output by the N-th multi-branch feature fusion module to the same spatial resolution as the i-th coding feature map.

[0029] Specifically, a two-dimensional discrete Haar wavelet transform is performed on the aggregated feature map to obtain the core energy map, including: Two-dimensional discrete Haar wavelet transform is performed on the aggregated feature map. Based on four two-dimensional orthogonal filter kernels, unbiased grouped depthwise convolution with a stride of 2 is used for filtering to obtain the low-frequency approximate sub-band, the first high-frequency detail sub-band, the second high-frequency detail sub-band and the third high-frequency detail sub-band. The high-frequency energy response is obtained by taking the absolute values ​​of the first high-frequency detail subband, the second high-frequency detail subband, and the third high-frequency detail subband respectively and then adding them together. Based on a preset scaling factor, the absolute values ​​of the high-frequency energy response and the low-frequency approximate subband are weighted and summed to obtain the core energy map.

[0030] Reference Figure 3 The diagram shown is a structural schematic of the frequency domain cross-layer feature fusion module; firstly, and Aggregation characteristics are obtained by adding elements together. ;right Perform a two-dimensional discrete Haar wavelet transform (2D-DWT). The DWT module uses four fixed two-dimensional orthogonal filter kernels, corresponding to low-pass and high-pass filtering respectively: , , , To ensure independent frequency domain decomposition of features in channel decoupling and achieve extremely high computational efficiency, the wavelet transform is equivalently reconstructed into an unbiased grouped depthwise convolution with a stride of 2. This is orthogonally decomposed into a low-frequency approximate subband LL and three high-frequency detail subbands LH, HL, and HH, respectively, as follows: , , , ;in, This represents a discrete convolution operator with a stride of 2.

[0031] In two-dimensional Haar wavelet decomposition The operator performs equal-weighted integral summation of the feature responses within the local spatial window; this operator effectively filters out abrupt texture changes and high-frequency imaging noise in local tissues of FFA images, extracting the low-frequency base component of fluorescein distribution, namely the core high-intensity subject of the leakage area and the macroscopically smooth background of fundus anatomy. The corresponding difference operator... Due to its zero-sum characteristic, it automatically suppresses the gentle low-frequency background, thus keenly capturing the weak high-frequency gradient of fluorescein when it diffuses at the edge.

[0032] To accurately locate edges and abrupt change regions, the high-frequency energy response of the DWT module is extracted and represented as follows: Combined with a certain proportion of low-frequency background information, a core energy map is constructed, represented as follows: , This is a learnable scaling factor. First, channel-dimensional compression is performed, then bilinear interpolation upsampling is used to restore the original resolution, and finally, a spatial attention map sensitive to high-frequency boundaries is generated by 3×3 convolution and the sigmoid function. , represented as: ;in, This represents average pooling along the channel dimension. This indicates upsampling. Meanwhile, for Generate channel attention maps using global pooling and MLP networks , represented as: Joint attention map Defined as mean modulation of both, it is expressed as: ;in, Indicates global average pooling. This indicates the sum of broadcasts.

[0033] To fully integrate the shallow spatial information of the encoder with the deep semantic information of the decoder, and to maintain the stability of the training process, this embodiment employs a residual enhancement fusion strategy. First, the encoder features... With deep upsampling features splicing is performed along the channel dimension. Then, using a set of 1×1 convolutions, group normalization, and ReLU activation functions, cross-channel information interaction and feature dimensionality reduction are performed to obtain the preliminary fused feature Z, which is expressed as: .

[0034] To achieve refined feature extraction, this embodiment utilizes a pre-computed joint attention map. The fused feature Z undergoes both spatial and channel reconstruction. Subsequently, the modulated feature is injected into the original encoder path via residual connections, as shown below: The features from each stage of the encoder are processed by the FCFM module to obtain multi-scale features. , , , .

[0035] In this embodiment, feature transfer between the downsampling encoding path and the upsampling decoding path is achieved through skip connections, including: adding an uncertainty-aware recursive gating module after the 2nd to N-1th frequency domain cross-layer feature fusion modules; the i-th uncertainty-aware recursive gating module includes: The cross-layer fused feature map output by the i-th frequency domain cross-layer feature fusion module and the i-th decoding side feature map are used as inputs; Based on the deep supervision mechanism, the i-th decoded side feature map is passed through a 1×1 convolutional auxiliary classification head and a Softmax function, and then the pixel-level normalized Shannon entropy is calculated to obtain the uncertain map. After concatenating the cross-layer fusion feature map and the decoding side feature map along the channel dimension, the initial jump information flow weight map is obtained by passing it through a gating network. Multiply the uncertainty graph by the preset non-negative intensity coefficient and add 1 to obtain the gain factor; multiply the gain factor element-wise with the initial jump information flow weight graph to obtain the modulation gating weight graph; The cross-layer fusion feature map is weighted element-wise using a modulation gate weight map to obtain the i-th effective skip feature map, which is then used as the optimized encoded feature map input to the decoding module. Wherein, when 2≤i≤N-2, the i-th decoding side feature map is the feature map obtained by upsampling the decoding feature map output by the (i+1)-th decoding module to the same spatial resolution as the i-th cross-layer fusion feature map; when i=N-1, the i-th decoding side feature map is the feature map obtained by upsampling the bottleneck layer feature map to the same spatial resolution as the i-th cross-layer fusion feature map.

[0036] Specifically, the cross-layer fusion feature map and the decoding-side feature map are concatenated along the channel dimension and then passed through a gating network to obtain the initial jump information flow weight map. , represented as: ; The gated network consists of 3×3 convolutions connected in series along the forward propagation direction. Group normalization Activation function 1×1 convolution With Sigmoid activation function ; This indicates the cross-layer fusion feature map With decoding side feature map Splice along the channel dimension.

[0037] Reference Figure 4The diagram shows the structure of the uncertainty-aware recursive gating module. To alleviate the problem of macular edema leakage and the easy omission or misclassification of small targets, the uncertainty-aware recursive gating module U-RGM is embedded in the 4th, 3rd and 2nd stages of the decoder. Shannon information entropy is used as a metric to measure the cognitive risk of the network. The feature flow in the skip connections is recursively guided by online risk assessment and prediction.

[0038] This embodiment uses the first... Taking the U-RGM module of stage 1 as an example, the working principle will be explained. For stage 2... The upsampling deep features of the stage are First, a deep supervision mechanism is used, employing a 1×1 convolutional auxiliary classification head to... Mapped to the log probabilities of the categories (Logits), and the probability distribution is obtained through Softmax. (K is the number of categories). Next, the pixel-level normalized Shannon entropy is calculated as an uncertainty map. , represented as: , It is a tiny constant greater than zero, used to prevent logarithmic zero overflow. This indicates taking the logarithm of a logarithmic value. This represents the summation of results across all categories; U is normalized to the range [0,1], with higher U values ​​indicating less confidence in the model's prediction of that pixel region. When performing skip feature injection, Alternating encoder skip features with the corresponding level Channel-dimensional concatenation is performed, and an initial jump information flow weight map is generated through a gating network consisting of 3×3 convolutions, group normalization, ReLU activation, 1×1 convolutions, and a sigmoid function. The core operation of the U-RGM module is to multiply the uncertainty graph U by a learnable non-negative intensity coefficient and add 1 as a gain factor. This gain factor is related to the weight graph. Element-wise multiplication yields the final modulation gating. , , These are learnable non-negative intensity coefficients; finally, the modulated gated weight map is used. Encoder skip features By performing element-wise weighting, the effective features for the injection decoding stage can be obtained. , represented as: When the model encounters a region of high uncertainty during the deep decoding stage, an increase in U will actively open the gate valve ( (Increase), forcing the network from shallow layers More high-resolution original spatial details are extracted for self-correction and edge refinement. Modulated and The channels are concatenated and then compressed using a 1×1 convolution to obtain the fused features. , represented as: On this basis, further The input consists of residual blocks composed of two 3×3 convolutions, with group normalization and ReLU activation function applied after each convolution layer. The residual connections then connect the transformed features to the input features. Add them together to obtain the output features of the current decoding stage. .

[0039] In this embodiment, the i-th decoding module outputs a decoding feature map, including: The i-th optimized encoded feature map and the i-th decoded feature map are concatenated along the channel dimension to obtain the concatenated feature map; 1≤i≤N-1; when i=1, the i-th optimized encoded feature map is the i-th cross-layer fusion feature map; when 1≤i≤N-2, the i-th decoded feature map is the feature map obtained by upsampling the decoded feature map output by the (i+1)-th decoded module to the same spatial resolution as the i-th optimized encoded feature map; when i=N-1, the i-th decoded feature map is the feature map obtained by upsampling the bottleneck layer feature map to the same spatial resolution as the i-th optimized encoded feature map. After passing the concatenated feature map through two cascaded convolutional residual blocks, it is concatenated with the residual of the concatenated feature map to obtain the i-th decoded feature map, represented as: ; in, Indicates the first The decoded feature maps output by each decoding module; the convolutional residual block consists of 3×3 convolutions sequentially connected along the forward propagation direction. Group normalization With activation function ; This represents the concatenation of feature maps, expressed as follows: , Represents a 1×1 convolution. This indicates splicing along the channel dimension. This represents the i-th decoder-side feature map. Indicates the input number Optimized encoding feature maps of each decoding module.

[0040] In this embodiment, the bottleneck layer is a hybrid Mamba interaction bottleneck module, comprising: The encoded feature map output by the Nth multi-branch feature fusion module, the cross-layer fusion feature map output by the N-1th frequency domain cross-layer feature fusion module, and the cross-layer fusion feature map output by the N-2th frequency domain cross-layer feature fusion module are unified to the spatial resolution of the Nth encoded feature map, then concatenated along the channel dimension, and then subjected to 1×1 convolution to map and obtain the local aggregated feature map. The local aggregated feature map is input into the visual Mamba block. Based on the 2D selective scanning mechanism, the local aggregated feature map is serialized, and a global spatiotemporal continuity mapping is established in four directions. The output is a global feature map with a global receptive field, including: Perform layer normalization on the local aggregated feature map to obtain the first normalized feature map; The first normalized feature map is processed by a linear layer, a depthwise separable convolution, a selective scanning unit, and a normalized layer sequentially connected along the forward propagation direction to obtain the second feature map; After passing the first normalized feature map through a linear layer, it is multiplied element-wise with the second feature map to obtain a gated fusion feature map. The gated fused feature map is passed through a linear layer and then broadcast and added to the local aggregated feature map to obtain the global feature map; The global feature map and the local aggregated feature map are input into the recurrent feature interaction unit for fusion to obtain a hybrid feature map, including: The local aggregated feature map is passed through a series of steps along the forward propagation direction, including global average pooling, 1×1 convolution, group normalization, activation function, 1×1 convolution and group normalization, to obtain the channel feature map. The local aggregated feature map is passed through a series of 1×1 convolutions, activation functions, depthwise separable convolutions, activation functions, depthwise separable convolutions, point convolutions, and group normalizations sequentially along the forward propagation direction to obtain the spatial feature map. After broadcasting and summing the channel feature map and the spatial feature map element by element, the normalized weight map is obtained by passing it through the Sigmoid activation function. The refined feature map is obtained by multiplying the normalized weight map element-wise with the global feature map; The refined feature map is passed through a series of 1×1 convolutions, group normalization, activation functions, 1×1 convolutions and sigmoid activation functions along the forward propagation direction, and then multiplied element-wise with the local aggregated feature map to obtain the mixed feature map. After passing the hybrid feature map through a 1×1 convolution, group normalization, and activation function, it is concatenated with the residual of the Nth encoded feature map to obtain the bottleneck layer feature map.

[0041] Reference Figure 5The diagram shows the structure of the Hybrid Mamba Interaction Bottleneck Module (HMIB). In the bottleneck layer of the FMI-Net network, to compensate for the inherent shortcomings of the pure CNN architecture in long-range dependency modeling, a global-local hybrid Mamba interaction bottleneck module (HMIB) was designed. First, the features from the encoder layer 3 of the FCFM module... 4th layer features Features of the bottleneck layer Each feature is uniformly resolved through bilinear interpolation and concatenated along the channel dimension. Then, semantically dense local aggregated features are generated via 1×1 convolution mapping. Subsequently, The input visual Mamba block (VSSBlock) serializes the image through a 2D selective scanning mechanism and establishes a global spatiotemporal continuity mapping in four directions, outputting features with a global receptive field. .

[0042] To bridge the semantic gap between CNN local features and Mamba global features, this module introduces a recurrent feature interaction unit. First, utilizing... Generate channel attention: to transfer features Global average pooling is used to compress the spatial dimension, followed by channel linear mapping using two 1×1 convolutions to recalibrate the feature channels. Group normalization and ReLU activation functions are introduced between the convolutions to enhance the non-linearity of feature representation. This process can be described by the following formula: ;in, This involves calculating a channel attention map for the input features. Then, local aggregated features are utilized. Generate spatial attention: Features are projected to a high-dimensional space using 1×1 convolutions to achieve inter-channel information recombination and feature dimensionality enhancement. Subsequently, a non-linear activation function is introduced to enhance feature expressive power. Then, using... Deep convolution models local neighborhood information in space, extracting fine-grained spatial features with low computational cost. Then, an activation function is used to further enhance the non-linear representation capability of the features. Based on this, a... Depthwise convolution further expands the receptive field. Then, pointwise convolution is used to remap the features back to the original dimensions, completing channel fusion. Finally, group normalization is used to normalize the output features, yielding the final spatial attention, represented as: ;in, This indicates that a spatial attention map is computed on the input features. This represents a 7×7 depth separable convolution.

[0043] Then channel attention components Spatial attention components Element-wise broadcast summation is performed, and the fusion result is normalized to a weight map in the [0,1] interval using the Sigmoid function. Finally, this attention weight is applied to the Mamba features for adaptive modulation, thereby correcting local detail representation biases and obtaining the corrected global features. , represented as: Next, the corrected global features are utilized. Callback applied to the feature again The enhanced bottleneck layer features are obtained. The specific approach is to Two 1×1 convolutional layers are applied sequentially, with group normalization and ReLU activation functions inserted in between. Then, the projected features are transformed into normalized weights using the Sigmoid function. Finally, the weights and features are... Element-by-element multiplication: Finally, regarding Use 1×1 convolution, group normalization, and ReLU activation function, and... Residual connections are used to obtain enhanced bottleneck layer features. , represented as: .

[0044] From a physical perspective, vascular leakage is a disordered diffusion process of fluorescein molecules, resulting in highly diffuse, gradually blurred lesion edges lacking sharp physical boundaries. Simultaneously, the fundus contains numerous visually similar interference items: avascular areas in the fovea, capillary non-perfusion areas, and numerous high / low fluorescence artifacts caused by refractive media opacities, retinal hemorrhage, or panretinal photocoagulation scars. To accurately isolate the true leakage boundary from these artifacts, this invention employs a dual strategy of local frequency domain sharpening and global coordinate anchoring. First, the FCFM module introduces a frequency domain attention mechanism during cross-layer fusion, utilizing the high-frequency energy response obtained from Haar wavelet decomposition to enhance boundary region expression and reduce the impact of low-frequency background interference and upsampling aliasing noise on shallow structures. Secondly, to address visually similar anatomical artifacts, the HMIB module introduces a global state space scan of visual Mamba. Mamba provides the network with an anatomical coordinate system covering the entire map, enabling the network to not only see local fluorescent patches, but also combine the global vascular tree and optic disc topology to geometrically eliminate interference terms that do not conform to the physical distribution of leakage.

[0045] Specifically, this embodiment employs a class-weighted DiceFocal composite loss during the training phase. Its core idea is to dynamically generate class weights based on the pixel frequency of the current training set, building upon the DiceFocalLoss, and to apply higher gradient contributions to classes with smaller proportions. Let the... The total number of pixels of class in the current segmentation training set is If the total number of pixels is N, then its pixel frequency is... for: To give larger training weights to categories with smaller proportions, while avoiding unstable optimization due to excessively rapid weight growth, the original weights are constructed using an inverse power-law form based on pixel frequency. , represented as: , This represents the weight adjustment parameter.

[0046] To avoid excessive underestimation by mean normalization in extreme cases where a small percentage of categories exist, an anchored normalization strategy is adopted. The original weights of the retinal vascular leakage category are used as the normalization benchmark, and its normalized weight is fixed at 1, resulting in a new category. , represented as: ; Category of retinal vascular leakage It is always equal to 1.

[0047] Based on the above embodiments, this embodiment of the invention trains a U-shaped encoder-decoder network by constructing a category-weighted DiceFocal composite loss based on optimized category weights; the construction of the category-weighted DiceFocal composite loss includes: No. The total number of pixels of each category in the current segmentation training set is ,based on With total number of pixels The ratio of the two values ​​is used to obtain the first value. Pixel frequency of category , represented as: ; Based on the Pixel frequency of category Construct the inverse power form of the first power, and construct the second power. Original weights of categories , represented as: ; This indicates the preset weight adjustment parameters; Based on anchoring normalization, using the original weights of the retinal vascular leakage category. Using the normalization benchmark, we obtain the first... Category optimization and category weighting , represented as: ; Based on optimized category weights Construct the category-weighted DiceFocal composite loss, expressed as: ; in, Represents the Dice loss function. This represents a probability prediction graph. Represents the actual label image; Represents the Focal loss function. This indicates the preset focus parameters.

[0048] In FFA images, the area of ​​leaking lesions is typically small (especially in early, localized macular edema leaks), leading to a severe foreground-background class imbalance. Furthermore, the variance of lesions varies greatly between images from different patients and at different angiographic stages: in some severe images, retinal vascular leaks may merge into large patches, occupying most of the field of view; while in other images, they may appear as tiny fluorescent dots. Faced with this highly unstable target distribution, traditional convolutional algorithms with fixed weights are prone to collapse or getting trapped in local optima, tending to predict tiny lesions as background. This invention introduces an uncertainty-driven dynamic enhancement mechanism based on information entropy through the U-RGM module. When the network predicts tiny lesions, the disorder in the prediction probability distribution triggers a surge in local information entropy. U-RGM utilizes this surge in signal to adaptively amplify the skip connection gating weights in the region, forcing the model to awaken the subtle lesion features masked by downsampling from the shallow layer of the encoder. Furthermore, by using an anchored class-weighted DiceFocal composite loss based on the pixel frequency of the training set, the loss constraint is made more consistent with the pixel distribution differences between retinal vascular leakage (RVL) and macular edema leakage (MEL), which is beneficial for improving the model optimization direction and convergence stability under class imbalance conditions.

[0049] In summary, this embodiment of the invention employs a multi-branch feature fusion (MBFB) module, consisting of a 1×1 convolutional projection, a 5×5 depth-separable convolutional branch, a 3×3 convolutional branch, SE channel recalibration, and residual connections, as the basic unit of the encoder. This unit can be repeatedly stacked to form a multi-scale feature enhancement encoder. A frequency-domain cross-layer feature fusion (FCFM) module is set in the skip connections, simultaneously performing summation and splicing branches on encoder features and deep upsampled features. A fixed Haar wavelet decomposition is used to generate high-frequency boundary energy responses, and spatial attention and channel attention are combined to perform selective fusion of the spliced ​​backbone features. A hybrid Mamba interaction bottleneck (HMIB) module is set at the top of the encoder to align and aggregate multi-level deep encoded features. After obtaining the global context through a visual Mamba block, the global features are corrected using a joint attention map generated from local features, and the corrected global features are used to back-modulate the local aggregated features. In the decoding stage, an uncertainty-aware recursive gating U-RGM module is set up to use the normalized Shannon entropy output by the auxiliary prediction head as an uncertainty map to dynamically modulate the initial gating generated by the gating network, thereby adaptively controlling the intensity of shallow detail information introduction.

[0050] In the embodiments of the present invention, the encoder based on the MBFB module, the frequency domain cross-layer fusion FCFM module, the Mamba hybrid interactive bottleneck HMIB module, and the uncertainty recursive gating U-RGM module can be used as a whole technical solution that works together, or each can be used as a plug-and-play module for other FFA image lesion segmentation networks.

[0051] Based on the above embodiments, experiments were conducted using the uveitis FFA dataset to verify the effectiveness of the method of the present invention. The present invention utilizes a uveitis lesion segmentation framework (Frequency–Mamba Interaction Network, FMI-Net) based on frequency-domain cross-layer fusion and Mamba hybrid interaction. This framework employs a multi-branch feature enhancement module in the encoding stage to accommodate lesions of different scales; designs a frequency-domain cross-layer feature fusion module in the cross-layer connections to reduce noise aliasing caused by direct spatial domain fusion; designs a global-local hybrid Mamba interaction module at the network bottleneck to enhance the global dependency modeling capability of deep features; introduces an uncertainty-aware recursive gating module in the decoding stage to adaptively adjust the information flow of skip connections based on prediction uncertainty; and combines a class-weighted DiceFocal composite loss based on training set pixel frequencies to improve the optimization strength and training stability for targets with a smaller proportion of the class. The experimental dataset contained 2,861 FFA images from 166 patients, which were randomly divided into 5 folds according to the principle of patient independence. The number of images in each fold was 571, 573, 572, 572, and 573, respectively. Among them, the number of images containing retinal vascular leakage was 517, 534, 550, 542, and 549, respectively, and the number of images containing macular leakage was 52, 47, 67, 46, and 43, respectively.

[0052] To standardize input image size and improve training efficiency, all images were resampled to 512×512. Online random data augmentation was employed, including at least random horizontal flipping, random vertical flipping, and random rotation within ±30°. To ensure comparable evaluation results, training, validation, and testing were all performed based on patient-independent 5-fold partitioning.

[0053] In this embodiment, the model training uses the AdamW optimizer, with an initial learning rate of 0.001, a batch size of 4, 100 training epochs, a weight decay coefficient of 0.05, and gradient clipping to improve training stability.

[0054] To objectively evaluate the performance of the method of this invention, Dice coefficient, Intersection over Union (IoU), Sensitivity (Sen), and Accuracy (Acc) are used as evaluation metrics. Dice and IoU are used to evaluate the degree of overlap of segmented regions, sensitivity is used to measure the recall ability of foreground lesions, and accuracy is used to measure the overall pixel classification accuracy.

[0055] To verify the performance of the FMI-Net of this invention, it was compared with several advanced deep learning segmentation networks, including UNet, AttUNet, HRNet, CPFNet, Missformer, VisionFM-BaseViT, TransUnet, H2Former, LM-Net, GH-UNet, and HGM. Table 1 shows the results of the comparative experiments.

[0056] Table 1 Comparative experimental results of the method of this invention with other advanced segmentation methods (RVL: retinal vascular leakage; MEL: macular edema leakage)

[0057] As shown in Table 1, the method of the present invention achieved the best results in five indicators: retinal vascular leakage Dice coefficient, macular edema leakage Dice coefficient, average Dice coefficient, crossover ratio (IoU), and sensitivity Sen, reaching 75.12±0.87, 74.73±4.11, 74.92±1.87, 63.24±1.80, and 76.05±0.83, respectively. In terms of accuracy (Acc), it reached 98.05±0.37, which is the second best value among all comparison methods, only slightly lower than AttUNet's 98.11±0.39.

[0058] In terms of the improvement, compared with the best comparison method for each indicator, the method of the present invention improves the Dice coefficient of retinal vascular leakage by 0.47 percentage points, the Dice of macular edema leakage by 2.07 percentage points, the average Dice by 1.70 percentage points, the IoU by 1.50 percentage points, and the Sensitivity by 1.65 percentage points. This indicates that the performance improvement of the method of the present invention does not only come from the optimization of segmentation of retinal vascular leakage with a large area, but also has a significant improvement in the segmentation performance of macular edema leakage with a smaller area.

[0059] The improvement in the Dice coefficient of macular edema leakage is the most significant, indicating that the frequency domain cross-layer fusion and uncertainty gating mechanism proposed in this invention has better adaptability to small targets, ambiguous boundaries, and high uncertainty regions. Compared with networks that rely solely on the superposition of spatial domain features, FCFM can introduce high-frequency boundary responses into the cross-layer fusion process; compared with networks that rely solely on local convolutions, the HMIB module enhances the network's utilization of global anatomical context; and U-RGM increases the injection intensity of shallow details when needed, thereby improving edge refinement and preventing missed detections.

[0060] Reference Figure 6 and Figure 7 The image shown is a comparison of the segmentation results of this embodiment and various existing methods on the FFA dataset. Pink represents retinal vascular leakage, and blue represents macular edema leakage. Figure 6 (a) is the original FFA image; Figure 6 (b) is a schematic diagram of the gold standard. Figure 6 (c) is a segmentation diagram of the UNet model; Figure 6 (d) is a segmentation diagram of the AttUNet model; Figure 6 (e) is a segmentation diagram of the HRNet model; Figure 6 (f) is a segmentation diagram of the CPFNet model; Figure 6 (g) is a schematic diagram of the segmentation of the Missformer model; Figure 6 (h) is a schematic diagram of the segmentation of the VisionFM-BaseViT model; Figure 7 (i) is a schematic diagram of the segmentation of the TransUnet model; Figure 7 (j) is a schematic diagram of the segmentation of the H2Former model; Figure 7 (k) is a segmentation diagram of the LM-Net model; Figure 7 (l) is a segmentation diagram of the GH-UNet model; Figure 7 (m) is a schematic diagram of the HGM model segmentation; Figure 7 'n' represents a segmentation diagram of the FMI-Net model provided by this invention. Figure 6 and Figure 7 The qualitative results show that the method of the present invention provides more complete coverage of large-scale diffuse leakage, more stable localization of smaller lesions near the macula, and fewer false detection areas in complex backgrounds, which can be corroborated by the quantitative results in Table 1.

[0061] Based on the above embodiments, ablation experiments were conducted on FMI-Net to analyze the function of each core module. The results of the ablation experiments are shown in Table 2. A basic U-shaped segmentation network consisting of an MBFB encoder and a conventional decoder was used as the baseline network, excluding the three enhancement modules FCFM, U-RGM, and HMIB.

[0062] Table 2 Ablation experimental results of the method of the present invention (Baseline: baseline network; RVL: retinal vascular leakage; MEL: macular edema leakage)

[0063] As shown in Table 2, when only FCFM is added to the baseline network, the average Dice increases from 74.14 to 74.54, and the Intersection over Union (IoU) increases from 62.18 to 62.96, representing increases of 0.40 and 0.78 percentage points, respectively. When only U-RGM is added, the average Dice and IoU increase by 0.28 and 0.54 percentage points, respectively. When only HMIB is added, the average Dice and IoU increase by 0.15 and 0.37 percentage points, respectively. Therefore, frequency domain cross-layer fusion contributes most directly to boundary modeling and cross-layer information alignment in this task.

[0064] Based on the results of the dual-module combination, the baseline network + FCFM + U-RGM has the highest average Dice (74.70) and IoU (62.97) among the dual-module schemes. The baseline network + FCFM + HMIB has the highest average Dice (74.60) and IoU (62.94), while the baseline network + U-RGM + HMIB has the highest average Dice (74.44) and IoU (62.93). This indicates that FCFM has good complementarity with the other two modules.

[0065] By simultaneously incorporating FCFM, U-RGM, and HMIB modules into the baseline network (i.e., the FMI-Net model of this invention), compared to the baseline network, the FMI-Net model of this invention improves the average Dice by 0.78 percentage points, the IoU by 1.06 percentage points, and the Dice for macular edema leakage by 0.86 percentage points. The improvement in segmentation performance for macular edema leakage is greater than that for retinal vascular leakage, indicating that the synergistic effect of these three modules is mainly reflected in the stable segmentation of small-area, highly uncertain lesions and regions with blurred boundaries.

[0066] Based on the above embodiments, in order to further analyze the impact of parameter settings in the loss function on model performance, the weight adjustment parameters in the class-weighted DiceFocal composite loss are analyzed. Ablation experiments were conducted, and the results are shown in Table 3.

[0067] Table 3. Ablation experimental results of the method of the present invention on the loss function hyperparameter.

[0068] In this experiment, the network structure and training strategy were kept unchanged, and only the parameters in the loss function were adjusted. This is used to control the influence of the class weights constructed based on pixel frequency on the overall loss function. When When = "-", it indicates that pixel frequency-based weight adjustment is not used, and only the basic DiceFocal loss is used; when Taking different values ​​indicates that the class weights are adjusted by inverse powers of different strengths to enhance the gradient contribution of the minority class during training.

[0069] As shown in Table 3, without class weight adjustment, the model achieves RVL, MEL, and average Dice scores of 74.90, 73.59, and 74.24, respectively. With... As the value of the model gradually increases, the model pays more attention to small sample classes, especially when the value of the model gradually increases. At α=4, the model achieves its highest MEL Dice and average Dice, at 74.73 and 74.92 respectively, with RVL Dice also reaching a high level of 75.12. Although RVL Dice further improves to 75.23 when α=5, MEL Dice decreases to 73.28, leading to a decrease in average Dice. Therefore, α=4 achieves a better balance between overall segmentation performance and small-sample class stability. When the weights are increased to 5, RVL Dice improves slightly, but MEL Dice decreases, resulting in a decrease in the overall average Dice. This indicates that excessively large weight coefficients may cause the model to overemphasize a particular class, thus affecting the overall segmentation stability.

[0070] The results above indicate that appropriate class weight adjustment helps alleviate the uneven pixel distribution among different lesion categories in the dataset. =4 achieves the best balance under the dataset and network structure used in this invention, and therefore this parameter setting is adopted in the training of the final model FMI-Net.

[0071] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0073] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0074] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0075] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for segmenting uveitis lesions based on a U-shaped encoding / decoding network, characterized in that, include: The FFA image to be segmented is input into a U-shaped encoder-decoder network. In the downsampling encoding path, it passes through N cascaded multi-branch feature fusion modules to obtain N encoded feature maps at different scales. In the upsampling decoding path, it passes through N-1 cascaded decoding modules to obtain the decoded feature map output by the first decoding module. This map is then input into the segmentation prediction head to obtain the segmentation probability map of the uveitis lesion. The segmentation result of the uveitis lesion is obtained based on the segmentation probability map. In this process, feature transfer between the downsampling encoding path and the upsampling decoding path is achieved through skip connections. This includes: the encoded feature map output by the i-th multi-branch feature fusion module is passed through the i-th frequency domain cross-layer feature fusion module to obtain the i-th cross-layer fused feature map, which serves as the i-th optimized encoded feature map, where 1 ≤ i ≤ N-1; after the 2nd to N-1th frequency domain cross-layer feature fusion modules, an uncertainty-aware recursive gating module is added; the N-th encoded feature map is input into the bottleneck layer to obtain the bottleneck layer feature map; the bottleneck layer is a hybrid Mamba interactive bottleneck module; and the bottleneck layer feature map serves as the starting deep feature of the upsampling decoding path. The multi-branch feature fusion module outputs an encoded feature map, which includes: passing the input feature map through two concatenated multi-branch fusion units with identical structures and independent parameters, and obtaining the output of the second multi-branch fusion unit as the encoded feature map; the multi-branch fusion unit includes: The input feature map is subjected to channel upscaling by passing it through a 1×1 convolution, group normalization, and activation function concatenated along the forward propagation direction to obtain an upscaled feature map. The upgraded feature map is passed through the first and second branches in parallel. In the first branch, it is passed through depthwise separable convolution, point convolution, group normalization and activation function concatenated along the forward propagation direction to obtain the context feature map. In the second branch, it is passed through 3×3 convolution, group normalization and activation function concatenated along the forward propagation direction to obtain the local detail feature map. After element-wise addition of the context feature map and the local detail feature map, channel recalibration is performed through the SE attention mechanism, followed by 1×1 convolution to obtain the convolutional feature map; The convolutional feature map is residually connected to the input feature map and used as the output of the multi-branch fusion unit. Among them, the N-1th to the 1st frequency domain cross-layer feature fusion modules are calculated sequentially from deep to shallow. In the i-th frequency domain cross-layer feature fusion module, the following is included: taking the encoded feature map output by the i-th multi-branch feature fusion module and the i-th deep upsampled feature map as input, introducing a frequency domain attention mechanism, obtaining the i-th cross-layer fused feature map, and using it as the optimized encoded feature map input to the decoding module; The i-th uncertainty-aware recursive gating module includes: taking the cross-layer fusion feature map output by the i-th frequency domain cross-layer feature fusion module and the i-th decoding-side feature map as input, obtaining the modulation gating weight map based on the deep supervision mechanism and gating network, performing element-wise weighting on the cross-layer fusion feature map, obtaining the i-th effective skip feature map, and using it as the optimized encoded feature map input to the decoding module; The Mamba interaction bottleneck module includes: unifying the encoded feature map output by the Nth multi-branch feature fusion module, the cross-layer fusion feature map output by the N-1th frequency domain cross-layer feature fusion module, and the cross-layer fusion feature map output by the N-2th frequency domain cross-layer feature fusion module to the spatial resolution of the Nth encoded feature map, then concatenating them along the channel dimension, and then performing a 1×1 convolution to map and obtain a local aggregated feature map; inputting the local aggregated feature map into the visual Mamba block, serializing the local aggregated feature map based on the 2D selective scanning mechanism, and establishing a global spatiotemporal continuity mapping in four directions to output a global feature map with a global receptive field; inputting the global feature map and the local aggregated feature map into the recurrent feature interaction unit for fusion to obtain a hybrid feature map; and then performing a 1×1 convolution, group normalization, and activation function on the hybrid feature map, followed by residual connection with the Nth encoded feature map to obtain the bottleneck layer feature map.

2. The uveitis lesion segmentation method based on a U-shaped encoding / decoding network according to claim 1, characterized in that, The i-th frequency domain cross-layer feature fusion module includes: The coded feature map output by the i-th multi-branch feature fusion module and the i-th deep upsampled feature map are used as inputs; The i-th encoded feature map is added element-wise to the i-th deep upsampled feature map to obtain the aggregated feature map; the aggregated feature map is subjected to a two-dimensional discrete Haar wavelet transform to obtain the core energy map, and then subjected to channel dimension compression, bilinear interpolation upsampling, 3×3 convolution and activation function to obtain the spatial attention map; the aggregated feature map is subjected to global pooling, multilayer perceptron and activation function to obtain the channel attention map; the spatial attention map and the channel attention map are subjected to mean modulation to obtain the joint attention map; After concatenating the i-th encoded feature map with the i-th deep upsampled feature map along the channel dimension, the initial fused feature map is obtained by performing 1×1 convolution, group normalization, and activation function. After reconstructing the initial fused feature map spatially and channel-wise using the joint attention map, it is then residually connected with the i-th encoded feature map to obtain the i-th cross-layer fused feature map, which is then used as the optimized encoded feature map input to the decoding module. Wherein, when 1≤i≤N-2, the i-th deep upsampled feature map is the feature map obtained by upsampling the cross-layer fusion feature map output by the (i+1)-th frequency domain cross-layer feature fusion module to the same spatial resolution as the i-th coding feature map; when i=N-1, the i-th deep upsampled feature map is the feature map obtained by upsampling the coding feature map output by the N-th multi-branch feature fusion module to the same spatial resolution as the i-th coding feature map.

3. The uveitis lesion segmentation method based on a U-shaped encoding / decoding network according to claim 2, characterized in that, The core energy map is obtained by performing a two-dimensional discrete Haar wavelet transform on the aggregated feature map, including: Two-dimensional discrete Haar wavelet transform is performed on the aggregated feature map. Based on four two-dimensional orthogonal filter kernels, unbiased grouped depthwise convolution with a stride of 2 is used for filtering to obtain the low-frequency approximate sub-band, the first high-frequency detail sub-band, the second high-frequency detail sub-band and the third high-frequency detail sub-band. The high-frequency energy response is obtained by taking the absolute values ​​of the first high-frequency detail subband, the second high-frequency detail subband, and the third high-frequency detail subband respectively and then adding them together. Based on a preset scaling factor, the absolute values ​​of the high-frequency energy response and the low-frequency approximate subband are weighted and summed to obtain the core energy map.

4. The uveitis lesion segmentation method based on a U-shaped encoding / decoding network according to claim 2, characterized in that, The i-th uncertainty-aware recursive gating module includes: The cross-layer fused feature map output by the i-th frequency domain cross-layer feature fusion module and the i-th decoding side feature map are used as inputs; Based on the deep supervision mechanism, the i-th decoded side feature map is passed through a 1×1 convolutional auxiliary classification head and a Softmax function, and then the pixel-level normalized Shannon entropy is calculated to obtain the uncertain map. After concatenating the cross-layer fusion feature map and the decoding side feature map along the channel dimension, the initial jump information flow weight map is obtained by passing it through a gating network. Multiply the uncertainty graph by the preset non-negative intensity coefficient and add 1 to obtain the gain factor; multiply the gain factor element-wise with the initial jump information flow weight graph to obtain the modulation gating weight graph; The cross-layer fusion feature map is weighted element-wise using a modulation gate weight map to obtain the i-th effective skip feature map, which is then used as the optimized encoded feature map input to the decoding module. Wherein, when 2≤i≤N-2, the i-th decoding side feature map is the feature map obtained by upsampling the decoding feature map output by the (i+1)-th decoding module to the same spatial resolution as the i-th cross-layer fusion feature map; when i=N-1, the i-th decoding side feature map is the feature map obtained by upsampling the bottleneck layer feature map to the same spatial resolution as the i-th cross-layer fusion feature map.

5. The uveitis lesion segmentation method based on a U-shaped encoding / decoding network according to claim 4, characterized in that, After concatenating the cross-layer fusion feature map and the decoding-side feature map along the channel dimension, the resulting image is passed through a gating network to obtain the initial skip information flow weight map. , represented as: ； The gated network consists of 3×3 convolutions connected in series along the forward propagation direction. Group normalization Activation function 1×1 convolution With Sigmoid activation function ; This indicates the cross-layer fusion feature map With decoding side feature map Splice along the channel dimension.

6. The uveitis lesion segmentation method based on a U-shaped encoding / decoding network according to claim 4, characterized in that... The i-th decoding module outputs a decoded feature map, including: The i-th optimized encoded feature map and the i-th decoded feature map are concatenated along the channel dimension to obtain the concatenated feature map; 1≤i≤N-1; when i=1, the i-th optimized encoded feature map is the i-th cross-layer fusion feature map; when 1≤i≤N-2, the i-th decoded feature map is the feature map obtained by upsampling the decoded feature map output by the (i+1)-th decoded module to the same spatial resolution as the i-th optimized encoded feature map; when i=N-1, the i-th decoded feature map is the feature map obtained by upsampling the bottleneck layer feature map to the same spatial resolution as the i-th optimized encoded feature map. After passing the concatenated feature map through two cascaded convolutional residual blocks, it is concatenated with the residual of the concatenated feature map to obtain the i-th decoded feature map, represented as: ； in, This represents the decoded feature map output by the i-th decoding module; the convolutional residual block consists of 3×3 convolutions sequentially connected along the forward propagation direction. Group normalization With activation function ; This represents the concatenation of feature maps, expressed as follows: , Represents a 1×1 convolution. This indicates splicing along the channel dimension. This represents the i-th decoder-side feature map. This represents the optimized encoded feature map input to the i-th decoding module.

7. The uveitis lesion segmentation method based on a U-shaped encoding / decoding network according to claim 1, characterized in that, The local aggregated feature map is input into the visual Mamba block. Based on the 2D selective scanning mechanism, the local aggregated feature map is serialized, and a global spatiotemporal continuity mapping is established in four directions. The output is a global feature map with a global receptive field, including: Perform layer normalization on the local aggregated feature map to obtain the first normalized feature map; The first normalized feature map is processed by a linear layer, a depthwise separable convolution, a selective scanning unit, and a normalized layer sequentially connected along the forward propagation direction to obtain the second feature map; After the first normalized feature is passed through a linear layer, it is multiplied element-wise with the second feature map to obtain a gated fusion feature map; The gated fusion feature map is passed through a linear layer and then broadcast and added to the local aggregated feature map to obtain the global feature map.

8. The uveitis lesion segmentation method based on a U-shaped encoding / decoding network according to claim 1, characterized in that, The global feature map and the local aggregated feature map are input into the recurrent feature interaction unit for fusion to obtain a hybrid feature map, including: The local aggregated feature map is passed through a series of steps along the forward propagation direction, including global average pooling, 1×1 convolution, group normalization, activation function, 1×1 convolution and group normalization, to obtain the channel feature map. The local aggregated feature map is passed through a series of 1×1 convolutions, activation functions, depthwise separable convolutions, activation functions, depthwise separable convolutions, point convolutions, and group normalizations sequentially along the forward propagation direction to obtain the spatial feature map. After broadcasting and summing the channel feature map and the spatial feature map element by element, the normalized weight map is obtained by passing it through the Sigmoid activation function. The refined feature map is obtained by multiplying the normalized weight map element-wise with the global feature map; The refined feature map is passed through a series of 1×1 convolutions, group normalization, activation functions, 1×1 convolutions, and sigmoid activation functions along the forward propagation direction, and then multiplied element-wise with the local aggregated feature map to obtain the mixed feature map.

9. The uveitis lesion segmentation method based on a U-shaped encoding / decoding network according to claim 1, characterized in that, The U-shaped encoder-decoder network is trained based on a class-weighted DiceFocal composite loss constructed by optimizing class weights; the construction of the class-weighted DiceFocal composite loss includes: No. The total number of pixels of each category in the current segmentation training set is ,based on With total number of pixels The ratio of the two values ​​is used to obtain the first value. Pixel frequency of category , represented as: ; Based on the Pixel frequency of category Construct the inverse power form of the first power, and construct the second power. Original weights of categories , represented as: ; This indicates the preset weight adjustment parameters; Based on anchoring normalization, using the original weights of the retinal vascular leakage category. Using the normalization benchmark, we obtain the first... Category optimization and category weighting , represented as: ; Based on optimized category weights Construct the category-weighted DiceFocal composite loss, expressed as: ； in, Represents the Dice loss function. This represents a probability prediction graph. Represents the actual label image; Represents the Focal loss function. This indicates the preset focus parameters.

Images