A spatial frequency cue collaborative network-based lesion segmentation method

Abstract

The application provides a kind of space frequency prompt collaborative network's focus segmentation method, belong to image processing technical field. Including: obtaining the OCTA image data of diabetic retinopathy and its corresponding focus annotation data constructs focus segmentation dataset;Construct space-frequency-prompt collaborative segmentation network, including spatial encoder, wavelet frequency encoder, focus prompt generator, prompt guide three modal adaptive fusion module and prompt-frequency perception decoder;The focus segmentation dataset after pre-processing is input into space-frequency-prompt collaborative segmentation network, and model training is completed;Collect the OCTA image data of diabetic retinopathy to be segmented, and input into the space-frequency-prompt collaborative segmentation network after training to output focus segmentation result.The application improves the accuracy, continuity and robustness of focus area segmentation.

Description

Technical Field

[0001] This invention relates to a lesion segmentation method using a spatial frequency cueing collaborative network, belonging to the field of image processing technology. Background Technology

[0002] Diabetic retinopathy is a common microvascular complication in diabetic patients, which can lead to vision impairment or even blindness as the disease progresses. Optical coherence tomography (OCTA) angiography can non-invasively and rapidly display the microvascular structure and local perfusion status of the retina, providing important imaging evidence for early screening, ischemic area analysis, and clinical intervention of diabetic retinopathy. Among these, lesion areas such as non-perfusion areas can reflect the degree of retinal ischemia and the progression of the disease; therefore, accurate segmentation of lesion areas in OCTA images is of significant clinical importance.

[0003] However, in actual OCTA images, diabetic retinopathy-related lesions typically exhibit blurred boundaries, irregular shapes, low local contrast, complex texture distribution, and significant interweaving with background vascular structures. Existing deep learning-based segmentation methods mostly rely on convolutional networks, attention networks, or Transformer networks for spatial domain feature extraction. While these methods can achieve some segmentation results, they are still prone to problems such as missed segmentation, missegmentation, and boundary discontinuities under low-contrast lesions, blurred boundaries, and complex vascular background interference.

[0004] Furthermore, OCTA images contain not only spatial structural information but also frequency information reflecting local boundaries, texture variations, and fine-grained abrupt changes. Existing methods often focus on spatial domain semantic modeling, failing to fully utilize the complementary relationship between low-frequency structural information and high-frequency detail information. Simultaneously, most existing segmentation networks rely on implicit feature learning, lacking explicit cues for lesion regions, resulting in insufficient attention to key lesion areas during feature fusion and decoding recovery stages. Therefore, it is necessary to design an OCTA lesion segmentation method capable of jointly modeling spatial structure, frequency details, and lesion cues to improve the accuracy and robustness of diabetic retinopathy lesion segmentation. Summary of the Invention

[0005] The purpose of this invention is to provide a lesion segmentation method based on spatial frequency cueing collaborative networks, which improves the accuracy, continuity, and robustness of lesion region segmentation.

[0006] To achieve the above objectives, the present invention employs the following technical solution: A lesion segmentation method using a spatial frequency cueing collaborative network includes the following steps: OCTA image data of diabetic retinopathy and its corresponding lesion annotation data were obtained to construct a lesion segmentation dataset, which was then preprocessed. A spatial-frequency-cue collaborative segmentation network is constructed, comprising a spatial encoder, a wavelet frequency encoder, a lesion cue generator, a cue-guided trimodal adaptive fusion module, and a cue-frequency-aware decoder. The spatial encoder and wavelet frequency encoder are input to the lesion segmentation dataset, respectively, to extract multi-scale spatial features and multi-scale frequency features. The lesion cue generator is input with the highest-level spatial features from the multi-scale spatial features and the highest-level frequency features from the multi-scale frequency features to generate global lesion cue and a top-level lesion cue map. The cue-guided trimodal adaptive fusion module is input with the top-level spatial features, the top-level frequency features, and the top-level lesion cue map, and performs feature fusion to obtain high-level joint features. The cue-frequency-aware decoder is input with the high-level joint features, multi-scale spatial features, multi-scale frequency features, and multi-scale lesion cue map, and outputs the lesion segmentation result. The preprocessed lesion segmentation dataset is input into the spatial-frequency-cue co-segmentation network, and the network parameters are jointly optimized by dynamic combination segmentation loss and cue alignment loss to complete model training. Acquire OCTA image data of diabetic retinopathy to be segmented, input the trained spatial-frequency-cue co-segmentation network, and output the lesion segmentation results.

[0007] Preferably, the spatial encoder includes a shallow feature extraction module and a hierarchical encoding stage; the shallow feature extraction module extracts initial spatial features through successive convolution, normalization, and activation operations; the hierarchical encoding stage includes four encoding stages, the first encoding stage sequentially including a convolutional block, a residual enhancement module, and a channel attention module, the second, third, and fourth encoding stages sequentially including a convolutional block, a residual enhancement module, a channel attention module, and downsampling; the first encoding stage takes the initial spatial features as input and encodes them into multi-scale spatial features. The second, third, and fourth encoding stages respectively use the multi-scale spatial features output from the previous encoding stage as input, and encode them into multi-scale spatial features. Multi-scale spatial features Multi-scale spatial features Multi-scale spatial features The highest level of spatial characteristics; The wavelet frequency encoder sequentially includes a wavelet decomposition module, a frequency domain shallow feature extraction module, and a frequency coding stage. OCTA image data undergoes two-dimensional discrete wavelet transform through the wavelet decomposition module, decomposing it into low-frequency sub-bands and high-frequency sub-bands. The low-frequency sub-bands and high-frequency sub-bands are spliced ​​along the channel dimension to form frequency domain input features. The frequency domain input features are initially encoded through the frequency domain shallow feature extraction module, and then multi-scale frequency features are extracted step by step through the frequency coding stage. The frequency encoding stage comprises four frequency encoding stages. The first frequency encoding stage sequentially includes a convolutional block, a frequency enhancement module, and a channel attention module. The second, third, and fourth frequency encoding stages sequentially include a convolutional block, a frequency enhancement module, a channel attention module, and downsampling, respectively. The first frequency encoding stage takes the initially encoded frequency domain input features as input and encodes them into multi-scale frequency features. The second, third, and fourth frequency encoding stages respectively use the multi-scale frequency features output from the previous frequency encoding stage as input, and encode them into multi-scale frequency features. Multi-scale frequency characteristics Multi-scale frequency characteristics Multi-frequency spatial characteristics This represents the highest-level frequency characteristics.

[0008] Preferably, the lesion indication generator includes a semantic anchoring branch and a spatial positioning branch; The semantic anchoring branch concatenates the highest-level spatial features and the highest-level frequency features after global average pooling, and then obtains a joint global semantic vector through a multilayer perceptron. The joint global semantic vector is then used to perform gating updates on the learnable lesion query vector to obtain the prompt query vector. The spatial positioning branch concatenates the highest-level spatial features and the highest-level frequency features, and then sequentially passes through... Convolution, normalization, and activation operations are mapped to a unified embedding space to obtain local feature representations. The similarity response between the query vector and the local feature representation is calculated, and the mean of the similarity responses is aggregated to generate an initial spatial cue response. The aggregated mean of the query vectors is then input into a bias mapping module to generate a global bias term. Finally, the global bias term and the initial spatial cue response are passed through an activation function to obtain the highest-level lesion cue map. The highest-level lesion visualization image is upsampled level by level to obtain multi-scale lesion visualization images. Multiscale lesion visualization Multiscale lesion visualization .

[0009] Preferably, the prompt guides the three-modal adaptive fusion module to perform feature fusion to obtain high-level joint features, specifically as follows: The spatial features, frequency features, and lesion indication map of the highest layer were analyzed respectively through... Channel alignment is performed using convolution, channel mapping layers, or channel expansion operations. The highest-level spatial features, highest-level frequency features, and highest-level lesion indication map after channel alignment are stitched together and input into a weight generation network. The adaptive weights for the three modes are obtained through the Softmax function. Based on adaptive weights, the highest-level spatial features, highest-level frequency features, and highest-level lesion indication map are weighted and summed to obtain the high-level joint features.

[0010] Preferably, the cue-frequency sensing decoder includes a top-level cue gating module and a multi-scale competitive decoding module; The top-level prompting gating module uses the highest-level lesion prompting map to perform residual enhancement on the high-level joint features to obtain the initial decoding features; The multi-scale competitive decoding module includes a four-stage decoding process, with inputs of initial decoding features and a multi-scale spatial feature set. Multi-scale frequency feature set Multiscale lesion atlas Multi-scale competitive decoding is performed sequentially from the highest layer to the shallowest layer. The decoding features of the first-level decoding stage are used to obtain the lesion segmentation probability map through the output convolutional layer and activation function.

[0011] Preferably, each decoding stage of the multi-scale competitive decoding module includes, in sequence: upsampling, competitive gating, dual-branch feature enhancement, skip feature fusion, and decoding block; The specific method for performing multi-scale competitive decoding sequentially from the highest layer to the shallowest layer is as follows: For each scale Multiscale lesion visualization based on the current scale By performing probability normalization and element-wise complementary transformation, we obtain the construct complementary hint graph. And calculate the confidence level of the prompt: , in, Indicates the first A lesion illustration at various scales, Indicates the first Confidence level of each scale, ; Will , and Input gating network, generate spatial branch weights and frequency branch weights : , in, Indicates a gating network. Indicates the first Spatial branch weights at each scale Indicates the first Frequency branch weights at each scale Indicates splicing; Based on the spatial branch weight and frequency branch weight, residual gating enhancement is performed on the spatial and frequency features at the corresponding scales to obtain the enhanced spatial features. and enhanced frequency characteristics : , in, Indicates the first Multi-scale spatial features at various scales; Indicates the first Multi-scale frequency features at multiple scales; This indicates the spatial characteristics after gating enhancement; This indicates the frequency characteristics after gating enhancement; The gated enhanced spatial and frequency features are spliced ​​together in the channel dimension, and then... Convolutional compression results in unified skip connection features; The skip connection features are then concatenated with the current scale decoding features, and the feature representation at the current scale is recovered through the decoding block: , in, Indicates the first Decoding features at each scale; This indicates the decoding features of the previous layer; This indicates the initial decoding characteristics after the top-level cue gating enhancement; Indicates the first Skip connectivity features at various scales; This indicates an upsampling operation or a scale alignment operation; Indicates the first Decoding blocks at various scales; when the highest-level features are consistent with the current decoding scale. Employ identity mapping or scale-aligned convolution. This indicates the initial decoding features after being enhanced by top-level cue gating.

[0012] Preferably, the total loss of the spatial-frequency-cue cooperative segmentation network is as follows: , , , in, Indicates the total training loss; Represents the dynamic combination segmentation loss; This indicates a loss of alignment. This indicates the alignment loss weights; express Middle The label value at each position, This image shows a downsampling pattern up to the highest layer of the lesion. True lesion annotation with the same spatial resolution; Image showing the lesion at the highest layer Middle The response value at each location; Indicates the number of pixels involved in the calculation of the cue alignment loss; Indicates training rounds, This represents the dynamic weights of the Dice loss during training. This represents the dynamic weights of the Tversky loss during training. This represents the dynamic weights of the FocalTversky loss during training; The Dice loss is used to improve the overlap between the predicted segmentation results and the ground truth labels. Tversky loss is used to mitigate the class imbalance between the foreground and background of lesions. FocalTversky loss is used to enhance the model's ability to learn from difficult samples, low-contrast regions, and complex boundary regions.

[0013] Preferably, the preprocessing includes size normalization, grayscale normalization, image enhancement, and label mask alignment of the OCTA image data; the image enhancement includes random flipping and / or random rotation and / or brightness and contrast adjustment and / or Gamma transformation and / or contrast-limited adaptive histogram equalization and / or low-intensity noise perturbation, and the image enhancement is applied synchronously to the original OCTA image data and the corresponding lesion label mask.

[0014] A lesion segmentation apparatus for a spatial frequency cueing cooperative network includes a processor and a memory storing program instructions, wherein the processor is configured to execute the lesion segmentation method for the spatial frequency cueing cooperative network when the program instructions are executed.

[0015] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the lesion segmentation method of the spatial frequency cueing cooperative network.

[0016] The advantages of this invention are as follows: This invention extracts spatial structural information and frequency detail information through a spatial encoder and a wavelet frequency encoder, respectively, enabling the model to simultaneously focus on the overall morphology, local texture, and blurred boundary changes of the lesion region, thus enhancing the ability to identify low-contrast lesion regions. By introducing a learnable lesion query vector, global semantic gating, and local spatial similarity calculation through a lesion cue generator, an explicit lesion cue map is generated, allowing the model to continuously focus on potential lesion regions during the fusion and decoding stages, reducing missed classifications. Through a cue-guided trimodal adaptive fusion module, the contributions of spatial features, frequency features, and cue features are dynamically adjusted according to the lesion response at different spatial locations, avoiding information redundancy and background interference caused by simple splicing or addition. Through a cue-frequency-aware decoder, a lesion cue map and a competitive gating mechanism are introduced during the progressive restoration of spatial resolution, dynamically adjusting the proportion of spatial and frequency features, improving the ability to restore lesion boundaries and suppress background interference. By dynamically combining segmentation loss and cue alignment loss for joint optimization, the model's training stability and lesion segmentation robustness are improved, taking into account region overlap, class imbalance, difficult sample learning, and cue branch consistency constraints.

[0017] In summary, this invention can enhance the model's ability to identify low-contrast lesions, blurred boundaries, and complex texture regions in OCTA images, thereby improving the accuracy and robustness of diabetic retinopathy lesion segmentation. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0019] Figure 1 This is a flowchart of the OCTA image lesion segmentation method of the present invention; Figure 2 This is a diagram illustrating the overall structure of the spatial-frequency-cue collaborative segmentation network of the present invention. Figure 3 This is a schematic diagram of the spatial encoder structure of the present invention; Figure 4 This is a schematic diagram of the wavelet frequency encoder structure of the present invention; Figure 5 This is a schematic diagram of the lesion segmentation visualization results of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1 like Figure 1 - Figure 2 As shown, a lesion segmentation method based on a spatial frequency cueing collaborative network addresses the problems of blurred lesion region boundaries, low local contrast, insufficient utilization of frequency details, and lack of explicit lesion guidance in existing technologies using OCTA (Optical Frequency Awareness and Tracing) methods. The method includes the following steps: S1: Obtain OCTA image data of diabetic retinopathy and its corresponding lesion annotation data to construct a lesion segmentation dataset and perform preprocessing; S2: Construct a spatial-frequency-cue collaborative segmentation network, including a spatial encoder, a wavelet frequency encoder, a lesion cue generator, a cue-guided three-modal adaptive fusion module, and a cue-frequency sensing decoder; The spatial encoder and wavelet frequency encoder are respectively input into the lesion segmentation dataset to extract multi-scale spatial features and multi-scale frequency features; The lesion indication generator takes the highest-level spatial feature of the multi-scale spatial features and the highest-level frequency feature of the multi-scale frequency features as inputs to generate a global lesion indication map and a highest-level lesion indication map. The prompting and guiding three-modal adaptive fusion module inputs the highest-level spatial features, the highest-level frequency features, and the highest-level lesion prompting map, and performs feature fusion to obtain high-level joint features; The prompt-frequency-aware decoder takes high-level joint features, multi-scale spatial features, multi-scale frequency features, and multi-scale lesion prompt map as inputs, and outputs lesion segmentation results; S3: Input the preprocessed lesion segmentation dataset into the spatial-frequency-cue co-segmentation network, and use dynamic combination segmentation loss and cue alignment loss to jointly optimize the network parameters to complete model training; S4: Collect OCTA image data of diabetic retinopathy to be segmented, input the trained spatial-frequency-cue co-segmentation network to output the lesion segmentation results.

[0022] As a refinement of the above embodiments, step S1 includes: Acquire OCTA image data of diabetic retinopathy and its corresponding lesion annotation data. The lesion annotation data includes annotations of non-perfusion areas, microvascular abnormalities, or other lesion areas related to diabetic retinopathy. Let the input OCTA image be... The actual lesion was marked as ,in , The input OCTA image is normalized in size and grayscale to obtain the preprocessed image. It is represented as follows: , in, Represents the original OCTA image. Indicates and The corresponding mask for the actual lesion, where H and W represent the image height and width, respectively. This represents the preprocessed OCTA image. This indicates a size normalization operation. This indicates a grayscale normalization operation. A mask is used to annotate lesions. Binarization or multi-class encoding is performed, ensuring strict spatial alignment with the original image. During training, data augmentation strategies such as random horizontal flipping, small-angle rotation, brightness and contrast adjustment, Gamma transformation, contrast-limited adaptive histogram equalization, and low-intensity noise perturbation can be employed. These augmentation operations are applied simultaneously to the original OCTA image and its corresponding lesion annotation mask, thereby improving the diversity of training samples and the model's generalization ability while maintaining the consistency of lesion structure.

[0023] As a refinement of the above embodiments, the spatial-frequency-cue collaborative segmentation network in step S2 includes: (1) Spatial encoder like Figure 3 As shown, the preprocessed OCTA image Input spatial encoder to obtain multi-scale spatial features , , , .

[0024] The spatial encoder comprises a shallow feature extraction module and multiple hierarchical encoding stages. The shallow feature extraction module extracts initial spatial features through successive convolution, normalization, and activation operations. The hierarchical encoding stages perform downsampling at each level and enhance feature representation through convolutional blocks, residual enhancement modules, and channel attention modules. The hierarchical encoding stages include four encoding phases: the first phase sequentially includes convolutional blocks, residual enhancement modules, and channel attention modules; the second, third, and fourth phases sequentially include convolutional blocks, residual enhancement modules, channel attention modules, and downsampling, respectively; the first encoding stage takes the initial spatial features as input and encodes them into multi-scale spatial features. The second, third, and fourth encoding stages respectively use the multi-scale spatial features output from the previous encoding stage as input, and encode them into multi-scale spatial features. Multi-scale spatial features Multi-scale spatial features Multi-scale spatial features This represents the highest-level spatial feature; specifically, it is expressed as follows: , , in, The shallow features representing spatial branches, i.e., the initial spatial features. This represents the shallow feature extraction module in the spatial encoder. Indicates the first Each spatial encoding stage Indicates the first Multi-scale spatial features This represents the initial spatial features input in the first spatial encoding stage. .

[0025] Shallow spatial features are used to characterize lesion edges, local textures, and fine structures, while high-level spatial features are used to characterize the overall morphology, semantic distribution, and contextual relationships of the lesion region. Through the output of spatial features at multiple scales, the spatial encoder can provide structured spatial representations for subsequent lesion prompt generation, trimodal fusion, and decoding recovery.

[0026] (2) Wavelet frequency encoder like Figure 4 As shown, the preprocessed OCTA image Input wavelet frequency encoder. The wavelet frequency encoder sequentially includes a wavelet decomposition module, a shallow frequency domain feature extraction module, and a frequency encoding stage.

[0027] The encoder first performs a two-dimensional discrete wavelet transform to decompose the input image into low-frequency subbands. and three high-frequency subbands , , Among them, the low-frequency sub-band The image primarily retains its overall brightness, coarse structure, and background distribution; the high-frequency subbands LH, HL, and HH represent edge and texture variations and local abrupt changes in different directions, respectively; as shown below: .

[0028] The low-frequency subband and high-frequency subband , , The features are concatenated along the channel dimension to form the frequency domain input features. Then, a shallow frequency domain feature extraction module performs preliminary encoding, mapping low-frequency structural information and high-frequency detail responses to the same feature space. Finally, multi-scale frequency features are extracted step-by-step through multiple frequency encoding stages. , , , Specifically: The frequency encoding stage comprises four frequency encoding stages. The first frequency encoding stage sequentially includes a convolutional block, a frequency enhancement module, and a channel attention module. The second, third, and fourth frequency encoding stages sequentially include a convolutional block, a frequency enhancement module, a channel attention module, and downsampling, respectively. The first frequency encoding stage takes the initially encoded frequency domain input features as input and encodes them into multi-scale frequency features. The second, third, and fourth frequency encoding stages respectively use the multi-scale frequency features output from the previous frequency encoding stage as input, and encode them into multi-scale frequency features. Multi-scale frequency characteristics Multi-scale frequency characteristics Multi-frequency spatial characteristics The highest-level frequency characteristics can be represented as follows: , , in, Represents a two-dimensional discrete wavelet transform; This indicates a channel splicing operation. This represents the shallow feature extraction module in the frequency domain. Indicates the first Each frequency coding stage Indicates the first Multi-scale frequency features.

[0029] The frequency encoding stage and the spatial encoding stage maintain a correspondence in scale organization, enabling spatial and frequency features to be aligned layer by layer in subsequent prompt generation, feature fusion, and decoding recovery processes. Through this design, the model can explicitly utilize low-frequency background structure and high-frequency boundary texture information to enhance its ability to identify low-contrast regions, regions with unclear boundaries, and fine-grained lesion patterns.

[0030] (3) Lesion indication generator Features of the highest level space and highest layer frequency characteristics Input a lesion hint generator to generate a global lesion hint representation and a top-level lesion hint map. The lesion hint generator includes a semantic anchoring branch and a spatial localization branch. The semantic anchoring branch is used to extract global lesion semantics from the high-level features of the two branches and adaptively update the learnable lesion query prototype. The spatial localization branch is used to generate a lesion hint map with spatial distribution significance by utilizing the similarity relationship between the updated hint query vector and local features.

[0031] In the semantic anchoring branch, respectively for and Perform global average pooling to obtain the spatial global context representation. and frequency global context representation After concatenating the two along the channel dimension, a joint global semantic vector is obtained through a multilayer perceptron or linear mapping network. This process can be represented as: , , in, Indicates the spatial characteristics of the highest level The obtained spatial global context representation, Indicates the frequency characteristics of the highest layer The obtained frequency global context representation, Indicates global average pooling. This indicates a channel splicing operation. This represents a multilayer perceptron or linear mapping network. This represents the joint global semantic vector. This joint global semantic vector contains both spatial structural semantics and frequency detail semantics, used to describe the overall semantic tendency of potential lesions in the current OCTA image.

[0032] Furthermore, a set of learnable lesion query vectors is set up. As a prototype for lesion indication, among which Indicates the number of query vectors. This indicates the embedding dimension of the prompt. It utilizes a joint global semantic vector. Generate gating coefficients Furthermore, by modulating the learnable lesion query vector through gating, a prompt query vector related to the current sample is obtained. It is represented as follows: , , in, This represents a learnable lesion query vector. Represented by the joint global semantic vector The generated gating coefficients, This represents the hint query vector after gating and updating. This represents the Sigmoid activation function. This represents element-wise multiplication. and These are learnable parameters. This design generates prompts through "global semantic modulation of learnable lesion prototypes," enabling the prompt representation to both retain shared lesion priors and adapt to different input samples.

[0033] In the spatial positioning branch, and splicing along the channel and through Convolution, normalization, and activation operations are mapped to a unified embedding space to obtain local feature representations. Then calculate the query vector. Representation of local features The similarity responses at various spatial locations yield multiple query response scores for each spatial location. To avoid limiting the scope of protection to a single normalization implementation, this invention summarizes the process as similarity response calculation: , , in, Indicates the spatial characteristics of the highest level and highest layer frequency characteristics The local feature representation obtained by mapping, Indicates the local features after flattening. Represents a local feature mapping function. Indicates the flattening operation. This represents the similarity calculation function. This indicates the similarity response between the query vector and the local spatial location.

[0034] In one implementation, the similarity calculation function It can be expressed in scaled dot product form as follows: , in, The transpose represents the flattened local features. This represents the scaling factor, used to avoid excessively large dot product values ​​when the embedding dimension is large, thereby improving training stability.

[0035] The initial spatial hint response is obtained by averaging the responses from multiple queries. Simultaneously, the updated mean of the query vector is input into the bias mapping module to generate a global bias term. This is done to adjust the overall response intensity of the cue map at a global level, and finally obtain the highest-level lesion cue map through an activation function. : , , , in, This indicates the mean aggregation operation. This represents the bias mapping function. This represents the global bias term generated by the suggested mean. This represents the operation of restoring the aggregated response to the shape of the spatial feature map. Indicates the temperature coefficient. This is a diagram illustrating the lesion at the highest layer. The diagram illustrates the lesion at the highest layer. This is used to represent the prior response that each spatial location in the input OCTA image belongs to the lesion region. Stepwise upsampling yields multi-scale lesion visualizations. , , This is for use by subsequent trimodal fusion and cue-frequency-aware decoders.

[0036] (4) Prompt and guide the three-modal adaptive fusion module Features of the highest level space Highest layer frequency characteristics Image showing the highest layer of lesions Input prompts guide the trimodal adaptive fusion module. This module is used to uniformly integrate spatial structural semantics, frequency detail response, and lesion prompt priors in a high-level semantic space.

[0037] First, using independent Convolutional, channel mapping layers, or channel expansion operations perform channel alignment on the three types of inputs, resulting in spatial alignment features, frequency alignment features, and cue alignment features with consistent channel dimensions. , in, This indicates the spatial characteristics after channel alignment. This indicates the frequency characteristics after channel alignment. This indicates the feature that indicates the channel alignment. , and These represent the channel alignment functions for spatial features, frequency features, and the suggestive map, respectively. When the lesion suggestive map... When it is a single-channel response map, it can be mapped to a channel dimension consistent with the spatial and frequency features by means of copying, convolutional mapping, or channel expansion.

[0038] Subsequently, the three types of features after channel alignment are concatenated and input into a lightweight weighted generation network to obtain the weights of spatial features, frequency features, and cue features at each spatial location; adaptive weights for the three modalities are then obtained through a softmax function. This process can be represented as: , Based on the adaptive weights, the three types of features are weighted and summed to obtain the high-level joint features. It is represented as: , in, This represents a weight generation network. represents the adaptive weights of spatial features, frequency features, and cue features, respectively, and Softmax represents the normalization function along the modality dimension. This represents the fused high-level joint features. These high-level joint features simultaneously include spatial structural semantics, frequency detail supplementation, and lesion indication priors, enabling them to provide a more lesion-sensitive high-level representation for subsequent decoding.

[0039] (5) Prompt - Frequency-aware decoder outputs lesion segmentation results High-level joint characteristics Multi-scale spatial feature sets Multi-scale frequency feature set Multiscale lesion atlas Input Prompt-Frequency Aware Decoder. The prompt-frequency awareness decoder includes a top-level prompt gating module and a multi-scale competitive decoding module, which are used to ensure that lesion prompt information runs through the decoding and recovery process, and to dynamically adjust the contribution of spatial structure information and frequency detail information at different scales.

[0040] In the initial decoding phase, the top-level prompting gating module utilizes the highest-level lesion prompting image. Characteristics of high-level joint operations Perform residual enhancement. Specifically, After normalization and gating mapping, a top-level gating graph is generated. Furthermore, it acts on the high-level joint features in the form of residuals, further enhancing the lesion-related features that indicate high-response regions, while preserving the original high-level semantic information: , , in, This is a diagram showing the lesion at the highest level. The generated top-level gating graph, Represents the gated mapping function. Indicates the strength enhancement coefficient. This indicates the initial decoding features after top-level cue gating enhancement. This indicates element-wise multiplication.

[0041] In the step-by-step decoding phase, the decoder performs multi-scale competitive decoding sequentially from the highest layer to the shallowest layer. For each scale... Multiscale lesion visualization based on the current scale By performing probability normalization and element-wise complementary transformation, we obtain the construct complementary hint graph. And calculate the suggestion confidence level. : , in, Indicates the first A lesion illustration at various scales, This represents a complementary hinting image to the aforementioned lesion hinting image. Indicates the first Confidence level of each scale, . Confidence level suggestion This is used to measure the clarity of a cue map at its current spatial location. Higher confidence is indicated by a cue response close to 0 or 1, while lower confidence is indicated by a cue response close to the median. , and Input gating network, generate spatial branch weights and frequency branch weights : , in, Indicates a gating network. Indicates the first Spatial branch weights at each scale Indicates the first The frequency branch weights at each scale are used to dynamically adjust the contributions of spatial and frequency features in the current scale decoding process. For regions with clear structural boundaries, the model can increase the contribution of spatial features; for regions with complex texture details, blurred boundaries, or low contrast, the model can increase the contribution of frequency features.

[0042] Based on the spatial branch weight and frequency branch weight, residual gating enhancement is performed on the spatial and frequency features at the corresponding scales to obtain the enhanced spatial features. and enhanced frequency characteristics It is represented as follows: , in, Indicates the first Multi-scale spatial features at various scales; Indicates the first Multi-scale frequency features at multiple scales; This indicates the spatial characteristics after gating enhancement; This indicates the frequency characteristics after gating enhancement.

[0043] The enhanced spatial and frequency features are spliced ​​together in the channel dimension, and then... Convolutional compression results in unified skip connection features; , in, express Convolution operation; Indicates the first Skip connection features at various scales.

[0044] The skip connection features are then concatenated with the current decoded features, and the feature representation at the current scale is recovered through the decoded block. This process can be represented as follows: , in, Indicates the first Decoding features at each scale; This indicates the decoding features of the previous layer; This represents the decoding features at the 5th scale; This indicates an upsampling operation or a scale alignment operation; Indicates the first Decoding blocks at each scale. When the highest-level features are consistent with the current decoding scale. Identity mapping or scale-aligned convolution can be used.

[0045] Finally, the lesion segmentation probability map is obtained by outputting the convolutional layer and activation function. It is represented as follows: , in, Indicates the final level of decoding features; This represents the final lesion segmentation probability map. The lesion segmentation probability map is generated based on a preset threshold. Binarization processing yields the lesion segmentation results of OCTA images of diabetic retinopathy.

[0046] As a refinement of the above embodiments, the total loss in step S3 includes: dynamic combination loss and cue alignment supervision, as detailed below: During model training, a dynamic combined segmentation loss and cue alignment loss are used to jointly optimize network parameters. The dynamic combined segmentation loss consists of Dice loss, Tversky loss, and FocalTversky loss. Dice loss is used to improve the region overlap between the predicted segmentation results and the ground truth annotations; Tversky loss is used to alleviate the class imbalance problem between the foreground and background of lesions; and FocalTversky loss is used to enhance the model's ability to learn difficult samples, low-contrast regions, and complex boundary regions.

[0047] The loss is represented as: , The loss is represented as: , The loss is represented as: , in, Indicates the first The predicted probability of each pixel. Indicates the first The true annotation of each pixel This indicates that the summation is performed on the pixels involved in the loss calculation. , and These represent true positives, false positives, and false negatives, respectively. and They represent Adjustment coefficients for false positives and false negatives in the loss. + =1, express The focusing parameter of the loss, This indicates a smoothing term to prevent the denominator from being zero. For Dice's loss, For Tversky's loss, Loss to FocalTversky.

[0048] The dynamic combination segmentation loss is expressed as: , in, Indicates training rounds, , and These represent the dynamic weights of Dice loss, Tversky loss, and FocalTversky loss during training. Each weight can be dynamically adjusted based on training epochs, changes in the validation set Dice or IoU metric, preset scheduling strategies, or combinations thereof. In the early stages of training, the weights of Dice loss and Tversky loss can be increased to prioritize the model's learning of the overall contour of the lesion region and the foreground / background distribution. As training progresses, the weight of FocalTversky loss can be gradually increased to allow the model to focus more on difficult-to-distinguish regions and complex boundary details.

[0049] To subject the lesion indication generator to explicit constraints, actual lesions are labeled. Downsampling to the highest layer lesion illustration The same spatial resolution is denoted as Furthermore, binary cross-entropy loss or other consistency loss methods are used to constrain the consistency between the two, resulting in the cue alignment loss. The cue alignment loss is expressed as: , in, This image shows a downsampling pattern up to the highest layer of the lesion. True lesion annotation with the same spatial resolution, express Middle The label value at each position, Image showing the lesion at the highest layer Middle The response value at each location, This indicates the number of pixels involved in the calculation of the cue alignment loss.

[0050] The final total loss is expressed as: , in, Indicates the total training loss; Represents the dynamic combination segmentation loss; This indicates a loss of alignment. This indicates the alignment loss weights. The value is set to 0.2. Through this joint optimization method, the main segmentation branch and the lesion indication branch can learn collaboratively, thereby enhancing the model's ability to locate and segment real lesion areas.

[0051] As a refinement of the above embodiments, step S4 includes: After model training is complete, the trained spatial-frequency-caution co-segmentation network parameters are saved. For the OCTA image to be segmented, size normalization and grayscale normalization are first performed in the same manner as during training, and then the trained lesion segmentation model is input. The model sequentially completes spatial feature extraction, frequency feature extraction, lesion cue generation, trimodal adaptive fusion, and cue-frequency perceptual decoding, finally outputting a lesion segmentation probability map. Binarizing the lesion segmentation probability map according to a preset threshold yields the lesion segmentation results for the diabetic retinopathy OCTA image.

[0052] Implementation effect verification To verify the effectiveness of the method of the present invention, in one embodiment, the lesion segmentation model was tested on the DRAC2022 dataset and the WFDR dataset, respectively, and Dice, IoU, Specificity, and Sensitivity were used as evaluation metrics. Dice and IoU are used to evaluate the degree of overlap between the predicted lesion region and the ground truth labeled region; Specificity is used to evaluate the model's ability to identify non-lesion background regions; and Sensitivity is used to evaluate the model's ability to detect ground truth lesion regions. The test results are only used to illustrate the technical effects of the method of the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0053] Table 1 presents the comparison results of the proposed method with several representative segmentation methods on the DRAC2022 dataset. As shown in Table 1, the proposed method achieves higher results in Dice, IoU, Specificity, and Sensitivity metrics, with Dice at 0.7658, IoU at 0.6210, Specificity at 0.9626, and Sensitivity at 0.8338. Compared with methods such as U-Net, AttenUNet, ResUnet, RetiFluidNet, OCT2Former, MsTGANet, and Swin-Unet, the proposed method can improve the detection capability and region overlap accuracy of lesions while maintaining high background recognition capability. This indicates that the collaborative modeling of spatial features, frequency features, and lesion indication information helps improve the lesion segmentation effect of OCTA.

[0054] Table 1. Comparison of the proposed method with different segmentation methods on the DRAC2022 dataset. Table 2 presents the comparison results of the proposed method with different segmentation methods on the WFDR dataset. As shown in Table 2, the proposed method also achieves superior performance on the WFDR dataset, with a Dice of 0.7764, IoU of 0.6352, Specificity of 0.9583, and Sensitivity of 0.8268. Compared to other comparative methods, the proposed method shows better performance in terms of lesion region overlap and detection capability, indicating that the spatial-frequency-cue co-segmentation network has good adaptability and robustness under different OCTA data sources.

[0055] Table 2 Comparison results of the proposed method and different segmentation methods on the WFDR dataset. like Figure 5 As shown, the present invention further provides visualization results of lesion segmentation. Figure 5 In this paper, the original OCTA image is used to display the background of the fundus vessels and lesions to be segmented, the ground truth annotation is used to represent the manually annotated lesion areas, the segmentation results of different comparison methods are used to demonstrate the recognition of lesion areas by each model, and the segmentation results of the method of this invention are used to demonstrate the output effect of the spatial-frequency-cue collaborative segmentation network. Figure 5 As can be seen, the method of the present invention can better maintain the overall contour of the lesion area, and has a relatively stable response to low-contrast, blurred boundary, and irregularly shaped areas, while reducing misclassification caused by background vascular texture. Combined with the quantitative results in Tables 1 and 2, it can be seen that the method of the present invention has good technical effects in terms of lesion area overlap, lesion detection capability, and background suppression capability.

[0056] Example 2 This disclosure also provides a lesion segmentation device for a spatial frequency cueing cooperative network, including a processor and a memory. Optionally, the device may further include a communication interface and a bus. The processor, communication interface, and memory can communicate with each other via the bus. The communication interface can be used for information transmission. The processor can call logical instructions in the memory to execute the lesion segmentation method for the spatial frequency cueing cooperative network described in the above embodiments.

[0057] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0058] Memory, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor executes the program instructions / modules stored in the memory to perform functional applications and data processing, thereby implementing the lesion segmentation method of the spatial frequency cueing cooperative network in the above embodiments.

[0059] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory may include high-speed random access memory and may also include non-volatile memory.

[0060] This disclosure provides a computer-readable storage medium storing computer-executable instructions configured to execute the aforementioned lesion segmentation method of the spatial frequency cueing cooperative network.

[0061] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0062] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media capable of storing program code. It can also be a transient storage medium.

[0063] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A lesion segmentation method using a spatial frequency cueing collaborative network, characterized in that, Includes the following steps: OCTA image data of diabetic retinopathy and its corresponding lesion annotation data were obtained to construct a lesion segmentation dataset, which was then preprocessed. A spatial-frequency-cue collaborative segmentation network is constructed, including a spatial encoder, a wavelet frequency encoder, a lesion cue generator, a cue-guided trimodal adaptive fusion module, and a cue-frequency sensing decoder; the spatial encoder and the wavelet frequency encoder are respectively input into the lesion segmentation dataset to extract multi-scale spatial features and multi-scale frequency features; The lesion indication generator takes the highest-level spatial features from the multi-scale spatial features and the highest-level frequency features from the multi-scale frequency features as input to generate a global lesion indication and a top-level lesion indication map; the indication-guided trimodal adaptive fusion module takes the highest-level spatial features, the highest-level frequency features, and the top-level lesion indication map as input to perform feature fusion to obtain high-level joint features; the indication-frequency-aware decoder takes the high-level joint features, multi-scale spatial features, multi-scale frequency features, and multi-scale lesion indication map as input to output the lesion segmentation result; The preprocessed lesion segmentation dataset is input into the spatial-frequency-cue co-segmentation network, and the network parameters are jointly optimized by dynamic combination segmentation loss and cue alignment loss to complete model training. Acquire OCTA image data of diabetic retinopathy to be segmented, input the trained spatial-frequency-cue co-segmentation network to output lesion segmentation results; The cue-frequency sensing decoder includes a top-level cue gating module and a multi-scale competitive decoding module; The top-level prompting gating module uses the highest-level lesion prompting map to perform residual enhancement on the high-level joint features to obtain the initial decoding features; The multi-scale competitive decoding module includes a four-stage decoding process, with inputs of initial decoding features and a multi-scale spatial feature set. Multi-scale frequency feature set Multiscale lesion atlas Multi-scale competitive decoding is performed sequentially from the highest layer to the shallowest layer. The decoding features of the first-level decoding stage are used to obtain the lesion segmentation probability map through the output convolutional layer and activation function. Each stage of the multi-scale competitive decoding module includes, in sequence: upsampling, competitive gating, dual-branch feature enhancement, skip feature fusion, and decoding block. The specific method for performing multi-scale competitive decoding sequentially from the highest layer to the shallowest layer is as follows: For each scale Multiscale lesion visualization based on the current scale By performing probability normalization and element-wise complementary transformation, we obtain the construct complementary hint graph. And calculate the confidence level of the prompt: ， in, Indicates the first A lesion illustration at various scales, Indicates the first Confidence level of each scale, ; Will , and Input gating network, generate spatial branch weights and frequency branch weights : ， in, Indicates a gating network. Indicates the first Spatial branch weights at each scale Indicates the first Frequency branch weights at each scale Indicates splicing; Based on the spatial branch weight and frequency branch weight, residual gating enhancement is performed on the spatial and frequency features at the corresponding scales to obtain the enhanced spatial features. and enhanced frequency characteristics : ， in, Indicates the first Multi-scale spatial features at various scales; Indicates the first Multi-scale frequency features at multiple scales; This indicates the spatial characteristics after gating enhancement; This indicates the frequency characteristics after gating enhancement; The gated enhanced spatial and frequency features are spliced ​​together in the channel dimension, and then... Convolutional compression results in unified skip connection features; The skip connection features are then concatenated with the current scale decoding features, and the feature representation at the current scale is recovered through the decoding block: ， in, Indicates the first Decoding features at each scale; This indicates the decoding features of the previous layer; This indicates the initial decoding characteristics after the top-level cue gating enhancement; Indicates the first Skip connectivity features at various scales; This indicates an upsampling operation or a scale alignment operation; Indicates the first Decoding blocks at various scales; when the highest-level features are consistent with the current decoding scale. Employ identity mapping or scale-aligned convolution. This indicates the initial decoding features after being enhanced by top-level cue gating.

2. The lesion segmentation method of spatial frequency cueing cooperative network according to claim 1, characterized in that, The spatial encoder includes a shallow feature extraction module and a hierarchical encoding stage; the shallow feature extraction module extracts initial spatial features through continuous convolution, normalization and activation operations; The hierarchical encoding stage comprises four encoding stages. The first encoding stage sequentially includes a convolutional block, a residual enhancement module, and a channel attention module. The second, third, and fourth encoding stages sequentially include a convolutional block, a residual enhancement module, a channel attention module, and downsampling, respectively. The first encoding stage takes the initial spatial features as input and encodes them into multi-scale spatial features. The second, third, and fourth encoding stages respectively use the multi-scale spatial features output from the previous encoding stage as input, and encode them into multi-scale spatial features. Multi-scale spatial features Multi-scale spatial features Multi-scale spatial features The highest level of spatial characteristics; The wavelet frequency encoder sequentially includes a wavelet decomposition module, a frequency domain shallow feature extraction module, and a frequency coding stage. OCTA image data undergoes two-dimensional discrete wavelet transform through the wavelet decomposition module, decomposing it into low-frequency sub-bands and high-frequency sub-bands. The low-frequency sub-bands and high-frequency sub-bands are spliced ​​along the channel dimension to form frequency domain input features. The frequency domain input features are initially encoded through the frequency domain shallow feature extraction module, and then multi-scale frequency features are extracted step by step through the frequency coding stage. The frequency encoding stage comprises four frequency encoding stages. The first frequency encoding stage sequentially includes a convolutional block, a frequency enhancement module, and a channel attention module. The second, third, and fourth frequency encoding stages sequentially include a convolutional block, a frequency enhancement module, a channel attention module, and downsampling, respectively. The first frequency encoding stage takes the initially encoded frequency domain input features as input and encodes them into multi-scale frequency features. The second, third, and fourth frequency encoding stages respectively use the multi-scale frequency features output from the previous frequency encoding stage as input, and encode them into multi-scale frequency features. Multi-scale frequency characteristics Multi-scale frequency characteristics Multi-frequency spatial characteristics This represents the highest-level frequency characteristics.

3. The lesion segmentation method of spatial frequency cueing cooperative network according to claim 1, characterized in that, The lesion indication generator includes a semantic anchoring branch and a spatial positioning branch; The semantic anchoring branch concatenates the highest-level spatial features and the highest-level frequency features after global average pooling, and then obtains a joint global semantic vector through a multilayer perceptron. The joint global semantic vector is then used to perform gating updates on the learnable lesion query vector to obtain the prompt query vector. The spatial positioning branch concatenates the highest-level spatial features and the highest-level frequency features, and then sequentially passes through... Convolution, normalization, and activation operations are mapped to a unified embedding space to obtain local feature representations. The similarity response between the query vector and the local feature representation is calculated, and the mean of the similarity responses is aggregated to generate an initial spatial cue response. The aggregated mean of the query vectors is then input into a bias mapping module to generate a global bias term. Finally, the global bias term and the initial spatial cue response are passed through an activation function to obtain the highest-level lesion cue map. The highest-level lesion visualization image is upsampled level by level to obtain multi-scale lesion visualization images. Multiscale lesion visualization Multiscale lesion visualization .

4. The lesion segmentation method of spatial frequency cueing cooperative network according to claim 1, characterized in that, The prompt guides the three-modal adaptive fusion module to perform feature fusion to obtain high-level joint features, as follows: The spatial features, frequency features, and lesion indication map of the highest layer were analyzed respectively through... Channel alignment is performed using convolution, channel mapping layers, or channel expansion operations. The highest-level spatial features, highest-level frequency features, and highest-level lesion indication map after channel alignment are stitched together and input into a weight generation network. The adaptive weights for the three modes are obtained through the Softmax function. Based on adaptive weights, the highest-level spatial features, highest-level frequency features, and highest-level lesion indication map are weighted and summed to obtain the high-level joint features.

5. The lesion segmentation method of spatial frequency cueing cooperative network according to claim 1, characterized in that, The total loss of the spatial-frequency-cue collaborative segmentation network is as follows: ， ， ， in, Indicates the total training loss; Represents the dynamic combination segmentation loss; This indicates a loss of alignment. This indicates the alignment loss weights; express Middle The label value at each position, This image shows a downsampling pattern up to the highest layer of the lesion. True lesion annotation with the same spatial resolution; Image showing the lesion at the highest layer Middle The response value at each location; Indicates the number of pixels involved in the calculation of the cue alignment loss; Indicates training rounds, This represents the dynamic weights of the Dice loss during training. This represents the dynamic weights of the Tversky loss during training. This represents the dynamic weights of the FocalTversky loss during training; The Dice loss is used to improve the overlap between the predicted segmentation results and the ground truth labels. Tversky loss is used to mitigate the class imbalance between the foreground and background of lesions. FocalTversky loss is used to enhance the model's ability to learn from difficult samples, low-contrast regions, and complex boundary regions.

6. The lesion segmentation method of spatial frequency cueing cooperative network according to claim 1, characterized in that, The preprocessing includes size normalization, grayscale normalization, image enhancement, and annotation mask alignment of the OCTA image data; The image enhancement includes random flipping and / or random rotation and / or brightness and contrast adjustment and / or Gamma transformation and / or limiting contrast adaptive histogram equalization and / or low-intensity noise perturbation, and the image enhancement is applied synchronously to the original OCTA image data and the corresponding lesion annotation mask.

7. A lesion segmentation device for a spatial frequency cueing cooperative network, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to execute the lesion segmentation method of the spatial frequency cueing cooperative network as described in any one of claims 1-6 when running the program instructions.

8. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the lesion segmentation method of the spatial frequency cueing cooperative network as described in any one of claims 1-6 above.

Images