An image intelligent segmentation method based on frequency domain-semantic alignment

Abstract

The application provides an image intelligent segmentation method based on frequency domain-semantic alignment, and belongs to the technical field of medical image processing. The method acquires an OCTA image and pre-processes the same, extracts visual features through a multi-scale visual coding network; extracts clinical text semantic features by using a CLIP encoder, constructs a sparse alignment cross-modal attention module and a significant position consistency loss, and realizes cross-modal semantic enhancement; performs frequency domain decomposition and lesion enhancement by using a discrete wavelet transform, and fuses the cross-modal and frequency domain features by using a hierarchical frequency guide modulation module to obtain collaborative features; finally, the decoder based on the U-Net architecture is input, the resolution is restored by layer-by-layer up-sampling and the introduction of a spatial attention module, and the segmentation result is output. The application effectively realizes the automatic identification and accurate segmentation of the lesion area in the complex OCTA image.

Description

Technical Field

[0001] This invention relates to an intelligent image segmentation method based on frequency domain-semantic alignment, belonging to the field of medical image processing technology. Background Technology

[0002] Diabetic retinopathy is a common microvascular complication of the eye caused by diabetes and a significant cause of vision impairment and even blindness in adults. With the continued increase in the number of diabetic patients, early screening and auxiliary diagnosis of diabetic retinopathy are gradually becoming an important direction in ophthalmological clinical research.

[0003] Optical coherence tomography (OCTA) is a non-invasive retinal blood flow imaging technique that provides high-resolution visualization of retinal capillary structures, blood flow distribution, and lesion areas, making it valuable in the auxiliary diagnosis of diabetic retinopathy. By automatically segmenting lesion areas in OCTA images, doctors can analyze pathological features such as microaneurysms, non-perfused capillary areas, and abnormal vascular proliferation, improving the efficiency of lesion screening and the accuracy of auxiliary diagnosis.

[0004] Currently, most research on lesion segmentation in OCTA images employs deep learning-based medical image processing methods. These methods extract lesion region features by constructing convolutional or attention networks to achieve automatic lesion identification and segmentation. However, in complex clinical scenarios, existing technologies still face the following challenges: (1) The lesion areas in OCTA images are usually small in size, have blurred boundaries, irregular shapes and low local contrast. Existing methods mainly model lesions based on spatial domain features, which makes it difficult to effectively preserve fine-grained edge information and local texture structure. (2) Most existing methods rely on single visual features for lesion identification, lacking effective use of clinical semantic information, resulting in insufficient ability to distinguish complex lesion areas from background areas; (3) There are significant differences in the response of different lesion regions to high-frequency texture information and low-frequency structural information. Existing methods have limited ability to model frequency domain features, which can easily lead to the loss of lesion edge details and the missed detection of small lesions. (4) Some cross-modal methods use a fully connected attention mechanism to establish the relationship between visual features and text features. In the feature interaction process, low-relevance matching information is easily introduced, which affects the cross-modal feature alignment effect. (5) Existing cross-modal fusion methods lack constraints on the consistency of lesion spatial location, which can easily lead to problems such as lesion response area shift and background area missegmentation.

[0005] Therefore, how to construct a lesion segmentation method that can synergistically integrate visual features, frequency domain structural information, and clinical semantic information to improve the lesion region recognition and segmentation accuracy in complex OCTA images has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent image segmentation method based on frequency domain-semantic alignment, which can automatically identify and accurately segment lesion regions in complex OCTA images, thereby improving the segmentation accuracy of small lesions, lesions with blurred boundaries, and lesions with low contrast.

[0007] To achieve the above objectives, the present invention employs the following technical solution: An intelligent image segmentation method based on frequency domain-semantic alignment includes the following steps: Acquire OCTA image data of diabetic retinopathy and preprocess the raw images; A multi-scale lesion perception visual coding network is constructed to extract features from the input OCTA image step by step, thereby obtaining multi-scale visual feature representations. Acquire clinical description text related to lesions, and use a pre-trained CLIP text encoder to perform semantic embedding encoding on the clinical description text to obtain semantic feature representation of lesions; A sparsely aligned cross-modal attention module is constructed to perform cross-modal association and semantic enhancement on the multi-scale visual feature representation and the lesion semantic feature representation; and a consistency constraint is applied to the cross-modal attention space response through salient position consistency loss to obtain the cross-modal enhanced representation. A frequency-domain guided lesion perception branch was constructed. Discrete wavelet transform was used to decompose OCTA image data in the frequency domain, and lesion regions in different frequency bands were enhanced to obtain frequency domain enhancement features. A hierarchical frequency-guided modulation module is constructed to jointly modulate and fuse cross-modal enhancement representations and frequency domain enhancement features to obtain multimodal collaborative feature representations; The multimodal collaborative feature representation is input into the lesion segmentation decoder; the lesion segmentation decoder is built on the U-Net decoder architecture, performs upsampling operation layer by layer, and introduces a spatial attention module to enhance the features after each layer completes upsampling, restores the image resolution layer by layer, and outputs the lesion segmentation result.

[0008] Preferably, the multi-scale lesion perception visual coding network includes a multi-scale lesion perception residual encoder and a lightweight spatial attention mechanism; the multi-scale lesion perception residual encoder adopts a four-stage progressive downsampling structure, and a lightweight spatial attention mechanism is introduced in each stage of downsampling. The downsampling structure of the multi-scale lesion perception residual encoder at each stage consists of a residual convolution module and a pooling operation. A three-branch dilated convolution structure is introduced into the residual convolution module to extract multi-scale context information in parallel. The pooling operation includes average pooling and global pooling. The input feature map is processed by multi-branch dilated convolution to extract multi-scale contextual information. The sum of the multi-scale contextual information is then added to the elements of the input feature map to obtain the final feature map. Feature map Spatial attention is obtained by performing average pooling and global pooling operations followed by convolution. This spatial attention is then used to apply spatial attention to the feature map. The downsampled feature map is obtained through enhancement; the downsampled feature map is then input into the next stage downsampled structure, and a multi-scale visual feature representation is obtained through a four-stage progressive downsampled structure.

[0009] Preferably, the sparse-aligned cross-modal attention module is as follows: The multi-scale visual feature representation and the semantic feature representation of the lesion are projected onto a shared space to obtain a text token and a visual token. Construct a bidirectional similarity matrix between text tokens and visual tokens. Based on the bidirectional similarity matrix, introduce structured sparse constraints and retain only the top-k associations with the highest similarity for each token to obtain a bidirectional sparse mask. Bidirectional sparse attention weights are calculated using the softmax function based on bidirectional sparse masks. The multi-scale visual feature representation and the lesion semantic feature representation are linearly transformed respectively, and then enhanced by the corresponding bidirectional sparse attention weights across modal features to obtain cross-modal enhanced representations, including cross-modal enhanced representations of text features and cross-modal enhanced representations of visual features.

[0010] Preferably, the salient location consistency loss defines text location saliency by calculating the variance of text-to-visual attention in the visual dimension, and constructs a sparse salient location indicator based on this. The consistency of cross-modal attention in salient spatial locations is constrained by maximizing the cosine similarity between the salient location indicator and the bidirectional attention statistic; specifically as follows: , , , , , in, For salient position consistency loss, To obtain the expected value for the batch sample, It is a sparse position indicator vector. For L2 normalization, For indicator functions, For text position salience, To select The largest value One element, For along the visual dimension Calculate the mean. For text-to-visual attention matrix, For the real number field, This is a statistical representation of visual-text attention mapped to the text semantic space. For learnable projection matrices, For visual-to-text attention matrix, This represents the average response strength of the text token in the visual space. For along the visual dimension Calculate the variance. The average response strength of the visual token to the text space. The batch size of the input samples for a single training session. The total number of visual tokens output after encoding the OCTA image.

[0011] Preferably, the frequency domain-guided lesion perception branch includes a frequency domain encoder, a clinically heuristic dual attention unit, and a dual-path cross-scale-frequency band fusion module; the clinically heuristic dual attention unit includes a lesion perception multi-scale filtering module and a diagnostic context adapter; The frequency domain encoder is based on two-level Haar discrete wavelet transform, which decomposes the input image into original sub-band features, including a low-frequency approximate sub-band feature and two sets of high-frequency sub-band features, and projects all sub-band features into a unified embedding space through convolution. The lesion sensing multi-scale filtering module consists of three parallel lightweight channel decoupling filtering paths with different dilation rates. Each lightweight channel decoupling filtering path includes depthwise convolution and pointwise convolution in sequence; specifically represented as follows: , , , in, For the first Output characteristics of a lightweight channel decoupling filter path For GELU activation function, Indicates BN layer, For pointwise convolution, For depthwise convolution, For the input sub-band features, The lesion sensing multi-scale filtering module outputs features for each sub-band. For sub-band set, For 1×1 convolutional fusion weights, The output characteristics of the first lightweight channel decoupling filter path, The output characteristics of the second lightweight channel decoupling filter path, The output characteristics of the third lightweight channel decoupling filter path. For channel-level splicing, This is a second-order wavelet low-frequency approximation subband. It is a secondary high-frequency sub-band. It is a second-level vertical high-frequency sub-band. It is a second-order diagonal high-frequency sub-band. It is a first-level horizontal high-frequency sub-band. It is a first-level vertical high-frequency sub-band. It is a first-level diagonal high-frequency sub-band; The diagnostic context adapter performs global context calibration on the features of each sub-band through global semantic aggregation and channel-sensitive recalibration; specifically, it includes: Will A global semantic vector is obtained through global semantic aggregation. Channel-sensitive recalibration is then performed on this global semantic vector to generate importance weights for each channel. These importance weights are then compared with... Multiply by each channel to obtain the features Combine the original subband features with , The sum is used as a feature output by the diagnostic context adapter.

[0012] Preferably, the dual-path cross-scale-frequency band fusion module performs bidirectional gated fusion of high-frequency subbands and low-frequency subbands at all levels in the output features of the diagnostic context adapter, and outputs the lesion sensing frequency embedding after channel recalibration as a frequency domain enhancement feature; By explicitly integrating bidirectional cross-scale interaction and lesion-sensitive channel recalibration, a unified lesion-sensing frequency embedding is generated; specifically including: The second-level high-frequency subband set from the diagnostic context adapter output features is concatenated and then subjected to global pooling and convolution operations. The weights of the lesion regions are calculated using the Sigmoid activation function to obtain the clinical semantic prior weights. These clinical semantic prior weights are then used to perform a weighted summation of the second-level high-frequency subbands to obtain the weighted subband features. ; The feature is obtained by concatenating the first-level high-frequency subband set from the diagnostic context adapter output features and then downsampling it. ,feature The first-level gating weights are obtained by concatenating the weighted subband features with the sigmoid activation function. Based on the first-level gating weights for features and weighted subband features Weighted fusion yields the feature map after the first level of fusion. : , The feature map after first-level fusion Second-level low-frequency subband in the diagnostic context adapter output characteristics The concatenated weights are then processed using the Sigmoid activation function to obtain the second-level gating weights. The feature map after first-level fusion is based on the second-level gating weights. The second-level fused feature map is obtained by weighted fusion of the second-level low-frequency subband. : ; For the fused feature map and Perform lesion-sensitive channel recalibration: , , , in, This indicates splicing along the channel dimension. express and Feature map after channel-level stitching Indicates channel weight, This represents element-wise multiplication. This represents the Sigmoid activation function. Represents convolution. This indicates adaptive global average pooling; The calibrated features are used to generate lesion-sensing frequency embeddings through convolution operations, which serve as frequency domain enhancement features.

[0013] Preferably, the hierarchical frequency-guided modulation module performs adaptive modulation and consistency fusion of cross-modal features through dual guidance of semantic focusing and dynamic frequency filtering; specifically including: Cross-modal augmentation representation of text features Mapped to a semantic focus graph and style modulation diagram The lesion sensing frequency is embedded and mapped as a dynamic filtering kernel. ; Visual features are obtained by performing semantic and frequency dual modulation on the cross-modal enhanced representation of visual features. The details are as follows: , , in, These are semantically enhanced visual features obtained after channel and spatial weighting of the text semantics; visual features The features are concatenated with the lesion perception frequency embedding channel dimension and cross-domain feature integration is achieved through fusion convolution. The original visual semantic structure and gradient flow stability are preserved through residual path to obtain multimodal collaborative feature representation.

[0014] Preferably, the cross-modal enhanced representation of visual features is sequentially processed by 3×3 convolution, GELU activation function, and 1×1 convolution, and then Tanh activation is used to generate a semantic style modulation map; The cross-modal enhanced representation of visual features is sequentially processed by 3×3 convolution, GELU activation function, and 1×1 convolution, and then a semantic focusing map is generated using Sigmoid activation. The lesion sensing frequency is embedded and mapped sequentially through 3×3 convolution, GELU activation function, and 1×1 convolution to obtain a dynamic filtering kernel.

[0015] Preferably, the OCTA image data of diabetic retinopathy uses the DRAC2022 challenge dataset and the WFDR wide field-of-view OCTA dataset, and is constructed and trained based on the PyTorch deep learning framework.

[0016] An image intelligent segmentation device based on frequency domain-semantic alignment includes a processor and a memory storing program instructions, wherein the processor is configured to execute the image intelligent segmentation method based on frequency domain-semantic alignment when running the program instructions.

[0017] The advantages of this invention are as follows: By introducing a cross-modal association mechanism between clinical semantic features and visual features, this invention achieves semantic enhancement of lesion regions, reduces background interference, and improves the recognition ability of complex lesion regions; through frequency domain decomposition and lesion perception enhancement processing, it achieves collaborative modeling of high-frequency texture information and low-frequency structural information, reducing the loss of lesion edge details and the problem of missed detection of small lesions; it adopts a sparse cross-modal association method to suppress low-correlation feature matching noise and improve the cross-modal feature alignment effect; it restricts the response position of lesion regions through significant position consistency loss, reducing lesion region offset and background missegmentation problems; and it achieves feature enhancement of complex lesion regions and improves lesion segmentation accuracy through a joint fusion of semantic modulation and frequency dynamic filtering. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0019] Figure 1 This is a structural diagram of the image intelligent segmentation method based on frequency domain-semantic alignment proposed in this invention.

[0020] Figure 2 This is a schematic diagram of the multi-scale lesion sensing residual encoder proposed in this invention.

[0021] Figure 3 Precision-recall curves were plotted on the DRAC2022 dataset for this invention.

[0022] Figure 4 Precision-recall curves were plotted on the WFDR dataset for this invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] like Figure 1 As shown, an intelligent image segmentation method based on frequency domain-semantic alignment includes the following steps: S1: Acquire OCTA image data of diabetic retinopathy and preprocess the raw images; S2: Construct a multi-scale lesion perception visual coding network to extract features from the input OCTA image step by step and obtain multi-scale visual feature representations; S3: Obtain the clinical description text related to the lesion, and use the pre-trained CLIP text encoder to perform semantic embedding encoding on the clinical description text to obtain the semantic feature representation of the lesion; S4: Construct a sparsely aligned cross-modal attention module to perform cross-modal association and semantic enhancement on the multi-scale visual feature representation and the lesion semantic feature representation; and through salient position consistency loss, impose consistency constraints on the cross-modal attention space response to obtain the cross-modal enhanced representation; S5: Construct a frequency domain-guided lesion perception branch, use discrete wavelet transform to perform frequency domain decomposition on OCTA image data, and enhance lesion regions in different frequency bands to obtain frequency domain enhancement features; S6: Construct a hierarchical frequency-guided modulation module to jointly modulate and fuse cross-modal enhancement representations and frequency domain enhancement features to obtain multimodal collaborative feature representations; S7: Input the multimodal collaborative feature representation into the lesion segmentation decoder; the lesion segmentation decoder is built on the U-Net decoder architecture, performs upsampling operation layer by layer, and introduces a spatial attention module to enhance the features after each layer completes upsampling, restores the image resolution layer by layer, and outputs the lesion segmentation result.

[0025] As a refinement of the above embodiments, step S1 preprocessing includes image size unification, pixel normalization, random flipping, random rotation, random cropping, brightness adjustment, and contrast adjustment; and dividing the dataset into training set, validation set, and test set according to a preset ratio.

[0026] This embodiment uses the publicly available DRAC2022 challenge dataset and WFDR wide-field OCTA dataset as experimental data sources. The DRAC2022 dataset was acquired using a VG200D sweep frequency source OCTA system with a center wavelength of 1050 nm and a scan rate of 200,000 A-scans / s. This embodiment selected 106 images containing lesions with non-perfusion regions for the experiment. The WFDR wide-field OCTA dataset contains wide-field OCTA clinical imaging data from 288 diabetic patients; all lesion regions were manually labeled by professional ophthalmologists. Note that in this embodiment, 80 images with high imaging quality and containing obvious lesion areas were selected for experimental analysis. To ensure the objectivity of model training and evaluation, both datasets were divided into training, validation, and test sets in an 8:1:1 ratio. Furthermore, to enhance the robustness of the model to changes in lesion morphology and imaging differences, the input images were subjected to data augmentation operations such as random flipping, rotation, cropping, scaling, brightness and contrast adjustment, and affine transformation. The pixel intensity of all images was linearly normalized from [0,255] to the [0,1] interval to improve the model training stability and convergence efficiency.

[0027] This invention's model is built and trained using the PyTorch deep learning framework. The model training process is completed in a workstation environment equipped with an NVIDIA GeForce RTX A4000 GPU with 16GB of video memory. This embodiment uses the AdamW optimizer to update the model parameters. By decoupling the weight decay and gradient optimization processes, the risk of overfitting in the complex high-dimensional parameter space is reduced, thereby improving the model's training stability and generalization ability. The initial learning rate of the model is set as follows: Furthermore, a cosine annealing learning rate scheduling strategy is used to dynamically adjust the learning rate during the training process in order to improve the model's convergence stability.

[0028] Furthermore, in this embodiment, the training batch size is set to 4 to balance memory usage and training efficiency.

[0029] In addition, to prevent overfitting during training, this invention introduces an early stopping strategy based on changes in validation set loss to dynamically monitor the training process.

[0030] When the validation set loss no longer decreases over multiple consecutive training epochs, the model training is automatically terminated, thereby obtaining model parameters with better generalization performance.

[0031] The input images are uniformly adjusted to H×W size, and the pixel values ​​are mapped to the [0,1] interval using a normalization method to reduce the data distribution differences caused by different device acquisition conditions.

[0032] As a refinement of the above embodiments, the multi-scale lesion perception visual coding network in step S2 is as follows: Figure 2 As shown, it includes a multi-scale lesion perception residual encoder and a lightweight spatial attention mechanism.

[0033] In terms of feature-level modeling, the multi-scale lesion-aware residual encoder constructs a multi-scale representation of the input image through a four-stage progressive downsampling structure. Each stage consists of a residual convolution module and pooling operations to achieve multi-level feature representation. To enhance the model's ability to model lesions at different scales, a multi-branch dilated convolution structure is introduced into the traditional residual block. Multi-scale contextual information is extracted in parallel through different dilation rates, thereby effectively capturing small lesions and large ischemic areas. Furthermore, a lightweight spatial attention mechanism is introduced after the output of each stage to adaptively enhance potential lesion regions and improve the discriminativeness of feature representation.

[0034] In the In this stage, the feature map is represented as , For the number of channels, For feature map batches, For the feature map channel dimension, For feature map height, The feature map width is defined as follows. The spatial resolution is compressed proportionally with each layer, while the [missing information] is [missing information]. Stage Feature Map Channel Dimension Gradually expand (e.g.) , (Initial number of channels). This collaborative change allows the model to continuously improve the semantic expressiveness of features while reducing spatial redundancy. This hierarchical structure allows shallow features to focus on characterizing fine-grained texture and edge information, while deep features gradually aggregate global context and structural semantics, thus forming a progressive representation from local to global. In the highest-level semantic space, the highest-level semantic features... The sequence is rearranged into a sequence form and projected onto a low-dimensional embedding space via a linear mapping to obtain... The final visual token represents, among which Embedded Dimensions Set to 32. This unified serialization representation provides a structurally consistent feature foundation for subsequent cross-modal semantic alignment. The specific representation is as follows: , , , , , , in, Indicates the input image. Indicates the first Feature map after stage residual encoding; This represents the feature map after spatial attention enhancement; Indicates the first Feature map after staged pooling (used to enter the next layer); This represents the highest-level semantic features (before expansion). This represents the final visual token. Indicates the expansion rate 3×3 convolution; Represents the set of expansion rates, used for multi-scale modeling; This represents the Sigmoid activation function; This represents the convolution operation used to generate spatial attention; This indicates average pooling (along the channel dimension). This indicates max pooling (along the channel dimension). Indicates a downsampling operation; This indicates that the two-dimensional feature map is flattened into a sequence.

[0035] As a refinement of the above embodiments, the acquisition of lesion-related clinical description text in step S3 is divided into two approaches: offline training batch generation and real-time generation at the clinical terminal. In the offline stage, the relative offset between the centroid of each lesion and the image is automatically calculated based on the lesion segmentation mask, and eight fundus anatomical regions are divided. Combined with standardized lesion morphology vocabulary, structured text is generated in batches according to a fixed template, and semantic accuracy is ensured through a two-stage quality control process of rule verification and manual random sampling. In the online inference stage, the basic text is automatically generated by the device pre-detection module, and then used after fine-tuning by the physician.

[0036] The clinical description text is a standardized structured prompt, which uniformly includes two parts: first, a description of the lesion morphology, recording the lesion type, size, and texture characteristics; second, a description of the spatial location, marking the eight anatomical regions of the fundus to which the lesion belongs. The text only objectively describes the lesion in the image and does not contain irrelevant information such as patient privacy or systemic medical history.

[0037] The pre-trained CLIP text encoder maps lesion-related clinical description text into compact semantic embeddings, which are used as conditional information for visual feature modulation and cross-modal alignment.

[0038] Specifically, the input text is first segmented and serialized using the CLIP tokenizer, and then uniformly truncated / padded to a maximum length of 77 to ensure semantic integrity. Subsequently, the token sequence is input into the CLIP text encoder (based on a Transformer structure) for contextual semantic modeling, outputting 768-dimensional text features. To reduce computational complexity and improve cross-modal fusion efficiency, a linear projection layer is introduced to map it to a 32-dimensional low-dimensional embedding space.

[0039] As a refinement of the above embodiments, the sparse alignment cross-modal attention module in step S4 is used to suppress cross-modal redundant associations and highlight semantic-visual effective pairings related to lesions.

[0040] This module first projects multi-scale visual feature representations and lesion semantic feature representations into a shared latent space, unifying the representation scale and eliminating modal differences. Then, it constructs a bidirectional similarity matrix between text tokens and visual tokens to quantify semantic relevance. Unlike fully connected attention, this module introduces structured sparsity constraints, retaining only the top-k associations with the highest similarity for each token, suppressing weakly correlated noise matching, and enabling the model to focus on lesion localization and structural discrimination regions. Based on this, symmetrical sparse modeling is performed on text-to-visual and visual-to-text attention, and normalization is achieved through a softmax function with a temperature coefficient. Bidirectional feature aggregation is achieved using sparse attention maps to obtain semantically consistent text and visual representations. This module improves semantic alignment stability and discriminativeness through sparse constraints on high-confidence cross-modal matching pairs, reducing the interference of visually significant but clinically irrelevant structures on lesion segmentation.

[0041] Specifically, it includes the following key steps: (1) Given the text feature tensor (lesion semantic feature representation) Visual feature tensors (multi-scale visual feature representations) Define a hierarchical orthogonal projection operator to project the multi-scale visual feature representation and the lesion semantic feature representation onto a shared space to obtain a text token and a visual token: Semantic query projection system: , , in, This is a mapping from text features to queries. This is a mapping from visual features to queries. , This is a hierarchical weight matrix used to map text features to the query space. Indicates the dimension of hidden layer features; is a learnable parameter matrix used to adjust the mapped query vector; GeLU is the GeLU activation function used to introduce non-linearity; LayerNorm is layer normalization used to stabilize the training process; Representing text features to query vector The mapping function; Representing text features Learnable bias terms for query branches; Representing visual features to query vector The mapping function; Representing visual features Learnable bias terms for query branches.

[0042] Structural key-value projection system: , , in, A mapping from visual features to keys; Mapping text features to keys; This indicates that the first layer of the visual branch can learn the projection matrix, completing the initial dimensional transformation of visual features. This indicates that the second layer of the visual branch can learn the projection matrix, and the normalized features are mapped again to obtain the intermediate representation of the visual key vector. The visual branch is used to scale the key vector. The learnable scaling factor, This represents the learnable bias parameters used by the visual branch to offset the key vector. This indicates that the first layer of the text branch is a learnable projection matrix, which completes the initial dimensional transformation of the text features. The text branch represents the second layer of learnable projection matrix, which maps the normalized features again to obtain the intermediate representation of the text key vector. The text branch is used to scale the key vector. The learnable scaling factor; This represents the learnable bias parameters used by the text branch to offset and adjust the key vector. This represents a text-key mapping function. This represents the visual-key mapping function.

[0043] (2) Construct a bidirectional similarity matrix between text tokens and visual tokens: , , in, The text-to-visual similarity matrix has the following shape: ; This is a visual-to-text similarity matrix with shape [value]. ; The number of encoded text tokens; This represents the total number of visual tokens. This indicates the matrix transpose.

[0044] (3) Based on the bidirectional similarity matrix, a structured sparsity constraint is introduced, and only the top-k associations with the highest similarity are retained for each token to obtain the bidirectional sparse mask: , , in, The function is defined as: , in, For a function, the similarity matrix Apply a Top-k sparse mask to retain the Top-k maximum value of each element and set the rest to 0; This is a hyperparameter that specifies the maximum number of connections to retain for each element; A sparse mask for text-to-visual representation, with shape [shape not specified]. ; A sparse mask for visual text, with shape as ; Representing the similarity matrix The Middle Individual source token and the first The similarity scores for each target token. Represents the similarity matrix of the first... All elements of the row, i.e., the first row The set of similarities between one token and all tokens on the other side. This indicates a sorting and selection operation, which sorts all similarity values ​​in a row from highest to lowest and selects the first value with the highest similarity score. One element, This indicates the first element in the generated sparse mask matrix. Line number Column elements.

[0045] (4) Calculate the bidirectional sparse attention weights using the softmax function based on the bidirectional sparse mask: Calculate sparse attention weights using a temperature-regulated softmax function: Text-to-visual attention: , Visual to text attention: , in, For Hadamard product (element-by-element multiplication); The sharpness of the softmax parameter is controlled by a learnable temperature parameter. The sparse attention weights from text to vision are of shape [shape]. ; The sparse attention weights from visual to text are of shape [shape missing]. .

[0046] (5) The multi-scale visual feature representation and the lesion semantic feature representation are linearly transformed respectively, and then enhanced by the corresponding bidirectional sparse attention weights across modal features to obtain the cross-modal enhanced representation, including the cross-modal enhanced representation of text features and the cross-modal enhanced representation of visual features: Value projection: , , in, and Linear transformation for feature enhancement; and These are learnable weight matrices, used for textual and visual features respectively; and It is a learnable bias vector; For enhanced representation of text features, the shape is ; For enhanced representation of visual features, the shape is .

[0047] Cross-modal feature enhancement: , , in, For cross-modal augmented representation of text features, the shape is ; For cross-modal enhanced representation of visual features, the shape is .

[0048] As a refinement of the above embodiments, the salient location consistency loss in step S4 does not rely on explicit region labeling or pixel-level supervision. Instead, it implicitly mines text tokens highly correlated with lesion spatial semantics by modeling the statistical characteristics of cross-modal attention distribution. Specifically, the variance of text-to-visual attention in the visual dimension is used as a measure of spatial discriminability to filter out texts with strong spatial selectivity, and a sparse salient location indicator is constructed accordingly. Based on this indicator, the salient location consistency loss explicitly constrains the consistency of bidirectional attention at salient spatial locations by maximizing its cosine similarity with the statistics of text-to-visual and visual-to-text attention while suppressing insignificant responses. This consistency constraint effectively promotes the accurate alignment of cross-modal semantics with lesion spatial locations and improves the stability and robustness of cross-modal interactions during training.

[0049] Specifically, it includes: (1) Automatic discovery of semantic salience: Based on the variance of attention distribution of text tokens in the visual dimension, the salience of text position is defined. : , in, For along the visual dimension Calculate the variance. For text-visual and sparse attention matrices.

[0050] (2) Before selection Construct a sparse location indicator vector from several significant tokens: , in, Indicates an indicator function, for Normalization; To select The largest value One element, This represents the proportional hyperparameter, with values ​​greater than 0 and less than 1. This indicates the total number of text tokens.

[0051] (3) Statistical consistency of bidirectional attention: Calculate the mean representation of bidirectional attention in the spatial dimension. , , in, For along the visual dimension Calculate the mean; This represents the average response strength of the text token in the visual space. This represents the average response strength of the visual token to the text space.

[0052] (4) Visual-to-text attention is projected into the text semantic space through learnable mapping: , in, For learnable projection matrix; This is a statistical representation of visual-text attention mapped to the text semantic space.

[0053] (5) Unified Loss Function: The salient position consistency loss is defined jointly by maximizing salient position consistency and suppressing insignificant positions: , in, For salient position consistency loss, To obtain the expected value for the batch of samples; This is the cosine similarity function.

[0054] As a refinement of the above embodiments, the frequency domain-guided lesion perception branch in step S5 includes a frequency domain encoder, a clinically heuristic dual attention unit, and a dual-path cross-scale-frequency band fusion module; the clinically heuristic dual attention unit includes a lesion perception multi-scale filtering module and a diagnostic context adapter.

[0055] Specifically: (1) The frequency domain encoder is based on two-level Haar discrete wavelet transform, which decomposes the input image into a low-frequency approximate sub-band and two sets of high-frequency sub-bands. The low-frequency component mainly preserves the overall structural information of retinal vessels, while the high-frequency sub-bands emphasize local texture changes and edge features, which are particularly important for the fine representation of lesions. To ensure the consistency of cross-frequency domain feature representation, all sub-band features are projected to a unified embedding space through 1×1 convolution.

[0056] (2) The lesion-sensing multi-scale filtering module, in order to enhance the expressive power of frequency domain features and realize cross-scale semantic modeling, consists of multiple lightweight channel decoupled filtering paths running in parallel in each frequency domain sub-band. Each path uses a different expansion rate, denoted as . This is to construct a hierarchical receptive field. Specifically, when When the value is 1, the filter focuses on local fine-grained features; Paths 2 and 3 progressively expand the receptive field to capture mesoscale to large-scale hemorrhage, exudation, and non-perfusion areas, achieving multi-level lesion feature modeling. Let the input sub-band features be... , No. The output of each branch can be represented as: , The outputs of each branch are concatenated along the channel dimension, and then subjected to fusion convolution and normalization operations to obtain: , , in, For the first Output characteristics of a lightweight channel decoupling filter path For GELU activation function, Indicates BN layer, For pointwise convolution, For depthwise convolution, For the input sub-band features, The lesion sensing multi-scale filtering module outputs features for each sub-band. For 1×1 convolutional fusion weights, The output characteristics of the first lightweight channel decoupling filter path, The output characteristics of the second lightweight channel decoupling filter path, The output characteristics of the third lightweight channel decoupling filter path. For channel-level splicing, This is a second-order wavelet low-frequency approximation subband. It is a secondary high-frequency sub-band. It is a second-level vertical high-frequency sub-band. It is a second-order diagonal high-frequency sub-band. It is a first-level horizontal high-frequency sub-band. It is a first-level vertical high-frequency sub-band. It is a first-level diagonal high-frequency sub-band.

[0057] This design enables the model to simultaneously capture macroscopic structures and microscopic lesion textures in the frequency domain, achieving adaptive perception and enhancement of lesions at different scales, thereby effectively improving the expression sensitivity and structural consistency of lesion regions.

[0058] (3) The diagnostic context adapter, in order to improve the global semantic consistency of the lesion perception features of each sub-band and to simulate the doctor's comprehensive judgment ability on the overall pathological background during the diagnosis process, receives the output features of the lesion perception multi-scale filtering module from each sub-band. Input features of each sub-band First, information compression and saliency extraction are performed through global semantic aggregation.

[0059] Global semantic vector output by global semantic aggregation After channel-sensitive recalibration, an importance weight is generated for each channel, and then the channel weights are... Broadcasting to the spatial dimension, recalibrating the original sub-band features, and finally, the clinically heuristic dual-attention unit follows a clinically heuristic residual fusion strategy. The final output of the sub-band through the clinically heuristic dual attention unit Defined as original subband features and , The outputs are summed (residual backflow): , , , , in, Simulates a doctor's overall perception of lesion distribution when observing the entire sub-band, and extracts global semantic features of each channel through spatial average pooling; , It is a learnable linear transformation; Use the Sigmoid activation function; To compress the number of channels, It is the compression factor; This represents channel-by-channel multiplication. Through this operation, the diagnostic context adapter can achieve global context calibration while preserving the local details of each subband, thereby enhancing the semantic recognizability of key lesions.

[0060] (4) The dual-path cross-scale-frequency band fusion module generates a unified lesion-sensing frequency embedding by explicitly integrating bidirectional cross-scale interaction and lesion-sensitive channel recalibration. First, a coarse-scale high-frequency sub-band is aggregated and adaptively reweighted using a clinically heuristic gating mechanism to enhance the lesion-related frequency domain response. Subsequently, the dual-path gating fusion strategy supports bidirectional information interaction between fine-scale and coarse-scale frequency domain features, while incorporating low-frequency structural context to alleviate cross-scale semantic inconsistency. Finally, the fusion representation is optimized by lesion-sensitive channel recalibration, and then a compact frequency domain enhanced feature map is generated through convolutional fusion. Specifically, it includes: set up This indicates the second-level low-frequency subband after clinically heuristic dual-attention unit processing. This represents the set of first-level high-frequency subbands after clinically heuristic dual-attention unit processing. This represents the set of high-frequency subbands at the second level after clinically heuristic dual-attention unit processing. Weights of lesion regions are calculated through global pooling and convolution operations, injecting clinical semantic priors. , in, This represents the Sigmoid activation function. yes Channel-level splicing. This is an adaptive global average pooling method used to reduce the spatial dimension of the feature map. The prior weight represents the importance of the lesion area.

[0061] The calculated weights are used to perform a weighted summation on the second-level high-frequency subbands to obtain the weighted subband features. : , in, This represents element-wise multiplication. For weighted subband features, The first [item] after processing by the LMF lesion sensing multi-scale filtering module Features of a secondary high-frequency sub-band Used to measure the importance of lesion information within the corresponding high-frequency subband, the first The prior weight matrix of the lesion corresponding to each sub-band is the weight coefficient that the network can learn.

[0062] (3) Cross-scale bidirectional gating fusion: The top-down path is as follows: , , , in, This indicates a downsampling operation, used to reduce the spatial dimensionality of the feature map. It is the first-level high-frequency subband set Channel-level splicing, This represents the first-level gating weights, used to control the information flow between feature maps of different scales. This represents the feature map after the first level of fusion, and this represents the result of fusing feature maps at different scales. This represents a learnable 1×1 convolution weight matrix. express The low-resolution features obtained after downsampling.

[0063] Bottom-up path: , , in, This represents the second-level gating weights, used to control the flow of information between feature maps of different scales. This represents the feature map after the second-level fusion, and the result of fusing feature maps at different scales. This represents a learnable 1×1 convolution weight matrix.

[0064] Recalibrate the lesion-sensitive channels in the fused feature map: , , , in, This indicates splicing along the channel dimension. express and Feature map after channel-level stitching Indicates channel weight, This represents element-wise multiplication. This represents the Sigmoid activation function. Represents convolution. This indicates adaptive global average pooling.

[0065] The lesion-sensing frequency embedding is generated through convolution operations and used as a frequency domain enhancement feature: , in, This represents the final fusion sub-band output, i.e., the lesion-sensing frequency embedding. This indicates the final fusion convolution (1×1) operation, used to generate the final fusion subband.

[0066] As a refinement of the above embodiment, the input to the hierarchical frequency-guided modulation module in step S6 consists of three parts: the cross-modal enhancement representation processed by the sparse-aligned cross-modal attention module and... The design of the layered frequency-guided modulation module references the "three-stage" observation mechanism used by doctors in the clinical diagnostic process: "semantic focus—detailed analysis—information integration." This module, guided by both semantic focus and dynamic frequency filtering, progressively completes adaptive modulation and consistency fusion of cross-modal features layer by layer.

[0067] (1) Cross-modal enhanced representation of text features in the semantic focusing and style modulation stage It is mapped to two guiding signals: one is a semantic focusing map. Cross-modal augmented representations of visual features in spatial dimensions It produces a higher response to semantically relevant regions; the second is the style modulation map. This approach enables fine-grained style adjustment of visual features along the channel dimension, thereby strengthening the semantic consistency between textual semantics and visual representation. Together, they achieve semantically guided visual feature recalibration.

[0068] The specific methods for generating semantic style modulation maps and semantic focus maps are as follows: The cross-modal enhanced representation of visual features is sequentially processed by 3×3 convolution, GELU activation function, and 1×1 convolution, and then Tanh activation is used to generate a semantic style modulation map; The cross-modal enhancement representation of visual features is sequentially processed by 3×3 convolution, GELU activation function, and 1×1 convolution, and a semantic focusing map is generated by Sigmoid activation.

[0069] (2) Subsequently, in the frequency-guided dynamic filtering stage, a unified high-dimensional frequency domain embedding is performed. Mapped to a dynamic filter kernel consistent with the visual feature scale This is used to perform frequency-aware adaptive modulation. This operation can simulate a physician's structural sensitivity at different spatial frequencies, thereby achieving hierarchical enhancement from low-frequency structure to high-frequency details.

[0070] The lesion sensing frequency is embedded and mapped sequentially through 3×3 convolution, GELU activation function, and 1×1 convolution to obtain a dynamic filtering kernel.

[0071] (3) Finally, in the adaptive fusion and residual enhancement stage, the hierarchical frequency-guided modulation module modulates the visual features that have undergone semantic and frequency dual modulation. With unified high-dimensional frequency domain embedding The features are concatenated along the channel dimension and integrated across domains through fusion convolution, while the stability of the original visual semantic structure and gradient flow is preserved through residual paths. The final output feature is defined as: , , , in, These are semantically enhanced visual features obtained after channel and spatial weighting of the text semantics. This represents a multimodal collaborative feature representation.

[0072] As a refinement of the above embodiments, the specific method of training using a training set is as follows: To comprehensively evaluate the performance of different methods in diabetic retinopathy lesion segmentation, this paper uses six commonly used pixel-level evaluation metrics for quantitative analysis, including Dice coefficient, intersection-over-union ratio, balanced accuracy, geometric mean, overall accuracy, and specificity. These metrics comprehensively measure the segmentation results from multiple perspectives, such as region overlap, robustness to class imbalance, and classification discriminative ability, and their definitions are as follows: , Wherein, TP (True Positives) is the number of correctly predicted lesion pixels, TN (True Negatives) is the number of correctly predicted non-lesion (normal) pixels, FP (False Positives) refers to non-lesion pixels that are incorrectly predicted as lesion pixels, FN (False Negatives) refers to lesion pixels that are incorrectly predicted as non-lesion pixels, DICE is the Dice coefficient, IoU is the crossover ratio, BA is the balanced accuracy, G-Mean is the geometric mean, ACC is the overall accuracy, and SPE is the specificity.

[0073] Table 1-2 summarizes a quantitative comparative analysis of the frequency-domain-semantic alignment-based intelligent image segmentation method with current mainstream convolutional and Transformer-based segmentation methods on the DRAC2022 and WFDR datasets. This evaluation covers a range of complementary metrics, including Dice coefficient, IoU, Balanced Accuracy (BA), Geometric Mean (G-Mean), Overall Accuracy (ACC), and Specificity (SPE), thus providing a comprehensive evaluation perspective for segmentation quality in class-imbalanced OCTA scenarios. In the table, p-value represents the P-value.

[0074] On the DRAC2022 dataset, the image intelligent segmentation method based on frequency domain-semantic alignment performed exceptionally well, achieving a Dice coefficient of 0.7602±0.0198 and an IoU of 0.6257±0.0142, both significantly outperforming other comparative methods. Compared to the best-performing baseline method MsTGANet, this method improved the Dice coefficient by approximately 0.9% and the IoU by approximately 1.5%, indicating more accurate overlap between the model's predictions and the actual lesion regions. Furthermore, this method achieved the best BA (balanced accuracy, 0.8604±0.0157), G-Mean (0.8521±0.0217), ACC (overall accuracy, 0.9969±0.0100), and specific SPE (0.9578±0.0054), reflecting a better balance between lesion sensitivity and background discrimination ability. These results demonstrate that the proposed model not only effectively maintains the integrity of the lesion area but also effectively reduces false positive predictions, which is of great value for clinical screening and auxiliary diagnosis of diabetic retinopathy.

[0075] On the WFDR dataset, the frequency-domain-semantic alignment-based intelligent image segmentation method also demonstrated superior performance, achieving the best Dice coefficient (0.7765±0.0217), IoU (0.6316±0.0231), G-Mean (0.8497±0.0186), and specificity (0.9568±0.0008) again, with all metrics significantly outperforming other comparative methods. Compared to the best-performing baseline method, this method improved the Dice coefficient by approximately 1.0% and the IoU by approximately 1.3%, indicating that the model's predictions more accurately overlap with the actual lesion regions. Furthermore, this invention also achieved the best balanced accuracy (BA) and overall accuracy (ACC), reflecting a better balance between lesion sensitivity and background discrimination ability.

[0076] To further evaluate the detection capability under different decision thresholds, precision-recall curves were plotted on the DRAC2022 and WFDR datasets. Compared with existing methods, the constructed image intelligent segmentation method based on frequency domain-semantic alignment consistently achieves higher precision across a wide range of recall values, indicating that this method possesses superior lesion discrimination capabilities. Specifically, when the recall approaches a high value, the precision of many baseline methods declines rapidly, indicating that they generate more false positive predictions; in stark contrast, this invention maintains a relatively stable precision level, fully demonstrating the model's stronger robustness in distinguishing between lesion regions and surrounding vascular structures.

[0077] Table 1 Comparison results on the DRAC2022 dataset. Table 2 shows the comparison results on the WFDR dataset. To intuitively verify the effectiveness and superiority of the frequency-domain-semantic alignment-based intelligent image segmentation method in the lesion segmentation task of diabetic retinopathy OCTA images, the segmentation results of test samples from the DRAC2022 and WFDR datasets are visualized for comparison. Figures 3-4 As shown, the proposed method is compared with mainstream baseline models such as U-Net, AttentionUNet, OCT2Former, and LGMS in terms of segmentation performance. Furthermore, by combining the original images with manually annotated gold standards, the segmentation accuracy, edge integrity, and background suppression capabilities of each model for lesion regions are analyzed from a visual perspective. The dashed circles in the figure indicate typical segmentation errors, including undersegmentation, oversegmentation, and missegmentation.

[0078] In contrast, the proposed frequency-domain semantic alignment-based intelligent image segmentation method demonstrates segmentation performance that better matches the gold standard of human annotation across all test samples. Its core advantage lies in its reliance on sparse alignment cross-modal attention mechanism and significant position consistency loss. The model effectively integrates clinical text semantic priors, accurately suppresses visually significant but clinically irrelevant background structural interference, significantly reduces false positive segmentation, and achieves clear boundary delineation between lesion areas and normal tissues. The frequency-domain guided lesion perception branch, through two-level HaarDWT frequency division and multi-scale filtering, fully captures the low-frequency overall vascular structure and high-frequency lesion texture / edge features in OCTA images, significantly improving the ability to identify hidden lesions such as small non-perfusion areas and bleeding points, thus solving the missed detection problem of the baseline model. The hierarchical frequency-guided modulation module achieves deep collaboration and alignment of text, visual, and frequency domain features, ensuring that the segmentation results maintain both the spatial integrity of the lesion area and the structural fidelity of fine edges. Even in samples with irregular lesion shapes and complex background textures, it can still accurately reproduce the lesion contours and details of the gold standard.

[0079] To quantitatively verify the effectiveness of each core component of the frequency-domain-semantic alignment-based intelligent image segmentation method and its contribution to segmentation performance, this study conducted systematic ablation experiments on the DRAC2022 and WFDR datasets. Multiple comparative models were constructed by progressively removing the sparse alignment cross-modal attention module, the clinical semantic adaptation module, the dual-path cross-scale-band fusion module, and the hierarchical frequency-guided modulation module. The results are shown in Tables 3 and 4.

[0080] By comprehensively comparing all ablation configurations, the proposed frequency-domain-semantic alignment-based intelligent image segmentation method achieves state-of-the-art performance on both the DRAC2022 and WFDR datasets, with core metrics significantly outperforming all ablation variants. In terms of class imbalance robustness metrics (G-Mean, BA) and specificity (SPE), the complete model also performs exceptionally well: on the DRAC2022 dataset, G-Mean reaches 0.8521±0.0217 and BA is 0.8604±0.0157; on the WFDR dataset, G-Mean is 0.8497±0.0186 and BA is 0.8305±0.0071, both significantly higher than other ablation models. This indicates that the image intelligent segmentation method based on frequency domain-semantic alignment, achieved through semantic-visual alignment via the sparse alignment cross-modal attention module, clinical context injection via the clinical semantic adaptation module, frequency domain multi-scale fusion via the dual-path cross-scale-frequency band fusion module, and multi-modal feature modulation via the hierarchical frequency-guided modulation module, not only enhances the fine-grained segmentation capability of lesions but also more effectively suppresses false positive predictions under class imbalance, thus improving the robustness of the model in clinical applications.

[0081] Table 3 Ablation Experiment 1 Table 4 Ablation Experiment 2 Example 2 This disclosure also provides a diabetic retinopathy lesion segmentation device based on frequency domain semantic fusion, including a processor and a memory. Optionally, the device may further include a communication interface and a bus. The processor, communication interface, and memory can communicate with each other via the bus. The communication interface can be used for information transmission. The processor can call logical instructions in the memory to execute the image intelligent segmentation method based on frequency domain-semantic alignment described in the above embodiments.

[0082] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0083] Memory, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor executes the program instructions / modules stored in the memory to perform functional applications and data processing, thereby realizing the image intelligent segmentation method based on frequency domain-semantic alignment in the above embodiments.

[0084] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory may include high-speed random access memory and may also include non-volatile memory.

[0085] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An intelligent image segmentation method based on frequency domain-semantic alignment, characterized in that, Includes the following steps: Acquire OCTA image data of diabetic retinopathy and preprocess the raw images; A multi-scale lesion perception visual coding network is constructed to extract features from the input OCTA image step by step, thereby obtaining multi-scale visual feature representations. Acquire clinical description text related to lesions, and use a pre-trained CLIP text encoder to perform semantic embedding encoding on the clinical description text to obtain semantic feature representation of lesions; A sparsely aligned cross-modal attention module is constructed to perform cross-modal association and semantic enhancement on the multi-scale visual feature representation and the lesion semantic feature representation; and a consistency constraint is applied to the cross-modal attention space response through salient position consistency loss to obtain the cross-modal enhanced representation. A frequency-domain guided lesion perception branch was constructed. Discrete wavelet transform was used to decompose OCTA image data in the frequency domain, and lesion regions in different frequency bands were enhanced to obtain frequency domain enhancement features. A hierarchical frequency-guided modulation module is constructed to jointly modulate and fuse cross-modal enhancement representations and frequency domain enhancement features to obtain multimodal collaborative feature representations; The multimodal collaborative feature representation is input into the lesion segmentation decoder; the lesion segmentation decoder is built on the U-Net decoder architecture, performs upsampling operation layer by layer, and introduces a spatial attention module to enhance the features after each layer completes upsampling, restores the image resolution layer by layer, and outputs the lesion segmentation result.

2. The image intelligent segmentation method based on frequency domain-semantic alignment according to claim 1, characterized in that, The multi-scale lesion perception visual coding network includes a multi-scale lesion perception residual encoder and a lightweight spatial attention mechanism; the multi-scale lesion perception residual encoder adopts a four-stage progressive downsampling structure, and a lightweight spatial attention mechanism is introduced in each stage of downsampling. The downsampling structure of the multi-scale lesion perception residual encoder at each stage consists of a residual convolution module and a pooling operation. A three-branch dilated convolution structure is introduced into the residual convolution module to extract multi-scale context information in parallel. The pooling operation includes average pooling and global pooling. The input feature map is processed by multi-branch dilated convolution to extract multi-scale contextual information. The sum of the multi-scale contextual information is then added to the elements of the input feature map to obtain the final feature map. Feature map Spatial attention is obtained by performing average pooling and global pooling operations followed by convolution. This spatial attention is then used to apply spatial attention to the feature map. The downsampled feature map is obtained through enhancement; the downsampled feature map is then input into the next stage downsampled structure, and a multi-scale visual feature representation is obtained through a four-stage progressive downsampled structure.

3. The image intelligent segmentation method based on frequency domain-semantic alignment according to claim 1, characterized in that, The sparse-aligned cross-modal attention module is as follows: The multi-scale visual feature representation and the semantic feature representation of the lesion are projected onto a shared space to obtain a text token and a visual token. Construct a bidirectional similarity matrix between text tokens and visual tokens. Based on the bidirectional similarity matrix, introduce structured sparse constraints and retain only the top-k associations with the highest similarity for each token to obtain a bidirectional sparse mask. Bidirectional sparse attention weights are calculated using the softmax function based on bidirectional sparse masks. The multi-scale visual feature representation and the lesion semantic feature representation are linearly transformed respectively, and then enhanced by the corresponding bidirectional sparse attention weights across modal features to obtain cross-modal enhanced representations, including cross-modal enhanced representations of text features and cross-modal enhanced representations of visual features.

4. The image intelligent segmentation method based on frequency domain-semantic alignment according to claim 3, characterized in that, The salient location consistency loss defines text location saliency by calculating the variance of text-to-visual attention in the visual dimension, and constructs a sparse salient location indicator based on this. The consistency of cross-modal attention in salient spatial locations is constrained by maximizing the cosine similarity between the salient location indicator and the bidirectional attention statistic; specifically as follows: ， ， ， ， ， in, For salient position consistency loss, To obtain the expected value for the batch sample, It is a sparse position indicator vector. For L2 normalization, For indicator functions, For text position salience, To select The largest value One element, For along the visual dimension Calculate the mean. For text-to-visual attention matrix, For the real number field, This is a statistical representation of visual-text attention mapped to the text semantic space. For learnable projection matrices, For visual-to-text attention matrix, This represents the average response strength of the text token in the visual space. For along the visual dimension Calculate the variance. The average response strength of the visual token to the text space. The batch size of the input samples for a single training session. The total number of visual tokens output after encoding the OCTA image.

5. The image intelligent segmentation method based on frequency domain-semantic alignment according to claim 1, characterized in that, The frequency domain-guided lesion perception branch includes a frequency domain encoder, a clinically heuristic dual attention unit, and a dual-path cross-scale-frequency band fusion module; the clinically heuristic dual attention unit includes a lesion perception multi-scale filtering module and a diagnostic context adapter. The frequency domain encoder is based on two-level Haar discrete wavelet transform, which decomposes the input image into original sub-band features, including a low-frequency approximate sub-band feature and two sets of high-frequency sub-band features, and projects all sub-band features into a unified embedding space through convolution. The lesion sensing multi-scale filtering module consists of three parallel lightweight channel decoupling filtering paths with different dilation rates. Each lightweight channel decoupling filtering path includes depthwise convolution and pointwise convolution in sequence; specifically represented as follows: ， ， ， in, For the first Output characteristics of a lightweight channel decoupling filter path For GELU activation function, Indicates BN layer, For pointwise convolution, For depthwise convolution, For the input sub-band features, The lesion sensing multi-scale filtering module outputs features for each sub-band. For the sub-band set, For 1×1 convolutional fusion weights, The output characteristics of the first lightweight channel decoupling filter path, The output characteristics of the second lightweight channel decoupling filter path, The output characteristics of the third lightweight channel decoupling filter path. For channel-level splicing, This is a second-order wavelet low-frequency approximation subband. It is a secondary high-frequency sub-band. It is a second-level vertical high-frequency sub-band. It is a second-order diagonal high-frequency sub-band. It is a first-level horizontal high-frequency sub-band. It is a first-level vertical high-frequency sub-band. It is a first-level diagonal high-frequency sub-band; The diagnostic context adapter performs global context calibration on the features of each sub-band through global semantic aggregation and channel-sensitive recalibration; specifically, it includes: Will A global semantic vector is obtained through global semantic aggregation. Channel-sensitive recalibration is then performed on this global semantic vector to generate importance weights for each channel. These importance weights are then compared with... Multiply by each channel to obtain the features Combine the original subband features with , The sum is used as a feature output by the diagnostic context adapter.

6. The image intelligent segmentation method based on frequency domain-semantic alignment according to claim 5, characterized in that, The dual-path cross-scale-frequency band fusion module performs bidirectional gated fusion of high-frequency and low-frequency subbands at all levels in the output features of the diagnostic context adapter, and outputs lesion-sensing frequency embedding after channel recalibration as a frequency domain enhancement feature. By explicitly integrating bidirectional cross-scale interaction and lesion-sensitive channel recalibration, a unified lesion-sensing frequency embedding is generated; specifically including: The second-level high-frequency subband set from the diagnostic context adapter output features is concatenated and then subjected to global pooling and convolution operations. The weights of the lesion regions are calculated using the Sigmoid activation function to obtain the clinical semantic prior weights. These clinical semantic prior weights are then used to perform a weighted summation of the second-level high-frequency subbands to obtain the weighted subband features. ; The feature is obtained by concatenating the first-level high-frequency subband set from the diagnostic context adapter output features and then downsampling it. ,feature The first-level gating weights are obtained by concatenating the weighted subband features with the sigmoid activation function. Based on the first-level gating weights for features and weighted subband features Weighted fusion yields the feature map after the first level of fusion. : ， The feature map after the first level of fusion Second-level low-frequency subband in the diagnostic context adapter output characteristics The concatenated weights are then processed using the Sigmoid activation function to obtain the second-level gating weights. The feature map after first-level fusion is based on the second-level gating weights. The second-level fused feature map is obtained by weighted fusion of the second-level low-frequency subband. : ； For the fused feature map and Perform lesion-sensitive channel recalibration: ， ， ， in, This indicates splicing along the channel dimension. express and Feature map after channel-level stitching Indicates channel weight, This represents element-wise multiplication. This represents the Sigmoid activation function. Represents convolution. This indicates adaptive global average pooling; The calibrated features are used to generate lesion-sensing frequency embeddings through convolution operations, which serve as frequency domain enhancement features.

7. The image intelligent segmentation method based on frequency domain-semantic alignment according to claim 6, characterized in that, The hierarchical frequency-guided modulation module adaptively modulates and achieves consistency fusion of cross-modal features through dual guidance of semantic focusing and dynamic frequency filtering; specifically, it includes: Cross-modal augmentation representation of text features Mapped to a semantic focus graph and style modulation diagram The lesion sensing frequency is embedded and mapped as a dynamic filtering kernel. ; Visual features are obtained by performing semantic and frequency dual modulation on the cross-modal enhanced representation of visual features. The details are as follows: ， ， in, These are semantically enhanced visual features obtained after channel and spatial weighting of the text semantics; visual features The features are concatenated with the lesion perception frequency embedding channel dimension and cross-domain feature integration is achieved through fusion convolution. The original visual semantic structure and gradient flow stability are preserved through residual path to obtain multimodal collaborative feature representation.

8. The image intelligent segmentation method based on frequency domain-semantic alignment according to claim 7, characterized in that, The cross-modal enhanced representation of visual features is sequentially processed by 3×3 convolution, GELU activation function, and 1×1 convolution, and then Tanh activation is used to generate a semantic style modulation map; The cross-modal enhanced representation of visual features is sequentially processed by 3×3 convolution, GELU activation function, and 1×1 convolution, and then a semantic focusing map is generated using Sigmoid activation. The lesion sensing frequency is embedded and mapped sequentially through 3×3 convolution, GELU activation function, and 1×1 convolution to obtain a dynamic filtering kernel.

9. The image intelligent segmentation method based on frequency domain-semantic alignment according to claim 1, characterized in that, The OCTA image data for diabetic retinopathy were constructed and trained using the DRAC2022 challenge dataset and the WFDR wide field-of-view OCTA dataset, and were based on the PyTorch deep learning framework.

10. An image intelligent segmentation device based on frequency domain-semantic alignment, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to execute the image intelligent segmentation method based on frequency domain-semantic alignment as described in any one of claims 1-8 when running the program instructions.

Images