A gradient-guided image segmentation method and related device

Abstract

The application discloses a gradient-guided image segmentation method and related equipment, the method comprising: preprocessing a retinal image set, and extracting information from a training image of the preprocessed retinal image set to obtain training gradient prior information; based on the preprocessed retinal image set and a corresponding blood vessel segmentation gold standard image set, constructing a training set, a validation set and a test set, and augmenting the training set and the validation set; inputting the augmented training set and the training gradient prior information into a multi-scale Mamba network for training, and based on the augmented validation set and the test set, verifying and evaluating the model of the trained multi-scale Mamba network; inputting a to-be-segmented retinal image and corresponding gradient prior information into a target multi-scale Mamba network, and outputting a retinal image segmentation result. The application can improve the blood vessel structure recognition capability and segmentation precision, and can be widely applied to the technical field of image processing.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a gradient-guided image segmentation method and related equipment. Background Technology

[0002] Currently, retinal vessels, as the only microcirculatory system in the human body that can be directly and non-invasively observed, are widely regarded as early biomarkers for various systemic diseases, such as diabetic retinopathy, hypertension, and arteriosclerosis, due to changes in their morphology and structure. Therefore, accurate and automated retinal vessel segmentation is of great value for clinical auxiliary diagnosis, disease screening, and quantitative analysis. However, due to the inherent complexity of retinal images, achieving precise segmentation still faces many technical challenges.

[0003] In related technologies, traditional image processing methods rely excessively on manually designed features, resulting in weak generalization ability and difficulty in handling low-contrast or structurally complex vascular regions. While supervised learning methods based on deep learning can improve segmentation performance, pure convolutional networks are limited by their receptive field, making it difficult to model long-distance vascular dependencies. Transformer-based methods can capture global context, but the secondary complexity of the self-attention mechanism leads to high computational costs, and they are prone to overfitting on small-scale medical image datasets, causing blurred vascular boundaries and loss of details. Hybrid models that have emerged in recent years still have shortcomings in feature fusion and noise suppression. The current core problem remains to achieve accurate modeling of vascular structures, effective capture of global dependencies, and fine reconstruction of boundary details while ensuring computational efficiency. However, related technologies generally struggle to balance global modeling and detail preservation, and their performance in complex vascular scenes remains unsatisfactory.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0005] The embodiments of this application aim to at least partially address one of the technical problems in related technologies. Therefore, the main objective of the embodiments of this application is to propose a gradient-guided image segmentation method and related equipment, which can significantly improve the ability to identify and segment vascular structures.

[0006] To achieve the above objectives, one aspect of this application proposes a gradient-guided image segmentation method, which includes the following steps: Obtain the original retinal vessel segmentation dataset; wherein, the original retinal vessel segmentation dataset includes the original retinal image set and the corresponding gold standard image set for vessel segmentation; The original retinal image set is preprocessed to obtain the target retinal image set, and the training images in the target retinal image set are subjected to gradient information extraction processing to obtain training gradient prior information. Based on the target retinal image set and the corresponding gold standard image set for vessel segmentation, a target retinal vessel segmentation dataset is constructed, and the training and validation sets in the target retinal vessel segmentation dataset are subjected to data augmentation processing. The amplified training set and the prior information of the training gradient are input into the pre-constructed original multi-scale Mamba network for end-to-end training to obtain the candidate multi-scale Mamba network. The candidate multi-scale Mamba network is validated based on the expanded validation set, and the performance of the validated candidate multi-scale Mamba network is evaluated based on the test set in the target retinal vessel segmentation dataset to obtain the target multi-scale Mamba network. The preprocessed retinal image to be segmented is obtained, and the retinal image to be segmented and the gradient prior information corresponding to the retinal image to be segmented are input into the target multi-scale Mamba network for inference, and the retinal image segmentation result is output.

[0007] In some embodiments, the image preprocessing of the original retinal image set to obtain the target retinal image set includes: The original retinal image set is converted to grayscale to obtain a grayscale image set; The grayscale image set is standardized to obtain a standardized image set; The standardized image set is subjected to adaptive histogram equalization to obtain an adaptively enhanced image set; The adaptively enhanced image set is subjected to gamma correction processing to obtain a gamma-corrected image set; The gamma-corrected image set is normalized to obtain the target retinal image set.

[0008] In some embodiments, the step of extracting gradient information from the training images in the target retinal image set to obtain prior training gradient information includes: The gradients of the training images in the target retinal image set in the horizontal and vertical directions are calculated respectively to obtain the horizontal gradient and the vertical gradient. Calculate the gradient magnitude map based on the horizontal and vertical gradients. The gradient magnitude map is linearly normalized to obtain the prior information of the training gradient.

[0009] In some embodiments, the target multi-scale Mamba network consists of an encoder, a bottleneck layer, and a decoder. The encoder includes four serial multi-scale state modules, the bottleneck layer includes a gradient-guided lightweight Transformer module, and the decoder includes an upsampling module, a gated skip fusion module, and a multi-scale feature bridge. The process of inputting the retinal image to be segmented and the corresponding gradient prior information into the target multi-scale Mamba network for inference, and outputting the retinal image segmentation result, includes: By using four serial multi-scale state modules and combining the gradient prior information corresponding to the retinal image to be segmented, the local texture details and global topological structure of the retinal vessels in the retinal image to be segmented are modeled, generating four different levels of multi-scale feature representations. The gradient-guided lightweight Transformer module, combined with the gradient prior information corresponding to the retinal image to be segmented, performs global guidance and blood vessel boundary enhancement on the fourth layer multi-scale feature representation to obtain the gradient-guided feature representation. The upsampling module performs an upsampling operation on the gradient-guided feature representation to obtain four different levels of upsampled output features. The gated skip fusion module performs gated cross-layer fusion processing on the four different levels of multi-scale feature representations and the four different levels of upsampled output features to obtain four different levels of cross-layer fused features. By using the multi-scale feature bridge and combining the gradient prior information corresponding to the retinal image to be segmented, adaptive multi-scale feature fusion processing is performed on the cross-layer fusion features at four different levels to output the retinal image segmentation result.

[0010] In some embodiments, the process involves using four serial multi-scale state modules, combined with the gradient prior information corresponding to the retinal image to be segmented, to model the local texture details and global topological structure of retinal vessels in the retinal image to be segmented, generating four different levels of multi-scale feature representations, including: The first multi-scale state module in the encoder performs a gated transformation on the gradient prior information corresponding to the retinal image to be segmented, generating a first local spatial modulation weight and a first global spatial modulation weight. The first-layer multi-scale state module combines the first local spatial modulation weight and the first global spatial modulation weight to perform weighted processing on the retinal image to be segmented, thereby obtaining the first local feature representation and the first global feature representation. The first local feature representation and the first global feature representation are then fused by adaptive weights to obtain the first-layer multi-scale feature representation. The second-layer multi-scale state module in the encoder performs a gated transformation on the gradient prior information corresponding to the retinal image to be segmented, generating a second local spatial modulation weight and a second global spatial modulation weight. By using the second-layer multi-scale state module, the first-layer multi-scale feature representation is weighted by combining the second local spatial modulation weight and the second global spatial modulation weight to obtain the second local feature representation and the second global feature representation. The second local feature representation and the second global feature representation are then fused by adaptive weights to obtain the second-layer multi-scale feature representation. The third-layer multi-scale state module in the encoder performs a gated transformation on the gradient prior information corresponding to the retinal image to be segmented, generating a third local spatial modulation weight and a third global spatial modulation weight. The second-layer multi-scale feature representation is weighted by the third-layer multi-scale state module, combined with the third local spatial modulation weight and the third global spatial modulation weight, to obtain the third local feature representation and the third global feature representation. The third local feature representation and the third global feature representation are then fused by adaptive weights to obtain the third-layer multi-scale feature representation. The fourth multi-scale state module in the encoder performs a gated transformation on the gradient prior information corresponding to the retinal image to be segmented, generating a fourth local spatial modulation weight and a fourth global spatial modulation weight. The third-layer multi-scale feature representation is weighted by the fourth-layer multi-scale state module, which combines the fourth local spatial modulation weight and the fourth global spatial modulation weight to obtain the fourth local feature representation and the fourth global feature representation. The fourth local feature representation and the fourth global feature representation are then fused by adaptive weights to obtain the fourth-layer multi-scale feature representation.

[0011] In some embodiments, the gradient-guided lightweight Transformer module, combined with the gradient prior information corresponding to the retinal image to be segmented, performs global guidance and vessel boundary enhancement processing on the fourth-layer multi-scale feature representation to obtain the gradient-guided feature representation, including: The convolution module in the gradient-guided lightweight Transformer module performs a convolution operation on the fourth layer multi-scale feature representation to obtain the convolutional transformation feature. The structure-aware attention unit in the gradient-guided lightweight Transformer module performs linear projection on the convolutional transformation features to obtain linear projection features, and then maps the linear projection features using a low-rank projection matrix to obtain low-rank mapped features. The structure-aware attention unit calculates low-rank multi-head self-attention based on the low-rank mapping features, and modulates the low-rank multi-head self-attention element-wise with the gradient prior information corresponding to the retinal image to be segmented after activation by the Sigmoid function to obtain gradient modulation features. The gradient-guided feature representation is obtained by performing feedforward mapping on the gradient modulation features through the feedforward network in the gradient-guided lightweight Transformer module, and then adding the feedforward-mapped features to the convolutional transformation features through residual connections.

[0012] In some embodiments, the gated skip fusion module performs gated cross-layer fusion processing on the four different levels of multi-scale feature representations and the four different levels of upsampled output features to obtain four different levels of cross-layer fused features, including: Convolution operations are performed on the four different levels of the multi-scale feature representation to obtain four encoded transformation features; Convolution operations are performed on the upsampled output features at four different levels to obtain four decoded transform features; The four decoding transformation features are activated and mapped using the Sigmoid activation function to generate four spatial gating maps. For each of the spatial gating maps, the corresponding encoded transformation features are modulated pixel by pixel to obtain four modulation features; Each of the modulation features is concatenated with the corresponding upsampled output feature to obtain four feature concatenation results; Convolution operations are performed on the four concatenated feature results to obtain four different levels of cross-layer fusion features.

[0013] In some embodiments, the adaptive multi-scale feature fusion processing of the cross-layer fusion features at four different levels, through the multi-scale feature bridge and in conjunction with the gradient prior information corresponding to the retinal image to be segmented, to output the retinal image segmentation result includes: Through the multi-scale feature bridge, upsampling and convolution operations are performed on the cross-layer fusion features at four different levels to obtain a fusion feature representation with four unified dimensions. Through the weight learning unit in the multi-scale feature bridge, global average pooling is performed on the fused feature representations of the four unified dimensions to obtain four global average pooled features. The four global average pooling features are concatenated to obtain global concatenated features, and the global concatenated features are input into the lightweight multilayer perceptron in the multi-scale feature bridge. The lightweight multilayer perceptron normalizes the global concatenated features to obtain four adaptive fusion weights. Based on the four adaptive fusion weights, the fusion feature representations of the four unified dimensions are weighted and fused, and the weighted fused features are convolved to generate the retinal image segmentation result.

[0014] To achieve the above objectives, another aspect of this application proposes a gradient-guided image segmentation apparatus, which includes the following modules: The original dataset acquisition module is used to acquire the original retinal vessel segmentation dataset; wherein, the original retinal vessel segmentation dataset includes the original retinal image set and the corresponding gold standard image set for vessel segmentation; The image preprocessing module is used to preprocess the original retinal image set to obtain the target retinal image set, and to extract gradient information from the training images in the target retinal image set to obtain training gradient prior information. The data augmentation processing module is used to construct a target retinal vessel segmentation dataset based on the target retinal image set and the corresponding gold standard image set for vessel segmentation, and to perform data augmentation processing on the training set and validation set in the target retinal vessel segmentation dataset; The network training module is used to input the amplified training set and the prior information of the training gradient into the pre-constructed original multi-scale Mamba network for end-to-end training to obtain a candidate multi-scale Mamba network. The network validation and testing module is used to perform model validation on the candidate multi-scale Mamba network based on the amplified validation set, and to evaluate the performance of the model-validated candidate multi-scale Mamba network based on the test set in the target retinal vessel segmentation dataset, so as to obtain the target multi-scale Mamba network. The network inference module is used to acquire the retinal image to be segmented after image preprocessing, and input the retinal image to be segmented and the gradient prior information corresponding to the retinal image to be segmented into the target multi-scale Mamba network for inference, and output the retinal image segmentation result.

[0015] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0016] The embodiments of this application include at least the following beneficial effects: This application provides a gradient-guided image segmentation method and related equipment. This method obtains a raw retinal vessel segmentation dataset; wherein, the raw retinal vessel segmentation dataset includes a raw retinal image set and a gold standard image set for vessel segmentation corresponding to the raw retinal image set; the raw retinal image set is preprocessed to obtain a target retinal image set, and gradient information extraction processing is performed on the training images in the target retinal image set to obtain training gradient prior information; based on the target retinal image set and the gold standard image set for vessel segmentation corresponding to the target retinal image set, a target retinal vessel segmentation dataset is constructed, and the training images in the target retinal vessel segmentation dataset are processed... The training and validation sets are augmented; the augmented training set and training gradient prior information are input into the pre-constructed original multi-scale Mamba network for end-to-end training to obtain candidate multi-scale Mamba networks; the candidate multi-scale Mamba networks are validated based on the augmented validation set, and the performance of the validated candidate multi-scale Mamba networks is evaluated based on the test set in the target retinal vessel segmentation dataset to obtain the target multi-scale Mamba network; the pre-processed retinal image to be segmented is acquired, and the retinal image to be segmented and the corresponding gradient prior information are input into the target multi-scale Mamba network for inference, outputting the retinal image segmentation result. This application's embodiments, by extracting and introducing training gradient prior information, can enhance the edge and fine branch features of retinal vessels, suppress background interference, and improve the integrity and accuracy of small vessel segmentation. Furthermore, by synchronously inputting gradient prior information and preprocessed images into a multi-scale Mamba network for end-to-end training, deep fusion of structural priors and multi-scale features can be achieved, taking into account both the global topological structure and local texture details of blood vessels, further improving image segmentation accuracy. Moreover, by augmenting the training and validation sets, sample diversity can be expanded without increasing acquisition costs, effectively improving the model's generalization ability and robustness. In short, the gradient-guided multi-module collaborative segmentation framework constructed in this application can significantly improve the ability to recognize and segment blood vessel structures. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the steps of a gradient-guided image segmentation method provided in an embodiment of this application. Figure 2 This is a schematic diagram of a gradient-guided multi-scale Mamba network provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a multi-scale state module provided in an embodiment of this application; Figure 4This is a schematic diagram of the structure of a gradient-guided lightweight Transformer module provided in an embodiment of this application; Figure 5 This is a schematic diagram of a multi-scale feature bridge provided in an embodiment of this application; Figure 6 This is a schematic diagram of the overall process of a gradient-guided image segmentation method provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a gradient-guided image segmentation device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0019] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0020] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0022] Currently, retinal vessels, as the only microcirculatory system in the human body that can be directly and non-invasively observed, are widely regarded as early biomarkers for various systemic diseases, such as diabetic retinopathy, hypertension, and arteriosclerosis, due to changes in their morphology and structure. Therefore, accurate and automated retinal vessel segmentation is of great value for clinical auxiliary diagnosis, disease screening, and quantitative analysis. However, due to the inherent complexity of retinal images, achieving accurate segmentation still faces many technical challenges. First, the complex structure of blood vessels, characterized by their elongated shape, numerous tiny branches, and low-contrast regions, makes them highly susceptible to background noise. Second, the scale of blood vessels varies greatly, from large trunks to tiny capillaries, requiring algorithms with strong multi-scale modeling capabilities. Furthermore, factors such as lighting conditions, imaging equipment, and patient cooperation often lead to blurring, reflections, and noise in retinal images, further exacerbating the segmentation difficulty. Among related technologies, traditional image processing methods, such as matched filtering, morphological manipulation, and model tracking, while capable of preliminary vessel extraction, rely excessively on manually designed features, have weak generalization ability, and struggle to handle low-contrast or structurally complex vascular regions. With the development of deep learning, supervised learning methods, represented by fully convolutional networks and U-Net, have significantly improved segmentation performance. However, pure convolutional networks are limited by their receptive field, making it difficult to model long-distance vascular dependencies. While Transformer-based methods can capture global context, the secondary complexity of the self-attention mechanism leads to high computational costs, and they are prone to overfitting on small-scale medical image datasets, resulting in blurred vascular boundaries and loss of details. Although hybrid models that have emerged in recent years attempt to combine the advantages of convolution and Transformers, they still lack systematic design in terms of feature fusion efficiency, noise suppression, and boundary enhancement. Therefore, how to achieve accurate modeling of vascular structures, effective capture of global dependencies, and fine reconstruction of boundary details while ensuring computational efficiency remains a key problem that urgently needs to be solved.

[0023] For example, the relevant technical solutions mainly cover methods based on convolutional neural networks, Transformers, hybrid architectures, and state-of-the-art state-space models. Convolutional network methods, represented by U-Net and its variants, such as U-Net++, effectively fuse shallow spatial information with deep semantic information through a symmetric encoder-decoder structure and skip connections. They have advantages such as simple structure, easy training, and good preservation of local details. However, limited by the receptive field, they struggle to model the global vascular topology, and skip connections may introduce noise, affecting segmentation quality. Transformer-based segmentation methods, such as TransUNet and Swin-UNet, borrow self-attention mechanisms to model long-distance dependencies and capture the global vascular topology. However, the quadratic complexity of self-attention makes high-resolution image processing difficult, and the lack of local inductive bias makes them prone to overfitting on small-scale medical datasets, resulting in blurred boundaries and loss of details. To balance local and global modeling, various hybrid CNN-Transformer architectures have emerged in recent years. For example, UCTransNet introduces Transformer into skip connections to enhance cross-layer feature fusion, GVT-RSNet combines graph convolution and multi-scale Transformer to improve vascular structure modeling, and TA-Mamba is the first to introduce the Mamba state-space model to replace Transformer. These methods, to some extent, integrate local details and global semantics, but still lack structural prior guidance as provided in this application, have limited noise suppression capabilities, and redundant information in skip connections still affects the decoding process. Furthermore, a systematic structural guidance mechanism as provided in this application has not yet been formed. Mamba-based methods, such as VMamba and TA-Mamba, achieve linear computational complexity using selective state-space mechanisms and show superior modeling potential compared to Transformer. However, current schemes often lack integration with structural priors, and a mature multi-scale, multi-modal framework has not yet been formed in medical image segmentation. There is still room for improvement in modeling vascular boundaries and fine structures. In summary, related technologies generally struggle to achieve a balance between global modeling and detail preservation, and lack the effective integration of explicit structural priors as provided in this application, resulting in unsatisfactory performance in complex vascular structures, low-contrast regions, and boundary fineness.

[0024] In view of this, this application provides a gradient-guided image segmentation method and related device. This method, by extracting and introducing training gradient prior information, can enhance the edge and fine branch features of retinal vessels, suppress background interference, and improve the integrity and accuracy of small vessel segmentation. Furthermore, by synchronously inputting the gradient prior information and the preprocessed image into a multi-scale Mamba network for end-to-end training, it can achieve deep fusion of structural priors and multi-scale features, taking into account both the global topological structure and local texture details of blood vessels, further improving image segmentation accuracy. Moreover, by augmenting the training and validation sets, it can expand the sample diversity without increasing the acquisition cost, effectively improving the model's generalization ability and robustness. In short, the gradient-guided multi-module collaborative segmentation framework constructed in this application can significantly improve the ability to recognize blood vessel structures and the accuracy of segmentation.

[0025] This application provides a gradient-guided image segmentation method, relating to the field of image processing technology. This gradient-guided image segmentation method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing a gradient-guided image segmentation method, but is not limited to the above forms.

[0026] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0027] Please see Figure 1 , Figure 1 This is an optional flowchart of a gradient-guided image segmentation method provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S101 to S106.

[0028] Step S101: Obtain the original retinal vessel segmentation dataset; wherein, the original retinal vessel segmentation dataset includes the original retinal image set and the gold standard image set for vessel segmentation corresponding to the original retinal image set; The original retinal vessel segmentation dataset is the complete set of raw data used in the retinal vessel segmentation task, consisting of pairs of raw retinal images and corresponding gold standard images for vessel segmentation. The raw retinal images are directly acquired raw fundus retinal images. The gold standard images for vessel segmentation are labeled images that are matched one-to-one with the raw retinal images, serving as the standard reference for model training and performance evaluation.

[0029] In the specific implementation, firstly, the original retinal vessel segmentation dataset for retinal vessel segmentation is obtained. Each sample in the dataset contains an original retinal image and its corresponding gold standard image for vessel segmentation. Then, the original retinal vessel segmentation dataset is divided into a training set, a validation set, and a test set according to a preset ratio. The division is carried out in a ratio of 70% training set, 15% validation set, and 15% test set, and the image distribution of each subset is kept consistent.

[0030] Step S102: Perform image preprocessing on the original retinal image set to obtain the target retinal image set, and perform gradient information extraction processing on the training images in the target retinal image set to obtain training gradient prior information. In some embodiments, the step of preprocessing the original retinal image set to obtain the target retinal image set may include: converting the original retinal image set to grayscale to obtain a grayscale image set; performing a normalization operation on the grayscale image set to obtain a normalized image set; performing adaptive histogram equalization on the normalized image set to obtain an adaptively enhanced image set; performing gamma correction on the adaptively enhanced image set to obtain a gamma-corrected image set; and performing normalization on the gamma-corrected image set to obtain the target retinal image set.

[0031] Optionally, the target retinal image set is a set of model input images with optimized quality and uniform specifications obtained after the original retinal images have undergone a complete image preprocessing process. The image preprocessing process may include, but is not limited to, grayscale conversion, standardization, adaptive histogram equalization, gamma correction, and normalization.

[0032] In the implementation, the training set, validation set, and test set all use the same image preprocessing method, and the image preprocessing process executed in the subsequent actual inference process also adopts the image preprocessing method exemplified in step S102. By performing image preprocessing on the retinal image, image noise can be reduced, the contrast between blood vessel pixels and background pixels can be improved, thereby enhancing the segmentation accuracy of the deep learning model.

[0033] Specifically, this application employs a grayscale conversion method that extracts the green channel (G channel) from the RGB image, as the green channel provides the highest contrast between the image's blood vessel contours and the background, while also minimizing noise. Standardization involves scaling the overall grayscale values ​​of the grayscaled image to prevent extreme pixel values ​​and alleviate uneven illumination in retinal images. Adaptive histogram equalization (AHEEE) is applied to the standardized image to stretch its grayscale histogram, improving local contrast while limiting extremely dark and bright areas to prevent noise amplification due to over-enhancement. Gamma correction is then applied to the AHEEE-processed image to suppress uneven illumination and reflections along the blood vessel centerline in retinal images. Normalization of the gamma-corrected retinal image data linearly maps the grayscale levels from 0-255 to 0-1, facilitating subsequent calculations and processing by deep learning models.

[0034] In some embodiments, the step of extracting gradient information from training images in the target retinal image set to obtain prior training gradient information may include: calculating the gradients of the training images in the target retinal image set in the horizontal and vertical directions respectively to obtain the horizontal gradient and the vertical gradient; calculating the gradient magnitude map based on the horizontal gradient and the vertical gradient; and performing linear normalization on the gradient magnitude map to obtain prior training gradient information.

[0035] Among them, the horizontal gradient is a feature data that characterizes the degree of horizontal brightness variation of image pixels, reflecting the edge differences in the horizontal direction of the image; the vertical gradient is a feature data that characterizes the degree of vertical brightness variation of image pixels, reflecting the edge differences in the vertical direction of the image.

[0036] The gradient magnitude map is a feature image obtained by combining horizontal and vertical bidirectional gradient calculations, which highlights key structural information such as blood vessel edges and contours.

[0037] The training gradient prior information refers to the gradient prior information corresponding to the training images in the training set after image preprocessing. It is the gradient feature information obtained after linear normalization of the gradient magnitude map corresponding to the training image. It serves as an auxiliary prior constraint for the model to segment retinal vessels and is used to train the model.

[0038] It should be noted that, for ease of description, step S102 primarily uses the training images in the preprocessed training set as an example to illustrate gradient information extraction. However, in practical applications, in addition to the training images in the preprocessed training set requiring gradient information extraction, the validation images in the preprocessed validation set, the test images in the test set, and the preprocessed retinal images to be segmented during actual inference all require gradient information extraction to obtain corresponding gradient prior information for input to the model. It can be understood that the training images in the preprocessed training set, the validation images in the validation set, the test images in the test set, and the retinal images to be segmented during actual inference all require gradient information extraction using the method exemplified in step S102 to obtain corresponding gradient prior information for input to the model.

[0039] Furthermore, during the model training, model validation, and model performance evaluation process, gradient information extraction can be performed on the training images in the preprocessed training set to obtain gradient prior information for model training input. Then, in the model validation and model performance evaluation steps, gradient information extraction can be performed on the validation images in the preprocessed training set and the test images in the test set to obtain the corresponding gradient prior information for input into the model. Alternatively, gradient information extraction can be performed on the retinal images in the preprocessed training set, validation set, and test set to obtain the corresponding gradient prior information for input into the model. In other words, the order of gradient information extraction on the retinal images in the training set, validation set, and test set is not restricted and can be set according to the actual situation. For example, step S102 mainly involves performing gradient information extraction processing on the training images in the training set to obtain gradient prior information for model training. During this process, gradient information extraction processing can also be performed on the validation images in the validation set to obtain gradient prior information for subsequent model validation, and gradient information extraction processing can also be performed on the test images in the test set to obtain gradient prior information for subsequent model testing (i.e., model performance evaluation). Similarly, gradient information extraction processing can also be performed on the retinal images in the validation set and the test set before performing model validation and model testing in other steps such as step S105. That is, the embodiments of this application do not restrict the order of gradient information extraction processing on the retinal images in the training set, validation set, and test set.

[0040] Specifically, after image preprocessing, this application first extracts structural prior information from the preprocessed retinal image to provide explicit vascular boundary guidance for subsequent networks. Specifically, for the preprocessed input image... The gradient response is calculated using the Sobel operator: First, the input images are calculated respectively. The gradients are calculated in the horizontal and vertical directions; then, based on the gradient responses in both directions, the gradient magnitude map is calculated. Finally, the resulting gradient magnitude map Linearly normalized to the [0,1] interval, serving as a fixed, unlearnable structural prior (the normalized structural prior is also used...). (Represented by...). This gradient magnitude map contains significant responses at the vessel boundaries and will be used in multiple modules of the subsequent network for feature modulation, attention guidance, and multi-scale fusion.

[0041] It is understandable that gradient prior information extraction should be performed on all preprocessed input images, including the training set, validation set, test set, and input images used in the actual inference stage. Gradient prior information is used in model training during the training phase, and corresponding gradient prior information is used for model validation and performance evaluation during the validation and testing phases. Similarly, gradient prior information needs to be extracted from the input images and used as model input during the actual inference stage. That is, gradient prior information, as part of the model input, is used not only in the training phase but also in the validation, testing, and actual inference phases. Gradient prior information is extracted from the corresponding retinal images and input into the model along with the preprocessed retinal images for image processing. However, gradient prior information extraction does not include the subsequently augmented input images; it only applies to the preprocessed input images before augmentation.

[0042] It is worth mentioning that the image preprocessing and gradient information extraction are performed only on retinal images and do not involve the processing of the gold standard images for blood vessel segmentation.

[0043] Step S103: Based on the target retinal image set and the gold standard image set for blood vessel segmentation corresponding to the target retinal image set, construct a target retinal blood vessel segmentation dataset, and perform data augmentation processing on the training set and validation set in the target retinal blood vessel segmentation dataset; The target retinal vessel segmentation dataset refers to a standardized and complete dataset constructed by pairing and integrating the preprocessed target retinal image set with the corresponding gold standard image set for vessel segmentation, which is used for model training, validation and testing.

[0044] Since the sample size of retinal image datasets is usually small, data augmentation processing is required for the training and validation sets to increase the amount of training data and prevent model overfitting. The data augmentation strategies adopted in this application include, but are not limited to, the following four: (1) random cropping: randomly cropping the image into 64×64 image blocks; (2) random horizontal flipping: flipping the image horizontally with a 50% probability; (3) random vertical flipping: flipping the image vertically with a 50% probability; (4) random rotation: randomly rotating the image within the range of 0° to 360°. After the above data augmentation processing, a large number of diverse training samples can be generated for the training and validation of the segmentation model.

[0045] It should be noted that during this data augmentation process, the target retinal image after image preprocessing and its corresponding gold standard image for vascular segmentation need to be augmented simultaneously to ensure data consistency.

[0046] Step S104: Input the amplified training set and the prior information of the training gradient into the pre-constructed original multi-scale Mamba network for end-to-end training to obtain a candidate multi-scale Mamba network. Please see Figure 2 , Figure 2 This is a schematic diagram of a gradient-guided multi-scale Mamba network provided in an embodiment of this application. This application proposes a gradient-guided multi-scale Mamba network, named GMM-Net, whose overall architecture is as follows: Figure 2 As shown, H represents the height of the image (number of pixels); W represents the width of the image (number of pixels); and 1 represents the number of channels (here, a single channel for a grayscale image). For encoder output, ; The intermediate features of the i-th layer of the decoder are obtained by upsampling the features output by the Gradient-guided Lightweight Transformer (GLT) module. The decoder outputs the i-th layer features after upsampling the features output by the GLT module. , Corresponding to Figure 2 In the diagram, D1, D2, D3, and D4 represent the gradient prior; S represents the gradient prior G; and the features are... The input is fed into the GSF module and fused with the gradient prior information G to obtain the output features of the GSF module. , This multi-scale Mamba network employs an encoder-decoder symmetric structure. First, the input image undergoes image preprocessing in step S102 (including grayscale conversion and intensity normalization) and Sobel gradient prior extraction, yielding a preprocessed grayscale image and its corresponding gradient magnitude map G. Subsequently, the grayscale image and gradient magnitude map G are fed into an encoder consisting of four serial multi-scale state modules (MSMs). Each MSM is followed by a downsampling operation to progressively extract multi-scale local and global vascular features, outputting the features. The highest-level feature output by the encoder (the highest-level feature is the output feature of the lowest-level encoder MSM_4). In the code when When entering the GLT module, denoted as X, it enters the Gradient Guided Lightweight Transformer (GLT) module of the bottleneck layer together with the gradient magnitude map G. This module models the global vascular topology through low-rank self-attention and gradient modulation mechanisms. The decoder part includes upsampling operations, a gated skip fusion (GSF) module, and a multi-scale feature bridge (MFB) at the end: in each decoding stage, the GSF utilizes upsampled features A pixel-level gated map is generated, and the corresponding layer encoder features are selectively modulated and then fused with the upsampled features to obtain the output features. Multi-scale features in all decoding stages Finally, the gradient magnitude map G is fed into the MFB (Medium-Fuse Booster) and adaptively weighted fusion is used to generate structurally enhanced fusion features. The final fusion features output from the MFB are then convolved to obtain a vessel segmentation probability map with the same resolution as the input.

[0047] For the multi-scale state module design in the encoder stage: please refer to... Figure 3 , Figure 3 This is a schematic diagram of the structure of a multi-scale state module provided in an embodiment of this application, such as... Figure 3 As shown, the Multi-Scale State Module (MSM) aims to simultaneously model the local texture details and global topology of retinal vessels. To address the challenges of small vessel size, diverse orientations, and blurred boundaries, the MSM employs a dual-path structure consisting of local convolutional branches and a global state space branch, and introduces a Sobel gradient prior to enhance its structural awareness of vessel boundaries. Specifically, the MSM first processes the input gradient prior map G (where G is in...) Figure 2-5 The input features (represented as S) undergo a gating transformation, followed by a Conv1×1 convolution and a Sigmoid activation function to generate spatial modulation weights for the two branches. Based on these generated spatial modulation weights, the input features are then processed. Weighted summaries are applied to obtain feature representations that emphasize local detail enhancement and global structure modeling, respectively. The local branch extracts fine-grained texture and edge information through double Conv3×3 convolution, batch normalization (BN), and ReLU linear activation function operations, while the global branch uses the Mamba-based selective state space mechanism (SSM) to model the long-range dependencies of the vascular network. Subsequently, the features from the two branches are adaptively weighted and fused to obtain a multi-scale representation that combines local details and global semantics. The multi-scale representation corresponding to four serial multi-scale state modules (MSMs) It can be represented as .

[0048] Reshape is an operation to restore the image dimension. Specifically, it involves adaptively weighting and fusing the features from the two branches, and then rearranging the shape of the fused features to restore their spatial dimension, thus obtaining a multi-scale feature representation.

[0049] To achieve efficient global modeling and enhance vascular boundary perception at the network bottleneck layer, this application proposes a gradient-guided lightweight Transformer module (GLT), which sits between the MSM-based encoder and decoder. (See [link to relevant documentation]). Figure 4 , Figure 4 This is a schematic diagram of the structure of a gradient-guided lightweight Transformer module provided in an embodiment of this application. Its structure is as follows: Figure 4 As shown, specifically, the Gradient Guided Lightweight Transformer (GLT) module receives the feature map output from the bottom-most encoder MSM_4 of the MSM encoder. (Right now and the pre-computed gradient prior map As input. First, the input features... Perform 3×3 convolution ( To enhance the local spatial representation, the transformed features are obtained. Then, the transformed features With gradient prior graph Feed into the structure-aware attention unit: This structure-aware attention unit processes the transformed features Perform linear projection to obtain Q, K, and V, and then use the low-rank projection matrix. , and Q, K, and V are mapped to a low-rank subspace to reduce computational complexity. Subsequently, based on the features mapped to the low-rank subspace, a low-rank multi-head self-attention is computed, and the attention output is compared with the gradient prior activated by the Sigmoid function. (G) performs element-wise modulation to guide attention to the edges and slender structures of blood vessels; finally, the modulated features are sequentially passed through a feedforward network (FFN) and residual connections with the transformed features. Add them together to generate the final output representation of GLT. .

[0050] In U-Net-like network architectures, skip connections are used to pass high-resolution spatial information from the encoder to the decoder. However, directly fusing shallow encoded features easily introduces background noise and irrelevant textures, thereby affecting high-level semantic representation and reducing the accuracy of vascular structure reconstruction. To address this, this application proposes a gated skip fusion module (GSF, e.g., Figure 2As shown in the GatedSkip Fusion (GSF) diagram, it is embedded in the skip connection structure. Its core idea is to utilize the high-level semantic information generated during the decoder upsampling process to generate pixel-level gating weights, selectively modulating the encoder features, thus transforming the skip connection from a simple feature transfer process into a semantically guided detail filtering process. Specifically, the GSF module receives a pair of features as input: the low-level features output by the encoder... and corresponding high-level features of the decoder (For example, the output characteristics of MSM_1) and upsampling features The output is a pair). For example... Figure 2 As shown, in the GSF module, firstly, for conduct Convolution is used to achieve channel alignment, resulting in the transformed encoder features. Simultaneously, for upsampling features conduct Convolution is performed and a spatial gating map is generated using the sigmoid activation function. The space gating diagram Transformed encoder features Pixel-by-pixel modulation is performed to suppress background noise while preserving key features with structural information; finally, the modulated result is... and After performing Concat, the final result is obtained through Conv convolution. , Through the aforementioned gated cross-layer fusion mechanism, GSF can effectively filter irrelevant textures while maintaining high-resolution spatial information, making skip connections more focused on vascular regions, thereby enhancing the decoder's ability to recover small blood vessels and boundary structures.

[0051] To mitigate the degradation of detail and blurring of boundaries during upsampling, this application designs a multi-scale feature bridge (MFB module) at the decoder end. Please refer to [link to relevant documentation]. Figure 5 , Figure 5 This is a schematic diagram of a multi-scale feature bridge provided in an embodiment of this application, such as... Figure 5 As shown, the MFB module is used to implement structure-guided adaptive multi-scale feature fusion. This module uses feature maps from four decoding stages. ( Corresponding to Figure 5 (F1, F2, F3, F4) and Sobel-based gradient priors As input. First, through upsampling operation and combined Convolutional pairs decode features at each layer By aligning spatial resolution with channel dimension, a unified feature representation is obtained. Subsequently, a structure-guided weight learning unit is constructed to learn each alignment feature. Global average pooling (GAP) is performed on both the gradient prior map G and the gradient prior map G. The representations obtained by global average pooling (GAP) are then concatenated and input into a lightweight multilayer perceptron (MLP). After Softmax normalization, adaptive fusion weights are obtained. Finally, based on the learned adaptive fusion weights... Multiscale features Perform weighted fusion, and through 1. Convolution generates the final output features .

[0052] Step S105: Validate the candidate multi-scale Mamba network based on the expanded validation set, and evaluate the performance of the validated candidate multi-scale Mamba network based on the test set in the target retinal vessel segmentation dataset to obtain the target multi-scale Mamba network. Specifically, the augmented training set images are input into the constructed GMM-Net for end-to-end training. The loss function used is binary cross-entropy loss, which measures the pixel-level difference between the predicted probability map and the true label. The optimizer is Adam, with an initial learning rate of 1×10⁻⁴, a batch size of 16, and a total of 100 training epochs. During training, the learning rate is decayed to 0.1 times its original value every 30 epochs to facilitate model convergence. The validation set is used to monitor model performance, and the model parameters with the highest F1 score on the validation set are saved as the final model. The test set images are input into the trained GMM-Net, and a blood vessel probability map is obtained through forward propagation. For images predicted using the sliding window strategy, the probability maps of each window are stitched together at their original positions, and the average value of the overlapping areas is used to obtain the blood vessel probability map. Finally, a threshold of 0.5 is set to binarize the blood vessel probability map, resulting in the final binary image of blood vessel segmentation. This result can be used for subsequent clinical indicator calculations and disease-aided diagnosis.

[0053] In the model validation and testing phase, the corresponding gradient prior information is also required for model validation and performance evaluation. That is, the gradient prior information is used as part of the model input, not only in the training phase, but also in the validation, testing and subsequent actual inference phases. The gradient prior information is extracted from the corresponding retinal image and input into the model together with the image preprocessed retinal image for image processing. However, the extraction of gradient prior information does not include the amplified input image, but only the input image after image preprocessing before amplification.

[0054] It is understandable that in step S105, the amplified validation set and the corresponding gradient prior information can be input into the candidate multi-scale Mamba network for model validation. Furthermore, the test set in the target retinal vessel segmentation dataset and its corresponding gradient prior information can be input into the candidate multi-scale Mamba network that has been validated for performance evaluation, and finally the target multi-scale Mamba network is obtained. Step S106: Obtain the retinal image to be segmented after image preprocessing, and input the retinal image to be segmented and the gradient prior information corresponding to the retinal image to be segmented into the target multi-scale Mamba network for inference, and output the retinal image segmentation result.

[0055] The target multi-scale Mamba network consists of an encoder, a bottleneck layer, and a decoder. The encoder includes four serial multi-scale state modules, the bottleneck layer includes a gradient-guided lightweight Transformer module, and the decoder includes an upsampling module, a gated jump fusion module, and a multi-scale feature bridge.

[0056] In some embodiments, step S106 may include: using four serial multi-scale state modules, combined with gradient prior information corresponding to the retinal image to be segmented, to model the local texture details and global topological structure of retinal vessels in the retinal image to be segmented, generating four different levels of multi-scale feature representations; using a gradient-guided lightweight Transformer module, combined with gradient prior information corresponding to the retinal image to be segmented, to perform global guidance and vessel boundary enhancement processing on the fourth-layer multi-scale feature representation, obtaining a gradient-guided feature representation; using an upsampling module, to perform an upsampling operation on the gradient-guided feature representation, obtaining four different levels of upsampled output features; using a gated skip fusion module, to perform gated cross-layer fusion processing on the four different levels of multi-scale feature representations and the four different levels of upsampled output features, obtaining four different levels of cross-layer fusion features; using a multi-scale feature bridge, combined with gradient prior information corresponding to the retinal image to be segmented, to perform adaptive multi-scale feature fusion processing on the four different levels of cross-layer fusion features, outputting the retinal image segmentation result.

[0057] In some specific embodiments, the steps of modeling the local texture details and global topological structure of retinal vessels in the retinal image to be segmented by using four serial multi-scale state modules, combined with the gradient prior information corresponding to the retinal image to be segmented, and generating four different levels of multi-scale feature representations, may include: using the first-layer multi-scale state module in the encoder to perform a gated transformation on the gradient prior information corresponding to the retinal image to be segmented, generating a first local spatial modulation weight and a first global spatial modulation weight; using the first-layer multi-scale state module, combining the first local spatial modulation weight and the first global spatial modulation weight, weighting the retinal image to be segmented to obtain a first local feature representation and a first global feature representation, and then fusing the first local feature representation and the first global feature representation through adaptive weighting to obtain a first-layer multi-scale feature representation; using the second-layer multi-scale state module in the encoder to perform a gated transformation on the gradient prior information corresponding to the retinal image to be segmented, generating a second local spatial modulation weight and a second global spatial modulation weight; using the second-layer multi-scale state module, combining the second local spatial modulation weight and the second global spatial modulation weight, weighting the first-layer multi-scale feature representation to obtain a second local feature representation. The second local feature representation and the second global feature representation are fused together by adaptive weights to obtain the second multi-scale feature representation. The third multi-scale state module in the encoder performs a gated transformation on the gradient prior information corresponding to the retinal image to be segmented, generating third local spatial modulation weights and third global spatial modulation weights. The third multi-scale state module, combined with the third local spatial modulation weights and the third global spatial modulation weights, performs a weighted processing on the second multi-scale feature representation to obtain the third local feature representation and the third global feature representation. The third local feature representation and the third global feature representation are then fused together by adaptive weights to obtain the third multi-scale feature representation. The fourth multi-scale state module in the encoder performs a gated transformation on the gradient prior information corresponding to the retinal image to be segmented, generating fourth local spatial modulation weights and fourth global spatial modulation weights. The fourth multi-scale state module, combined with the fourth local spatial modulation weights and the fourth global spatial modulation weights, performs a weighted processing on the third multi-scale feature representation to obtain the fourth local feature representation and the fourth global feature representation. The fourth local feature representation and the fourth global feature representation are then fused together by adaptive weights to obtain the fourth multi-scale feature representation.

[0058] In some specific embodiments, the step of performing global guidance and vessel boundary enhancement processing on the fourth-layer multi-scale feature representation through a gradient-guided lightweight Transformer module, combined with the gradient prior information corresponding to the retinal image to be segmented, to obtain a gradient-guided feature representation, may include: performing a convolution operation on the fourth-layer multi-scale feature representation through the convolution module in the gradient-guided lightweight Transformer module to obtain convolutional transformation features; performing linear projection on the convolutional transformation features through the structure-aware attention unit in the gradient-guided lightweight Transformer module to obtain linear projection features, and mapping the linear projection features using a low-rank projection matrix to obtain low-rank mapping features; calculating low-rank multi-head self-attention based on the low-rank mapping features through the structure-aware attention unit, and modulating the low-rank multi-head self-attention element-wise with the gradient prior information corresponding to the retinal image to be segmented after activation by the Sigmoid function to obtain gradient-modulated features; performing feedforward mapping processing on the gradient-modulated features through the feedforward network in the gradient-guided lightweight Transformer module, and adding the feedforward-mapped features with the convolutional transformation features through residual connections to obtain the gradient-guided feature representation.

[0059] In some specific embodiments, the step of performing gated cross-layer fusion processing on four different levels of multi-scale feature representations and four different levels of upsampled output features through a gated skip fusion module to obtain four different levels of cross-layer fused features may include: performing convolution operations on the four different levels of multi-scale feature representations to obtain four encoded transform features; performing convolution operations on the four different levels of upsampled output features to obtain four decoded transform features; performing activation mapping processing on the four decoded transform features using a sigmoid activation function to generate four spatial gating maps; performing pixel-wise modulation on the corresponding encoded transform features through each spatial gating map to obtain four modulated features; concatenating each modulated feature with the corresponding upsampled output feature to obtain four feature concatenation results; and performing convolution operations on the four feature concatenation results to obtain four different levels of cross-layer fused features.

[0060] In some specific embodiments, the step of using a multi-scale feature bridge, combined with gradient prior information corresponding to the retinal image to be segmented, to perform adaptive multi-scale feature fusion processing on four different levels of cross-layer fusion features and output the retinal image segmentation result may include: performing upsampling and convolution operations on the four different levels of cross-layer fusion features through the multi-scale feature bridge to obtain four unified-dimensional fusion feature representations; performing global average pooling on the four unified-dimensional fusion feature representations through the weight learning unit in the multi-scale feature bridge to obtain four global average pooling features; concatenating the four global average pooling features to obtain global concatenated features, and inputting the global concatenated features into a lightweight multilayer perceptron in the multi-scale feature bridge, normalizing the global concatenated features through the lightweight multilayer perceptron to obtain four adaptive fusion weights; performing weighted fusion on the four unified-dimensional fusion feature representations according to the four adaptive fusion weights, and performing convolution operations on the weighted fused features to generate the retinal image segmentation result.

[0061] It should be noted that the image preprocessing and gradient prior information extraction processes involved in the actual inference process in step S106 are the same as those in step S102. Furthermore, the module processing processes of the multi-scale state module in the encoder, the gradient-guided lightweight Transformer module in the bottleneck layer, the upsampling module, the gated jump fusion module, and the multi-scale feature bridge in the decoder are all the same as those described in step S104. Therefore, the detailed description of each step in the actual inference process can be found in the relevant content of the aforementioned embodiments. The content obtained by each processing step of each module in the actual inference process also corresponds one-to-one with the steps of the aforementioned embodiments, and will not be elaborated here.

[0062] Steps S101 to S106 as illustrated in the embodiments of this application, by extracting and introducing training gradient prior information, can enhance the edge and fine branch features of retinal vessels, suppress background interference, and improve the integrity and accuracy of small vessel segmentation. Furthermore, by synchronously inputting the gradient prior information and the preprocessed image into a multi-scale Mamba network for end-to-end training, deep fusion of structural priors and multi-scale features can be achieved, taking into account both the global topological structure and local texture details of blood vessels, further improving image segmentation accuracy. Moreover, by augmenting the training and validation sets, sample diversity can be expanded without increasing acquisition costs, effectively improving the model's generalization ability and robustness. In short, the gradient-guided multi-module collaborative segmentation framework constructed in this application can significantly improve the ability to recognize and segment blood vessel structures.

[0063] To explain in detail the principles of the technical solution of this application, the overall process of this application will be described below with reference to some specific embodiments. It is easy to understand that the following is an explanation of the technical principles of this application and should not be regarded as a limitation of this application.

[0064] Please see Figure 6 , Figure 6 This is a schematic diagram of the overall process of a gradient-guided image segmentation method provided in an embodiment of this application, as shown below. Figure 6 As shown, the overall implementation flow of a gradient-guided image segmentation method provided in this application includes, but is not limited to, steps S601-S608: Step S601, Obtain and partition the dataset: In the specific implementation, firstly, a dataset for retinal vessel segmentation is obtained, in which each sample contains an original retinal image and its corresponding gold standard image for vessel segmentation; then, the dataset is divided into a training set, a validation set, and a test set according to a preset ratio, with the ratio of 70% training set, 15% validation set, and 15% test set, and ensuring that the image distribution of each subset is consistent.

[0065] Step S602, Image preprocessing: In practical implementation, preprocessing all raw retinal images reduces image noise and improves the contrast between blood vessel pixels and background pixels, thereby enhancing the segmentation accuracy of the deep learning model. It is understood that the training set, validation set, and test set all use the same image preprocessing method, and the subsequent image preprocessing process executed during actual inference follows the same principle. Specifically, the image preprocessing process includes the following sub-steps S6021-S6025: Sub-step S6021, grayscale processing: The grayscale conversion method used in this application is to extract the green channel (G channel) from the RGB image because the contrast between the blood vessel outline and the background is highest and the noise is lowest in the green channel. The grayscale conversion formula is shown below: ; in, This indicates the coordinate system of the grayscale image obtained after grayscale conversion. grayscale value at that location This represents the coordinates of a pixel in the image. This indicates the coordinates of the green channel of the original RGB image. The pixel value at that location.

[0066] Sub-step S6022, standardized operation: Specifically, the overall grayscale value of the image after grayscale conversion in sub-step S6021 is scaled to prevent extreme pixel values ​​in the image data and to alleviate the uneven illumination problem in retinal images. The standardization formula is shown below: ; in, The standardized image pixel values, These are the image pixel values ​​before standardization. The mean of the image pixel values. The standard deviation of the image pixel values. and The calculation formula is shown below: ; ; in, The width of the image (number of pixel columns). The height of the image (number of pixel rows). For the row index of the image, Column index for the image.

[0067] Sub-step S6023, Adaptive histogram equalization: Specifically, adaptive histogram equalization is performed on the image after standardization in sub-step S6022. This aims to stretch the grayscale histogram of the image to improve the image contrast in local areas, while limiting the extremely dark and bright areas in the image to avoid noise amplification caused by over-enhancement.

[0068] Sub-step S6024, Gamma correction: Specifically, gamma correction is applied to the image after adaptive histogram equalization in sub-step S6023 to suppress phenomena such as uneven illumination and reflections along the central line of blood vessels in the retinal image. The gamma correction formula used in this application is shown below: ; in, The input pixel value, The value is the corrected pixel value; in this application, variable A is set to 1 and variable γ is set to 0.8. The combination of the parameters of variable A and variable γ can effectively suppress uneven illumination while maintaining the integrity of the blood vessel structure.

[0069] Sub-step S6025, normalization processing: Finally, the retinal image data after gamma correction in sub-step S6024 is normalized, and the gray levels of the image are linearly mapped from 0 to 255 to between 0 and 1, so as to facilitate the subsequent calculation and processing of image data by the deep learning model.

[0070] Step S603, Gradient prior information extraction: After image preprocessing, this application first extracts structural prior information from the preprocessed retinal image to provide explicit vascular boundary guidance for subsequent networks. Specifically, for the preprocessed input image... The gradient response is calculated using the Sobel operator: First, the input images are calculated respectively. Gradients in the horizontal and vertical directions: ; in, The response is a gradient in the horizontal direction. The vertical gradient response, The horizontal kernel of the Sobel operator. The kernel of the Sobel operator is located in the vertical direction. Then, based on the gradient responses in both directions, the gradient magnitude map is calculated. : ; Finally, the resulting gradient magnitude map Linearly normalized to the [0,1] interval, serving as a fixed, unlearnable structural prior (the normalized structural prior is also used...). (Represented by...). This gradient map contains significant responses at the vessel boundaries and will be used in multiple modules of the subsequent network for feature modulation, attention guidance, and multi-scale fusion.

[0071] Step S604, Data Augmentation: Since retinal image datasets typically have a small sample size, data augmentation is required for both training and validation sets to increase the amount of training data and prevent model overfitting. The data augmentation strategies employed in this application include, but are not limited to, the following four: (1) Random cropping: randomly crop the image into 64×64 image blocks; (2) Random horizontal flip: The image is flipped horizontally with a 50% probability; (3) Random vertical flip: The image is flipped vertically with a 50% probability; (4) Random rotation: Randomly rotate the image within the range of 0° to 360°; After the above data augmentation process, a large number of diverse training samples can be generated for the training and validation of the segmentation model.

[0072] Step S605: Construct a gradient-guided multi-scale Mamba network: This application proposes a gradient-guided multi-scale Mamba network, named GMM-Net, whose overall architecture is as follows: Figure 2 As shown, this multi-scale Mamba network employs an encoder-decoder symmetric structure. First, the input image undergoes image preprocessing in step S602 (including grayscale conversion and intensity normalization) and Sobel gradient prior extraction in step S603, yielding a preprocessed grayscale image and its corresponding gradient magnitude map G. Subsequently, the grayscale image and gradient magnitude map G are fed into an encoder consisting of four serial multi-scale state modules (MSMs). Each MSM is followed by a downsampling operation to progressively extract multi-scale local and global vascular features, outputting the features. The highest-level feature output by the encoder (the highest-level feature is the output feature of the lowest-level encoder MSM_4). In the code when When entering the GLT module, denoted as X, it enters the Gradient-Guided Lightweight Transformer (GLT) module of the bottleneck layer together with the gradient magnitude map G. This module models the global vascular topology through low-rank self-attention and gradient modulation mechanisms. The decoder part includes upsampling operations, a gated cross-layer feature fusion (GSF) module, and a multi-scale feature bridge (MFB) at the end: In each decoding stage, the GSF uses the upsampled features to generate a pixel-level gated map, selectively modulates the corresponding layer encoder features, and then fuses them with the upsampled features to obtain the output features. Multi-scale features in all decoding stages Finally, the gradient magnitude map G is fed into the MFB, and structurally enhanced fusion features are generated through adaptive weighted fusion. The fusion features output by the MFB are then convolved to obtain a vessel segmentation probability map with the same resolution as the input. This application proposes a gradient-guided multi-scale Mamba network with the following sub-steps S6051-S6054: Sub-step S6051, multi-scale state module design for the encoder stage: like Figure 3 As shown, the Multiscale State Module (MSM) aims to simultaneously model the local texture details and global topology of retinal vessels. To address the challenges of small vessels, diverse orientations, and blurred boundaries, the MSM employs a dual-path structure consisting of local convolutional branches and a global state space branch, and introduces a Sobel gradient prior to enhance its structural awareness of vessel boundaries.

[0073] Specifically, the multi-scale state module (MSM) first processes the input gradient prior map G (the gradient prior map G in...) Figure 2-5 The input features (represented as S) undergo a gating transformation, followed by a Conv1×1 convolution and a Sigmoid activation function to generate spatial modulation weights for the two branches. Based on these generated spatial modulation weights, the input features are then processed. Weighted summaries are applied to obtain feature representations that emphasize local detail enhancement and global structure modeling, respectively. The local branch extracts fine-grained texture and edge information through double Conv3×3 convolution, batch normalization (BN), and ReLU linear activation function operations, while the global branch uses the Mamba-based selective state space mechanism (SSM) to model the long-range dependencies of the vascular network. Subsequently, the features from the two branches are adaptively weighted and fused to obtain a multi-scale representation that combines local details and global semantics. The multi-scale representation corresponding to four serial multi-scale state modules (MSMs) It can be represented as .

[0074] The complete calculation process of MSM is shown in the following formula: ; ; = ; in, The gradient prior map is the input. and This represents two 1×1 convolutional layers, and their parameters are not shared. express An activation function is used to constrain the weights within the interval [0,1]. Spatial weighting of local convolutional branches is used to guide the network to focus on the edges and texture details of blood vessels; Spatial modulation weights for the global Mamba branch are used to guide the network to focus on the overall topology and long-range dependencies of the blood vessels; The feature transformation function represents the local convolution branch; Represents the global The characteristic transformation function of the branch; Represents an adaptive fusion weight, in the form of a learnable scalar ( Implemented in the form of ), used to balance the contributions of the two branches to the final feature representation; The feature map is input to the multi-scale state module (MSM); This provides a multi-scale representation of the output of the Multi-Scale State Module (MSM), which combines local details with global semantics.

[0075] Sub-step S6052, gradient-guided lightweight Transformer module design for the bottleneck layer: To achieve efficient global modeling and enhance vascular boundary perception at the network bottleneck layer, this application proposes a gradient-guided lightweight Transformer module (GLT). This module is located between the MSM-based encoder and decoder, and its structure is as follows: Figure 4 As shown, specifically, the Gradient Guided Lightweight Transformer (GLT) module receives the feature map output from the bottom-most encoder MSM_4 of the MSM encoder. (Right now and the pre-computed gradient prior map As input. First, the input features... Perform 3×3 convolution ( To enhance the local spatial representation, the transformed features are obtained. Then, the transformed features With gradient prior graph Feed into the structure-aware attention unit: This structure-aware attention unit processes the transformed features Perform linear projection to obtain Q, K, and V, and then use the low-rank projection matrix. , and Q, K, and V are mapped to a low-rank subspace to reduce computational complexity. Subsequently, based on the features mapped to the low-rank subspace, a low-rank multi-head self-attention is computed, and the attention output is compared with the gradient prior activated by the Sigmoid function. (G) performs element-wise modulation to guide attention to the edges and slender structures of blood vessels; finally, the modulated features are sequentially passed through a feedforward network (FFN) and residual connections with the transformed features. Add them together to generate the final output representation of GLT. The calculation process of GLT is as follows: ; ; ; Where Q, K, and V are respectively derived from the input features The attention query, key, and value vectors are obtained through linear projection. , and Let Q, K, and V be the low-rank projection matrices; X be the output features from the lowest-level encoder MSM_4. In the code when When entering the GLT module, it is represented as X; This is a low-rank attention computation unit in the gradient-guided lightweight Transformer module, used for processing input features. With gradient prior graph Joint processing is performed to generate attention features focused on the vessel edges. ; It is the final output feature of the GLT module, which takes into account global modeling capabilities, local detail perception and blood vessel edge focusing characteristics.

[0076] Sub-step S6053, design of the gated jump fusion module for decoder jump connection: In U-Net-like network architectures, skip connections are used to pass high-resolution spatial information from the encoder to the decoder. However, directly fusing shallow encoded features easily introduces background noise and irrelevant textures, thereby affecting high-level semantic representation and reducing the accuracy of vascular structure reconstruction. To address this, this application proposes a gated skip fusion module (GSF, e.g., Figure 2 As shown in the GatedSkip Fusion, it is embedded in the skip connection structure. Its core idea is to use the high-level semantic information generated during the decoder upsampling process to generate pixel-level gating weights, selectively modulate the encoder features, and transform the skip connection from a simple feature transfer to a semantically guided detail filtering process.

[0077] Specifically, the GSF module receives a pair of features as input: the low-level features output by the encoder. and corresponding high-level features of the decoder (For example, the output characteristics of MSM_1) and upsampling features The output is a pair). For example... Figure 2 As shown, in the GSF module, firstly, for conduct Convolution is used to achieve channel alignment, resulting in the transformed encoder features. Simultaneously, for upsampling features conduct Convolution is performed and a spatial gating map is generated using the sigmoid activation function. The space gating diagram Transformed encoder features Pixel-by-pixel modulation is performed to suppress background noise while preserving key features with structural information; finally, the modulated result is... and After performing Concat, the final result is obtained through Conv convolution. , .

[0078] Through the aforementioned gated cross-layer fusion mechanism, GSF can effectively filter irrelevant textures while maintaining high-resolution spatial information, making skip connections more focused on vascular regions, thereby enhancing the decoder's ability to recover small blood vessels and boundary structures. The specific calculation process of GSF is as follows: ; ; Sub-step S6054, design of the multi-scale feature bridge module at the output end: To mitigate the degradation of detail and blurring of boundaries during upsampling, this application designs a multi-scale feature bridge (MFB) at the decoder end. Figure 5 As shown in the figure, this module is used to implement structure-guided adaptive multi-scale feature fusion. It utilizes feature maps from four decoding stages. and gradient priors based on Sobel As input, where Corresponding to Figure 5 The numbers are F1, F2, F3, and F4.

[0079] First, through upsampling operation and combined Convolutional pairs decode features at each layer By aligning spatial resolution with channel dimension, a unified feature representation is obtained. Subsequently, a structure-guided weight learning unit is constructed to learn each alignment feature. Global average pooling (GAP) is performed on both the gradient prior map G and the gradient prior map G. The representations obtained by global average pooling (GAP) are then concatenated and input into a lightweight multilayer perceptron (MLP). After Softmax normalization, adaptive fusion weights are obtained. Finally, based on the learned adaptive fusion weights... Multiscale features Perform weighted fusion, and through 1. Convolution generates the final output features The specific calculation process is as follows: ; ; ; in, Indicates an upsampling operation. Used for channel alignment. Indicates global average pooling. This is the adaptive fusion weight vector. Through the multi-scale feature fusion mechanism guided by the above structure, decoded features at different scales can be effectively integrated, thereby improving the reconstruction capability of small vascular structures and enhancing the clarity of vascular boundaries.

[0080] Step S606, Network Training and Optimization: Specifically, the training set images augmented in step S604 and the corresponding gradient prior information are input into the constructed GMM-Net for end-to-end training. The loss function is binary cross-entropy loss, used to measure the pixel-level difference between the predicted probability map and the true label. The optimizer is Adam, with an initial learning rate of 1×10⁻⁴, a batch size of 16, and a total of 100 training epochs. During training, the learning rate is decayed to 0.1 times its original value every 30 epochs to facilitate model convergence. The augmented validation set and the corresponding gradient prior information are input into the trained GMM-Net to monitor model performance, and the model parameters with the highest F1 score on the validation set are saved as the final model.

[0081] Step S607, Test Reasoning: The test set images are input into the trained GMM-Net, and a vessel probability map is obtained through forward propagation. For images predicted using the sliding window strategy, the probability maps of each window are stitched together at their original positions, and the average value of the overlapping areas is taken to obtain the vessel probability map. Finally, a threshold of 0.5 is set to binarize the vessel probability map to obtain the final binary image of vessel segmentation. This result can be used for subsequent clinical indicator calculation and disease-aided diagnosis.

[0082] After training the retinal vessel segmentation model, its performance was evaluated using test set images, and the segmentation results were quantitatively analyzed using various evaluation metrics. Commonly used metrics include accuracy (ACC), sensitivity (SE), specificity (SP), and F1-score, which are calculated using the following formulas: ; ; ; ; ; ; Among these metrics, TP (True Positive) represents pixels that are actually blood vessels and are correctly predicted as such by the model; TN (True Negative) represents pixels that are actually background and are correctly identified as such; FP (False Positive) represents pixels that are actually background but are misclassified as blood vessels; and FN (False Negative) represents pixels that are actually blood vessels but are misclassified as background. Precision represents accuracy, and Recall represents recall. All the above evaluation metrics range from 0 to 1, with higher values ​​indicating better segmentation performance.

[0083] Among them, ACC represents the proportion of correctly predicted pixels out of the total number of pixels, which reflects the overall classification performance of the model; SE (also known as true positive rate) represents the proportion of correctly detected blood vessel pixels out of the real blood vessel pixels, which measures the model's ability to detect blood vessel regions; SP represents the proportion of pixels correctly identified as background out of the real background pixels, which evaluates the model's ability to distinguish background regions; and F1-score is the harmonic mean of Precision and Recall, which is used to comprehensively measure the model's overall performance in the blood vessel detection task.

[0084] In this embodiment, ablation experiments of the gradient-guided image segmentation method on the DRIVE dataset are provided, as shown in Table 1 below. The results for GMM-Net (w / o Grad) are those without gradient prior, while the results for GMM-Net are those of the gradient-guided image segmentation method proposed in this application. As shown in Table 1, the gradient prior method proposed in this application has a positive impact on the segmentation effect, as detailed below: Table 1:

[0085] It should be noted that this embodiment is only a brief illustrative description of the overall process of a gradient-guided image segmentation method. Detailed descriptions of each step can be found in the relevant content of the foregoing embodiments, and will not be repeated here. It is understood that this application does not impose any limitations on this.

[0086] In summary, this application addresses the shortcomings of existing retinal vessel segmentation methods, such as insufficient detection of small vessels, limited global structural modeling capabilities, and inadequate multi-scale feature fusion. It proposes a precise image segmentation method based on gradient prior guidance and multi-scale state modeling. Traditional convolutional neural networks (CNNs) in vessel segmentation tasks primarily rely on local convolution operations, making it difficult to fully utilize the long-range dependencies of vascular structures. Furthermore, they are susceptible to background noise interference during feature fusion, leading to the breakage or blurred boundaries of small vessels. To address these issues, this application improves the network structure from three aspects: structural prior guidance, global dependency modeling, and cross-layer feature fusion, forming a gradient-guided multi-module collaborative segmentation framework, thereby significantly improving the ability to recognize vascular structures and the accuracy of segmentation. The main key points and beneficial effects of this application are as follows.

[0087] (1) Multi-scale state feature modeling mechanism based on gradient prior guidance: This application designs a multi-scale state module (MSM) in the encoding stage, which combines convolutional branches and state space modeling branches to achieve the collaborative extraction of local detail information and global structural information. Specifically, the module uses gradient prior information to generate spatially modulated weights to adaptively enhance the input features, so that the local convolutional branches focus on extracting vascular texture and edge information, while the state space branches are responsible for capturing long-distance dependencies and the overall vascular topology.

[0088] Unlike traditional convolutional networks that rely solely on local receptive fields, this application introduces a state-space modeling mechanism to achieve global structural modeling while maintaining low computational complexity, thereby effectively alleviating the problems of ruptured small blood vessels and structural discontinuities.

[0089] As shown in Table 1, after introducing the MSM module into the DRIVE dataset, the baseline model's sensitivity (SE) increased from 0.7645 to 0.8125, and its F1-score increased from 0.8151 to 0.8233, indicating that the MSM structure can significantly enhance the detection capability of small blood vessels.

[0090] (2) A lightweight global attention modeling mechanism based on gradient guidance: To further enhance the network's ability to perceive the overall structure of blood vessels, this application designs a gradient-guided lightweight Transformer (GLT) module in the network's bottleneck layer. This module combines convolutional feature extraction with a low-rank self-attention mechanism to achieve global semantic modeling while reducing computational complexity. Furthermore, this application utilizes gradient prior information to modulate the attention weights, making the attention mechanism focus more on blood vessel boundaries and slender structural regions, thereby improving the network's response to key structural regions.

[0091] Compared to traditional Transformer structures, this gradient-guided lightweight Transformer module approach not only reduces computational overhead but also improves the ability to express fine-grained vascular structures.

[0092] As shown in Table 1, after adding the GLT module, the model sensitivity (SE) was further improved to 0.8313 and the F1-score was improved to 0.8279, indicating that the mechanism can effectively enhance the network's ability to model the global vascular structure.

[0093] (3) Semantically guided gating cross-layer feature fusion mechanism: In the traditional U-Net architecture, skip connections between the encoder and decoder typically directly transmit shallow features, which easily introduces a large amount of background noise and affects segmentation accuracy. To address this issue, this application proposes a gated cross-layer feature fusion module (GSF).

[0094] This GSF module utilizes high-level semantic information from the decoding stage to generate pixel-level gating weights, selectively modulating encoder features to suppress irrelevant texture information and highlight vascular structural features. Through this semantically guided cross-layer fusion mechanism, noise propagation can be reduced while preserving high-resolution detail information, improving the decoding stage's ability to recover small blood vessels and boundary structures.

[0095] As shown in Table 1, after introducing this module, the model's F1-score improved to 0.8346, further verifying the effectiveness of this structure in feature fusion.

[0096] (4) A multi-scale feature fusion bridging mechanism based on structure awareness: To address the issue of detail loss during upsampling, this application designs a multi-scale feature bridging module (MFB) at the decoder end. This module first spatially and channel-aligns the decoded features at different scales, and then achieves adaptive fusion through a structure-guided weight learning mechanism. Specifically, this application utilizes global average pooling and a lightweight perceptron to calculate the importance weights of features at each scale, and combines this with gradient prior information for joint modulation. This allows the fusion process to simultaneously consider semantic and vascular structure information, thereby obtaining more refined segmentation results.

[0097] As shown in Table 1, the experimental results on the DRIVE dataset show that the sensitivity (SE) of the complete model reaches 0.8404 and the F1-score reaches 0.8498, which are about 7.6% and 3.5% higher than the basic network, respectively, proving that the multi-scale fusion mechanism can effectively improve the reconstruction ability of small blood vessels and the overall segmentation quality.

[0098] (5) Overall performance improvement effect: In this application embodiment, experimental comparison results of the method provided in this application with other related methods on the CHASEDB1, STARE, and HRF datasets are also provided, as shown in Tables 2-4 below: Table 2

[0099] Table 3

[0100] Table 4

[0101] Through the collaborative design of the aforementioned modules, this application constructs a gradient prior-guided multi-scale Mamba retinal vessel segmentation network (GMM-Net). Experimental results in Tables 1-4 demonstrate that this method achieves superior performance compared to related methods on multiple publicly available datasets. For example: On the DRIVE dataset, the F1-score reaches 0.8498; On the CHASE-DB1 dataset, the sensitivity (SE) reaches 0.8686; On the STARE dataset, the F1-score reaches 0.8541; On the HRF dataset, the F1-score reaches 0.8581.

[0102] Experimental results show that this application can significantly improve the detection integrity, structural continuity and boundary accuracy of retinal vessels while ensuring computational efficiency, and has good application value and promotion prospects.

[0103] This application's embodiments, by extracting and introducing training gradient prior information, can enhance the edge and fine branch features of retinal vessels, suppress background interference, and improve the integrity and accuracy of small vessel segmentation. Furthermore, by synchronously inputting gradient prior information and preprocessed images into a multi-scale Mamba network for end-to-end training, deep fusion of structural priors and multi-scale features can be achieved, taking into account both the global topological structure and local texture details of blood vessels, further improving image segmentation accuracy. Moreover, by augmenting the training and validation sets, sample diversity can be expanded without increasing acquisition costs, effectively improving the model's generalization ability and robustness. In short, the gradient-guided multi-module collaborative segmentation framework constructed in this application can significantly improve the ability to recognize and segment blood vessel structures.

[0104] Please see Figure 7 This application also provides a gradient-guided image segmentation device 700 that can implement the above-described method. The device includes the following modules: The original dataset acquisition module 701 is used to acquire the original retinal vessel segmentation dataset; wherein, the original retinal vessel segmentation dataset includes the original retinal image set and the corresponding gold standard image set for vessel segmentation; The image preprocessing module 702 is used to preprocess the original retinal image set to obtain the target retinal image set, and to extract gradient information from the training images in the target retinal image set to obtain training gradient prior information. The data augmentation processing module 703 is used to construct a target retinal blood vessel segmentation dataset based on the target retinal image set and the gold standard image set for blood vessel segmentation corresponding to the target retinal image set, and to perform data augmentation processing on the training set and validation set in the target retinal blood vessel segmentation dataset; The network training module 704 is used to input the amplified training set and the prior information of the training gradient into the pre-constructed original multi-scale Mamba network for end-to-end training to obtain a candidate multi-scale Mamba network. The network validation and testing module 705 is used to perform model validation on the candidate multi-scale Mamba network based on the amplified validation set, and to evaluate the performance of the model-validated candidate multi-scale Mamba network based on the test set in the target retinal vessel segmentation dataset, so as to obtain the target multi-scale Mamba network. The network inference module 706 is used to acquire the retinal image to be segmented after image preprocessing, and input the retinal image to be segmented and the gradient prior information corresponding to the retinal image to be segmented into the target multi-scale Mamba network for inference, and output the retinal image segmentation result.

[0105] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0106] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0107] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0108] Please see Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 801 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 802 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called and executed by the processor 801 using the methods described in the embodiments of this application. The 803 input / output interface is used to implement information input and output. The communication interface 804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 805 transmits information between various components of the device (e.g., processor 801, memory 802, input / output interface 803, and communication interface 804); The processor 801, memory 802, input / output interface 803, and communication interface 804 are connected to each other within the device via bus 805.

[0109] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0110] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0111] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0112] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0113] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0114] This application provides a gradient-guided image segmentation method and related equipment. By extracting and introducing training gradient prior information, it can enhance the edge and fine branch features of retinal vessels, suppress background interference, and improve the integrity and accuracy of small vessel segmentation. Furthermore, by synchronously inputting the gradient prior information and the preprocessed image into a multi-scale Mamba network for end-to-end training, it can achieve deep fusion of structural priors and multi-scale features, taking into account both the global topological structure and local texture details of blood vessels, further improving image segmentation accuracy. In addition, by augmenting the training and validation sets, it can expand the sample diversity without increasing the acquisition cost, effectively improving the model's generalization ability and robustness. In short, the gradient-guided multi-module collaborative segmentation framework constructed in this application can significantly improve the ability to recognize blood vessel structures and the accuracy of segmentation.

[0115] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0116] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0117] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0118] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0119] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0120] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0122] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0124] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0125] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A gradient-guided based image segmentation method, characterized in that, The method includes the following steps: Obtain the original retinal vessel segmentation dataset; wherein, the original retinal vessel segmentation dataset includes the original retinal image set and the corresponding gold standard image set for vessel segmentation; The original retinal image set is preprocessed to obtain the target retinal image set, and the training images in the target retinal image set are subjected to gradient information extraction processing to obtain training gradient prior information. Based on the target retinal image set and the corresponding gold standard image set for vessel segmentation, a target retinal vessel segmentation dataset is constructed, and the training and validation sets in the target retinal vessel segmentation dataset are subjected to data augmentation processing. The amplified training set and the prior information of the training gradient are input into the pre-constructed original multi-scale Mamba network for end-to-end training to obtain the candidate multi-scale Mamba network. The candidate multi-scale Mamba network is validated based on the expanded validation set, and the performance of the validated candidate multi-scale Mamba network is evaluated based on the test set in the target retinal vessel segmentation dataset to obtain the target multi-scale Mamba network. The preprocessed retinal image to be segmented is obtained, and the retinal image to be segmented and the gradient prior information corresponding to the retinal image to be segmented are input into the target multi-scale Mamba network for inference, and the retinal image segmentation result is output.

2. The method according to claim 1, characterized in that, The step of preprocessing the original retinal image set to obtain the target retinal image set includes: The original retinal image set is converted to grayscale to obtain a grayscale image set; The grayscale image set is standardized to obtain a standardized image set; The standardized image set is subjected to adaptive histogram equalization to obtain an adaptively enhanced image set; The adaptively enhanced image set is subjected to gamma correction processing to obtain a gamma-corrected image set; The gamma-corrected image set is normalized to obtain the target retinal image set.

3. The method according to claim 1, characterized in that, The step of extracting gradient information from the training images in the target retinal image set to obtain prior training gradient information includes: The gradients of the training images in the target retinal image set in the horizontal and vertical directions are calculated respectively to obtain the horizontal gradient and the vertical gradient. Calculate the gradient magnitude map based on the horizontal and vertical gradients. The gradient magnitude map is linearly normalized to obtain the prior information of the training gradient.

4. The method according to claim 1, characterized in that, The target multi-scale Mamba network consists of an encoder, a bottleneck layer, and a decoder. The encoder includes four serial multi-scale state modules. The bottleneck layer includes a gradient-guided lightweight Transformer module. The decoder includes an upsampling module, a gated skip fusion module, and a multi-scale feature bridge. The target multi-scale Mamba network is inferred by inputting the retinal image to be segmented and the corresponding gradient prior information into the network for inference, and outputs the retinal image segmentation result, including: By using four serial multi-scale state modules and combining the gradient prior information corresponding to the retinal image to be segmented, the local texture details and global topological structure of the retinal vessels in the retinal image to be segmented are modeled, generating four different levels of multi-scale feature representations. The gradient-guided lightweight Transformer module, combined with the gradient prior information corresponding to the retinal image to be segmented, performs global guidance and blood vessel boundary enhancement on the fourth layer multi-scale feature representation to obtain the gradient-guided feature representation. The upsampling module performs an upsampling operation on the gradient-guided feature representation to obtain four different levels of upsampled output features. The gated skip fusion module performs gated cross-layer fusion processing on the four different levels of multi-scale feature representations and the four different levels of upsampled output features to obtain four different levels of cross-layer fused features. By using the multi-scale feature bridge and combining the gradient prior information corresponding to the retinal image to be segmented, adaptive multi-scale feature fusion processing is performed on the cross-layer fusion features at four different levels to output the retinal image segmentation result.

5. The method according to claim 4, characterized in that, The process involves using four serial multi-scale state modules, combined with the gradient prior information corresponding to the retinal image to be segmented, to model the local texture details and global topological structure of the retinal vessels in the retinal image to be segmented, generating four different levels of multi-scale feature representations, including: The first multi-scale state module in the encoder performs a gated transformation on the gradient prior information corresponding to the retinal image to be segmented, generating a first local spatial modulation weight and a first global spatial modulation weight. The first-layer multi-scale state module combines the first local spatial modulation weight and the first global spatial modulation weight to perform weighted processing on the retinal image to be segmented, thereby obtaining the first local feature representation and the first global feature representation. The first local feature representation and the first global feature representation are then fused by adaptive weights to obtain the first-layer multi-scale feature representation. The second-layer multi-scale state module in the encoder performs a gated transformation on the gradient prior information corresponding to the retinal image to be segmented, generating a second local spatial modulation weight and a second global spatial modulation weight. By using the second-layer multi-scale state module, the first-layer multi-scale feature representation is weighted by combining the second local spatial modulation weight and the second global spatial modulation weight to obtain the second local feature representation and the second global feature representation. The second local feature representation and the second global feature representation are then fused by adaptive weights to obtain the second-layer multi-scale feature representation. The third-layer multi-scale state module in the encoder performs a gated transformation on the gradient prior information corresponding to the retinal image to be segmented, generating a third local spatial modulation weight and a third global spatial modulation weight. The second-layer multi-scale feature representation is weighted by the third-layer multi-scale state module, combined with the third local spatial modulation weight and the third global spatial modulation weight, to obtain the third local feature representation and the third global feature representation. The third local feature representation and the third global feature representation are then fused by adaptive weights to obtain the third-layer multi-scale feature representation. The fourth multi-scale state module in the encoder performs a gated transformation on the gradient prior information corresponding to the retinal image to be segmented, generating a fourth local spatial modulation weight and a fourth global spatial modulation weight. The third-layer multi-scale feature representation is weighted by the fourth-layer multi-scale state module, which combines the fourth local spatial modulation weight and the fourth global spatial modulation weight to obtain the fourth local feature representation and the fourth global feature representation. The fourth local feature representation and the fourth global feature representation are then fused by adaptive weights to obtain the fourth-layer multi-scale feature representation.

6. The method according to claim 4, characterized in that, The gradient-guided lightweight Transformer module, combined with the gradient prior information corresponding to the retinal image to be segmented, performs global guidance and vessel boundary enhancement processing on the fourth-layer multi-scale feature representation to obtain the gradient-guided feature representation, including: The convolution module in the gradient-guided lightweight Transformer module performs a convolution operation on the fourth layer multi-scale feature representation to obtain the convolutional transformation feature. The structure-aware attention unit in the gradient-guided lightweight Transformer module performs linear projection on the convolutional transformation features to obtain linear projection features, and then maps the linear projection features using a low-rank projection matrix to obtain low-rank mapped features. The structure-aware attention unit calculates low-rank multi-head self-attention based on the low-rank mapping features, and modulates the low-rank multi-head self-attention element-wise with the gradient prior information corresponding to the retinal image to be segmented after activation by the Sigmoid function to obtain gradient modulation features. The gradient-guided feature representation is obtained by performing feedforward mapping on the gradient modulation features through the feedforward network in the gradient-guided lightweight Transformer module, and then adding the feedforward-mapped features to the convolutional transformation features through residual connections.

7. The method according to claim 4, characterized in that, The gated skip fusion module performs gated cross-layer fusion processing on the four different levels of multi-scale feature representations and the four different levels of upsampled output features to obtain four different levels of cross-layer fused features, including: Convolution operations are performed on the four different levels of the multi-scale feature representation to obtain four encoded transformation features; Convolution operations are performed on the upsampled output features at four different levels to obtain four decoded transform features; The four decoding transformation features are activated and mapped using the Sigmoid activation function to generate four spatial gating maps. For each of the spatial gating maps, the corresponding encoded transformation features are modulated pixel by pixel to obtain four modulation features; Each of the modulation features is concatenated with the corresponding upsampled output feature to obtain four feature concatenation results; Convolution operations are performed on the four concatenated feature results to obtain four different levels of cross-layer fusion features.

8. The method according to claim 4, characterized in that, The process involves using the multi-scale feature bridge, combined with the gradient prior information corresponding to the retinal image to be segmented, to perform adaptive multi-scale feature fusion processing on the cross-layer fusion features at four different levels, and outputting the retinal image segmentation result, including: Through the multi-scale feature bridge, upsampling and convolution operations are performed on the cross-layer fusion features at four different levels to obtain a fusion feature representation with four unified dimensions. Through the weight learning unit in the multi-scale feature bridge, global average pooling is performed on the fused feature representations of the four unified dimensions to obtain four global average pooled features. The four global average pooling features are concatenated to obtain global concatenated features, and the global concatenated features are input into the lightweight multilayer perceptron in the multi-scale feature bridge. The lightweight multilayer perceptron normalizes the global concatenated features to obtain four adaptive fusion weights. Based on the four adaptive fusion weights, the fusion feature representations of the four unified dimensions are weighted and fused, and the weighted fused features are convolved to generate the retinal image segmentation result.

9. A gradient-guided image segmentation device, characterized in that, The device includes the following modules: The original dataset acquisition module is used to acquire the original retinal vessel segmentation dataset; wherein, the original retinal vessel segmentation dataset includes the original retinal image set and the corresponding gold standard image set for vessel segmentation; The image preprocessing module is used to preprocess the original retinal image set to obtain the target retinal image set, and to extract gradient information from the training images in the target retinal image set to obtain training gradient prior information. The data augmentation processing module is used to construct a target retinal vessel segmentation dataset based on the target retinal image set and the corresponding gold standard image set for vessel segmentation, and to perform data augmentation processing on the training set and validation set in the target retinal vessel segmentation dataset; The network training module is used to input the amplified training set and the prior information of the training gradient into the pre-constructed original multi-scale Mamba network for end-to-end training to obtain a candidate multi-scale Mamba network. The network validation and testing module is used to perform model validation on the candidate multi-scale Mamba network based on the amplified validation set, and to evaluate the performance of the model-validated candidate multi-scale Mamba network based on the test set in the target retinal vessel segmentation dataset, so as to obtain the target multi-scale Mamba network. The network inference module is used to acquire the retinal image to be segmented after image preprocessing, and input the retinal image to be segmented and the gradient prior information corresponding to the retinal image to be segmented into the target multi-scale Mamba network for inference, and output the retinal image segmentation result.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 8.

Images