A method and device for segmenting a blood vessel plaque of a lower limb

Abstract

The application provides a lower limb blood vessel plaque segmentation method and device, the method comprising: acquiring global space-time feature samples of lower limb CTA image training samples through a DINOv3-ViT large model of a space-time feature extraction module and a time sequence feature extraction model; acquiring several layers of down-sampling convolution feature samples of the lower limb CTA image training samples through a down-sampling feature extraction module; fusing the global space-time feature samples and the down-sampling convolution feature samples of the bottom layer through a feature fusion module to obtain fused feature samples; performing fusion up-sampling processing and classification processing on the fused feature samples and the several layers of down-sampling convolution feature samples through an up-sampling classification module to obtain sample blood vessel plaque segmentation results, which are used to construct a loss function and train a medical image segmentation model through the loss function; during training, LoRA is used to fine-tune the parameters of the DINOv3-ViT large model; and a target lower limb CTA image is input into the trained medical image segmentation model to obtain lower limb blood vessel plaque segmentation results of the target lower limb CTA image.

Description

Technical Field

[0001] This application relates to the technical field of lower limb image segmentation processing, specifically to a method and apparatus for segmenting lower limb vascular plaques. Background Technology

[0002] Peripheral arterial disease (PAD) has a global incidence rate of 15%-20% among the elderly population, and accurate segmentation of lower extremity vascular plaques is crucial for the clinical diagnosis and treatment of PAD. However, while early threshold segmentation combined with region growing algorithms (such as adaptive multi-threshold methods) offers high computational efficiency, it suffers from poor noise resistance, deterioration in low-contrast region segmentation, and an inability to distinguish the fine boundaries of the three types of tissues. Methods based on random forests and conditional random fields (CRFs) rely on manual feature engineering, lacking the ability to model long-range anatomical dependencies in lower extremity vessels and exhibiting poor generalization. While recent visual foundational models have shown application potential due to their large-scale pre-trained general feature representation capabilities, the fusion of existing foundational models with medical segmentation networks is often a simple stitching process, failing to incorporate multiple dimensions for image segmentation and resulting in poor image segmentation accuracy. Summary of the Invention

[0003] The purpose of this application is to overcome the shortcomings and deficiencies of the prior art and to provide a method and apparatus for segmenting lower limb vascular plaques.

[0004] The first aspect of this application provides a method for segmenting lower limb vascular plaques, including: Training samples of lower limb CTA images and a medical image segmentation model are obtained; the medical image segmentation model includes a global spatiotemporal feature extraction module, a downsampling feature extraction module, a feature fusion module, and an upsampling classification module; the global spatiotemporal feature extraction module includes a DINOv3-ViT large model and a temporal feature extraction model; The lower limb CTA image training samples are processed by global semantic extraction using the DINOv3-ViT large model of the spatiotemporal feature extraction module to obtain global semantic feature samples. Then, the global semantic feature samples are processed by temporal feature extraction using the temporal feature extraction model to obtain global spatiotemporal feature samples. The downsampling feature extraction module performs local feature extraction on the lower limb CTA image training samples to obtain several layers of downsampling convolutional feature samples; wherein, the bottom layer of downsampling convolutional feature samples has the same spatial dimension and channel dimension as the global spatiotemporal feature samples. The feature fusion module fuses the global spatiotemporal feature samples and the lowest-level downsampled convolutional feature samples to obtain fused feature samples. The upsampling classification module performs fusion upsampling and classification processing on the fused feature samples and several layers of downsampling convolutional feature samples to obtain the sample vascular plaque segmentation result. A loss function is constructed based on the standard vascular plaque segmentation results corresponding to the training samples of the lower limb CTA images and the vascular plaque segmentation results of the samples. The module parameters of the global spatiotemporal feature extraction module, downsampling feature extraction module, feature fusion module and upsampling classification module of the medical image segmentation model are trained through the loss function. During training, LoRA is used to fine-tune the parameters of the DINOv3-ViT large model. The target lower limb CTA image is input into the trained medical image segmentation model to obtain the lower limb vascular plaque segmentation result of the target lower limb CTA image.

[0005] As one implementation, the DINOv3-ViT large model includes a multi-layer Transformer Encoder, which includes a multi-head self-attention module, a feedforward neural network module, and a residual connection module. The multi-head self-attention module receives sequence features after image block embedding and position encoding. It generates query (Q), key (K), and value (V) vectors through three sets of parallel linear layers. Q, K, and V are uniformly divided into multiple independent attention heads along the feature dimension. The scaled dot product attention weight is calculated independently for each head. After normalization and softmax activation, the weight is multiplied by the corresponding V to obtain the feature output. The outputs of all attention heads are concatenated along the feature dimension and fused into a multi-dimensional sequence feature through a linear projection layer. The feedforward neural network module is used to extract the nonlinear features of the multidimensional sequence features; The residual connection module is used to add and fuse the multidimensional sequence features and the nonlinear features to obtain the global semantic feature sample.

[0006] As one implementation method, during training, several target-independent attention heads of each Transformer Encoder layer are fine-tuned using LoRA.

[0007] As one implementation method, the steps for fine-tuning parameters using LoRA include: Low-rank matrices A and B are embedded in parallel and concatenated to form a LoRA layer in the qkv linear layer of the target independent attention head; low-rank matrices A and B are bypasses composed of two linear layers. During training, the pre-trained weight matrix of the target independent attention head is frozen, and only the low-rank matrix A and the low-rank matrix B are trained. After training, the low-rank matrix A and the low-rank matrix B are multiplied and then combined to form the pre-trained weight matrix as the weight matrix parameters of the target independent attention head after fine-tuning.

[0008] As one implementation method, the step of the temporal feature extraction model performing temporal feature extraction processing on the global semantic feature samples to obtain global spatiotemporal feature samples includes: The temporal feature extraction model calculates the attention weights of several adjacent slices before and after the current slice relative to the current slice using cosine similarity. Then, based on the attention weights, it fuses the features of the several adjacent slices with the features of the current slice to obtain the global spatiotemporal feature samples.

[0009] In one implementation, the downsampling feature extraction module includes an initial convolutional layer and a plurality of downsampling convolutional layers connected in sequence; The initial convolutional layer performs convolution processing on the lower limb CTA image training samples to obtain sample convolutional features; The highest-level downsampling convolutional layer performs downsampling convolution processing on the sample convolutional features to obtain the corresponding downsampling convolutional feature samples. Each of the remaining downsampling convolutional layers performs downsampling convolution on the downsampling convolutional feature samples output by the previous downsampling convolutional layer to obtain the corresponding downsampling convolutional feature samples.

[0010] As one implementation method, the feature fusion module obtains the fused feature samples using the following formula: Fused_feature=gate_weight × DINO_proj + (1 - gate_weight) ×bottleneck_feat; Wherein, Fused_feature is the fused feature sample, gate_weight is the gate weight map generated by the gated network, DINO_proj is the global spatiotemporal feature sample, and bottleneck_feat is the bottom-level downsampled convolutional feature sample.

[0011] In one implementation, the upsampling classification module includes a plurality of upsampling convolutional layers and activation classification units connected in sequence; After the bottom-most upsampling convolutional layer upsamples the fused feature sample, it performs a fusion convolution process with the downsampling convolutional feature sample of the corresponding downsampling convolutional layer to obtain the corresponding upsampled fused feature sample. Each of the remaining upsampling convolutional layers upsamples the upsampled fused feature sample output by the previous upsampling convolutional layer, and then performs a fused convolution with the downsampled convolutional feature sample of the corresponding downsampling convolutional layer to obtain the corresponding upsampled fused feature sample. The activation classification unit performs activation classification processing on the upsampled fusion feature samples output by the highest upsampled convolutional layer to obtain the sample vascular plaque segmentation result.

[0012] As one implementation method, after inputting the target lower limb CTA image into the trained medical image segmentation model to obtain the lower limb vascular plaque segmentation result of the target lower limb CTA image, the method further includes: Based on the number of segmentation mask pixels in the lower limb vascular plaque segmentation results, the mask region area of ​​the blood vessel or plaque in each slice is obtained:

[0013] in, Let be the area of ​​the mask region for blood vessels or plaques on the i-th slice. This represents the number of segmentation mask pixels in the masked region of the corresponding slice within the blood vessel or plaque. It is the pixel spacing of the slice along the x-axis. It is the pixel spacing of the slice along the y-axis; Based on the lower limb vascular plaque segmentation results, obtain the total volume of the blood vessel or plaque:

[0014] in, The total volume of the blood vessel or plaque. The thickness of the slice interval. The total number of slices covering the masked region corresponding to a blood vessel or plaque; Based on the total volume of the plaque and the total volume of the blood vessels, the three-dimensional volumetric parameters of the target lower limb CTA image are obtained:

[0015] in, For three-dimensional volume indicators, The total volume of the patch. This represents the total volume of the blood vessels.

[0016] Compared to related technologies, the medical image segmentation model in this application utilizes LoRA fine-tuning technology during training to perform anatomically-aware fine-tuning of the DINOv3 large model in the global spatiotemporal feature extraction module, thereby improving training efficiency and reducing training costs. Furthermore, in application, it can be combined with cross-slice temporal feature enhancement to fully capture global anatomical semantics and temporal correlations between slices, forming high-quality global spatiotemporal features. Moreover, it employs a downsampling feature extraction module to focus on extracting local spatial details and boundary features as a precise supplement to global semantic features, enabling accurate acquisition of lower limb vascular plaque segmentation results from target lower limb CTA images.

[0017] A second aspect of this application provides a lower limb vascular plaque segmentation device, including a storage device, a processor, and a computer program stored in the storage device and executable by the processor. When the processor executes the computer program, it inputs a target lower limb CTA image into a medical image segmentation model that has been trained as described above, and obtains the lower limb vascular plaque segmentation result of the target lower limb CTA image.

[0018] To provide a clearer understanding of this application, the specific embodiments of this application will be described below in conjunction with the accompanying drawings. Attached Figure Description

[0019] Figure 1 This is a flowchart of a lower limb vascular plaque segmentation method according to an embodiment of this application; Figure 2 This is a schematic diagram of refined annotation labels for lower limb CTA image training samples according to an embodiment of this application; Figure 3 This is a schematic diagram of the architecture of the DINOv3-ViT large model according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a Transformer Encoder according to an embodiment of this application; Figure 5 This is a schematic diagram of the structure of parallel embedded low-rank matrix A and low-rank matrix B according to an embodiment of this application; Figure 6 This is a schematic diagram of feature fusion in the ResUNet architecture; Figure 7 This is a schematic diagram of a medical image segmentation model according to an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a medical image segmentation model according to an embodiment of this application; Figure 9 This is a schematic diagram of the reconstruction of the segmentation result according to an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0021] It should be understood that the described embodiments are merely some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0022] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances. The singular forms "a," "the," and "the" used in this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. The word "if" as used herein can be interpreted as "when," "when," or "in response to determination."

[0023] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0024] Please see Figure 1 This is a flowchart of a lower extremity vascular plaque segmentation method according to the first embodiment of this application. The method includes: S1: Obtain lower limb CTA image training samples and a medical image segmentation model; the medical image segmentation model includes a global spatiotemporal feature extraction module, a downsampling feature extraction module, a feature fusion module, and an upsampling classification module; the global spatiotemporal feature extraction module includes a DINOv3-ViT large model and a temporal feature extraction model; The training samples for lower limb CTA images can be obtained through the following methods: We acquired 1,000 lower extremity CTA images from different patients using different scanning equipment (64-slice and 128-slice CT) and with different scanning parameters. These images covered cases of different ages (40-85 years), genders, and severity of PAD (grades I-IV). We also included special and difficult-to-distinguish scenarios such as vascular bifurcation, plaque mixing, and bone overlap to enhance the diversity and challenge of the dataset.

[0025] Please see Figure 2 Then, two experienced chief radiologists manually delineated the segmentation boundaries using a double-blind method, forming a three-category fine-grained label: red for blood vessel lumen, green for soft plaques (low-density areas, CT value <100 HU), and purple for calcified plaques (high-density areas, CT value >800 HU). After the labeling was completed, it was reviewed and corrected by third-party experts to ensure the accuracy of the labels.

[0026] To alleviate the problem of limited medical image annotation data and improve the generalization and anti-interference capabilities of the dual-path coding model (the global spatiotemporal feature extraction module and the downsampling feature extraction module of the medical image segmentation model), lower limb CTA image training samples can be preprocessed using a multi-dimensional data augmentation strategy of geometric transformation + pixel-level enhancement + anatomical region adaptive enhancement, including: 1) Random geometric transformation: The image is randomly rotated within ±20 degrees, randomly flipped with a 50% probability based on the horizontal / vertical axis, and randomly scaled within a scale of 0.8-1.2 times to simulate different scanning angles, changes in patient position and differences in vascular scale, and to adapt to the dual-path encoding requirements for capturing features at different scales. 2) Pixel-level noise and contrast enhancement: Add random Gaussian noise (simulating CT equipment noise) with a standard deviation of 0-0.02, randomly adjust the contrast within ±30%, and randomly shift the brightness within ±20% to enhance the adaptability of the dual-path coding model to different scanning parameters and contrast agent concentrations. 3) Standardization processing: The CT values ​​of the image are normalized to the range of [-1,1], the image is resampled, and the pixel spacing is uniformly set to 0.5×0.5×1.0 mm to ensure that the image size and feature scale of the input dual-path coding model are consistent, which lays the foundation for the alignment and fusion of the features of the two paths (the global spatiotemporal feature extraction module and the downsampling feature extraction module of the medical image segmentation model).

[0027] S2: The lower limb CTA image training samples are processed by global semantic extraction using the DINOv3-ViT large model of the spatiotemporal feature extraction module to obtain global semantic feature samples. Then, the global semantic feature samples are processed by temporal feature extraction using the temporal feature extraction model to obtain global spatiotemporal feature samples. The DINOv3-ViT large model is the officially pre-trained DINOv3-ViT-Base model, containing a 12-layer Transformer Encoder. With an embedding dimension of 768, it possesses highly discriminative global features (CLS tokens) and pixel-level dense features (patch tokens), and introduces a dynamic register tokens mechanism and Gram Anchoring strategy to adapt to dense prediction tasks in medical images, providing a foundation for global semantic capture. Its architecture is as follows... Figure 3 As shown.

[0028] Please see Figure 4 The DINOv3-ViT large model includes a multi-layer Transformer Encoder, which includes a multi-head attention (MHA) module, a feedforward neural network (MLP) module, and a residual connection module. The multi-head self-attention module receives sequence features after image block embedding and position encoding. It generates query (Q), key (K), and value (V) vectors through three sets of parallel linear layers. Q, K, and V are uniformly divided into multiple independent attention heads along the feature dimension. The scaled dot product attention weight is calculated independently for each head. After normalization and softmax activation, the weight is multiplied by the corresponding V to obtain the feature output. The outputs of all attention heads are concatenated along the feature dimension and fused into a multi-dimensional sequence feature through a linear projection layer. The feedforward neural network module is used to extract the nonlinear features of the multidimensional sequence features; The residual connection module is used to add and fuse the multidimensional sequence features and the nonlinear features to obtain the global semantic feature sample.

[0029] The global semantic feature samples can also be mapped to 1024 dimensions via a feature projection module activated by a 1×1 convolutional layer, batch normalization, and LeakyReLU. At the same time, the feature map spatial resolution is adjusted to match the features output by the downsampling feature extraction module through bilinear interpolation.

[0030] S3: The downsampling feature extraction module performs local feature extraction processing on the lower limb CTA image training samples to obtain several layers of downsampling convolutional feature samples; wherein, the bottom layer of downsampling convolutional feature samples has the same spatial dimension and channel dimension as the global spatiotemporal feature samples. S4: The feature fusion module fuses the global spatiotemporal feature samples and the lowest-level downsampled convolutional feature samples to obtain fused feature samples; S5: The upsampling classification module performs fusion upsampling and classification processing on the fused feature samples and several layers of downsampling convolutional feature samples to obtain the sample vascular plaque segmentation result; S6: Construct a loss function based on the standard vascular plaque segmentation results corresponding to the training samples of the lower limb CTA images and the vascular plaque segmentation results of the samples, and train the module parameters of the global spatiotemporal feature extraction module, downsampling feature extraction module, feature fusion module and upsampling classification module of the medical image segmentation model through the loss function; during training, LoRA is used to fine-tune the parameters of the DINOv3-ViT large model; The loss function is a composite loss function consisting of the sum of cross-entropy loss and multi-class Dice loss.

[0031] For example, X=[ , , [] is the one-hot representation of the true value of the sample label. When the sample is input into the j-th class, =1, otherwise =0. Y=[ , , [] represents the predicted values ​​for the four categories. The predicted value when the sample is input into the j-th category is [] The cross-entropy loss function for M categories is defined as follows:

[0032] The multi-class Dice loss function directly optimizes the overlap between the segmented region and the ground truth annotation, making it particularly effective for segmenting small objects. During the calculation, the Dice coefficient is calculated independently for each class; therefore, the multi-class Dice loss function is defined as follows:

[0033] in, To provide accurate annotations of the corresponding standard vascular plaque segmentation results, This is the prediction result of the corresponding sample vascular plaque segmentation result.

[0034] Furthermore, a dynamically optimized training strategy was employed during training, significantly improving the model's convergence and generalization capabilities. The basic hyperparameters were set as follows: initial learning rate 0.0005, maximum training epochs 1000, and batch size 2 (limited by GPU memory, but experimentally verified to be the most stable for medical image segmentation tasks). The optimizer used was AdamW, with its weight decay parameter set to 0.01 to effectively prevent overfitting.

[0035] The learning rate employs a cosine annealing scheduling strategy. Specifically, within each cycle T_0 = 50 rounds, the learning rate smoothly decreases from its initial value to 1e-6 using a cosine function, then resets and begins a new cycle. This strategy allows the model to fully explore the parameter space while also converging precisely to a high-quality local minimum.

[0036] Early stopping strategy: Set a validation set loss. If the validation set loss does not decrease for 50 consecutive rounds, stop training to prevent the model from overfitting.

[0037] S7: Input the target lower limb CTA image into the trained medical image segmentation model to obtain the lower limb vascular plaque segmentation result of the target lower limb CTA image.

[0038] Compared to related technologies, the medical image segmentation model in this application utilizes LoRA fine-tuning technology during training to perform anatomically-aware fine-tuning of the DINOv3 large model in the global spatiotemporal feature extraction module, thereby improving training efficiency and reducing training costs. Furthermore, in application, it can be combined with cross-slice temporal feature enhancement to fully capture global anatomical semantics and temporal correlations between slices, forming high-quality global spatiotemporal features. Moreover, it employs a downsampling feature extraction module to focus on extracting local spatial details and boundary features as a precise supplement to global semantic features, enabling accurate acquisition of lower limb vascular plaque segmentation results from target lower limb CTA images.

[0039] Please see Figure 3 In the medical image segmentation model, the global spatiotemporal feature extraction module and the downsampling feature extraction module serve as two independent and complementary feature extraction paths. They are specifically designed to extract large-scale spatiotemporal fine-tuning features (i.e., global semantic feature samples of lower limb CTA image training samples) and local features (i.e., downsampling convolutional feature samples of lower limb CTA image training samples), respectively. This approach not only highlights the global temporal advantages of large-scale spatiotemporal fine-tuning features but also compensates for the lack of local details, significantly improving segmentation accuracy and boundary refinement. This constructs an efficient fusion segmentation architecture. Then, through the feature fusion module and the upsampling classification module, fusion processing and upsampling classification processing are performed, which can accurately achieve lower limb vascular plaque segmentation.

[0040] In a feasible implementation, during training, several target-independent attention heads of each Transformer Encoder layer are fine-tuned using LoRA, the steps of which include: Please see Figure 5 In the qkv linear layer of the target independent attention head, low-rank matrices A and B are embedded in parallel and cascaded to form a LoRA layer; low-rank matrices A and B are bypasses composed of two linear layers. During training, the pre-trained weight matrix of the target independent attention head is frozen, and only the low-rank matrix A and the low-rank matrix B are trained. After training, the low-rank matrix A and the low-rank matrix B are multiplied and then combined to form the pre-trained weight matrix as the weight matrix parameters of the target independent attention head after fine-tuning.

[0041] Here, the target independent attention heads refer to the 4th, 8th, and 12th independent attention heads (Multi-Head Attention, MHA) in the Transformer Encoder. The pre-trained weight matrix of the target independent attention heads is as follows: low-rank matrix low-rank matrix The parameters of the fine-tuned weight matrix ,in, This is the scaling factor.

[0042] In a feasible embodiment, the step of the temporal feature extraction model performing temporal feature extraction processing on the global semantic feature samples to obtain global spatiotemporal feature samples includes: The temporal feature extraction model calculates the attention weights of several adjacent slices before and after the current slice relative to the current slice using cosine similarity. Then, based on the attention weights, it fuses the features of the several adjacent slices with the features of the current slice to obtain the global spatiotemporal feature samples.

[0043] Specifically, addressing the inherent temporal continuity of lower extremity vascular CTA tomographic images and the technical limitations of single-slice semantic information scarcity and difficulty in fully representing continuous tubular vascular structures, this application adds a cross-slice temporal feature enhancement module after extracting global semantic features from the DINOv3 large model. This module compensates for the lack of semantic information in single slices, optimizes the temporal dimension of the large model features, and completes the spatiotemporal fine-tuning feature construction of the large model adapted for lower extremity vascular CTA image analysis. Specifically, this module uses the DINO global semantic features of the current slice t as its core, selects the global semantic features of the three preceding consecutive slices t-3, t-2, and t-1 and the three following consecutive slices t+1, t+2, and t+3 of the current slice t to jointly form a temporal context window, and calculates the attention weight of each adjacent slice relative to the current slice using feature cosine similarity. Furthermore, all weights satisfy the normalization constraint ∑wi=1. Then, the weighted features of adjacent slices are fused with the features of the current slice to obtain global spatiotemporal feature samples. The feature fusion formula is as follows:

[0044] in, This is the global spatiotemporal feature sample of the current slice. The global semantic features of the current slice. For the first The attention weight of each slice relative to the current slice. For the first Global semantic features of each slice.

[0045] In one feasible embodiment, the downsampling feature extraction module includes an initial convolutional layer and a plurality of downsampling convolutional layers connected in sequence; The initial convolutional layer performs convolution processing on the lower limb CTA image training samples to obtain sample convolutional features; The highest-level downsampling convolutional layer performs downsampling convolution processing on the sample convolutional features, resulting in the corresponding downsampling convolutional feature samples. Each of the remaining downsampling convolutional layers performs downsampling convolution on the downsampling convolutional feature samples output by the previous downsampling convolutional layer to obtain the corresponding downsampling convolutional feature samples.

[0046] Please see Figure 6 The ResUNet architecture includes a ResUNet encoder and a ResUNet decoder, with the downsampling feature extraction module being the ResUNet encoder. Each downsampling convolutional layer consists of several residual blocks. The input image is first processed through an initial convolutional layer, followed by feature extraction and spatial downsampling through a series of stride convolutions and residual connections. The resolution of the feature maps output by each downsampling convolutional layer decreases sequentially, while the number of channels increases accordingly, ultimately reaching the preset spatial size and channel dimension at the bottleneck layer. Finally, the feature tensor output by the bottleneck layer is completely consistent with the projected DINO features (the global spatiotemporal feature samples) in both spatial and channel dimensions, creating conditions for accurate fusion.

[0047] In a feasible embodiment, the feature fusion module obtains the fused feature sample using the following formula: Fused_feature=gate_weight × DINO_proj + (1 - gate_weight) ×bottleneck_feat; Wherein, Fused_feature is the fused feature sample, gate_weight is the gate weight map generated by the gated network, DINO_proj is the global spatiotemporal feature sample, and bottleneck_feat is the bottom-level downsampled convolutional feature sample.

[0048] Specifically, the feature fusion module abandons the coarse fusion methods such as fixed-weight concatenation and weighted averaging in existing technologies. Instead, it designs a pixel-level adaptive gating fusion mechanism for the two types of core features extracted by the dual paths (global spatiotemporal feature extraction module and downsampling feature extraction module). This mechanism enables dynamic adaptation and precise fusion of spatiotemporal fine-tuning features of the large model and local features. Based on the type and feature characteristics of each pixel, the weight ratio of the two types of features is dynamically allocated to ensure that the fused features retain the global temporal advantages of the spatiotemporal fine-tuning features of the large model (global spatiotemporal feature samples) while also taking into account the precision of local features (downsampling convolutional feature samples). This constructs an efficient fusion and segmentation architecture, as detailed below: Please see Figure 7 After dual-path feature extraction, the DINO spatiotemporal fine-tuning features (global spatiotemporal feature samples) and the ResUNet bottleneck features (bottom-level downsampled convolutional feature samples) are completely consistent in both spatial and channel dimensions, creating conditions for fusion. The fusion process first concatenates the two features along the channel dimension to form a combined feature tensor. This tensor is then passed through a gated network, which starts with a 3×3 convolutional layer. Its large receptive field allows the network to consider the contextual information of the neighborhood around each pixel. After the convolution, a batch normalization layer and a LeakyReLU activation function are applied to enhance the feature representation capability. Subsequently, a 1×1 convolutional layer compresses the features to a single channel, generating a coarse weight map. Finally, the Sigmoid activation function compresses the weight values ​​to the range [0,1] to obtain the Gate Weight, where each pixel value represents the relative importance of the DINO spatiotemporal fine-tuning feature at that location.

[0049] This design enables pixel-level dynamic feature selection. The fused features are further integrated through a 1×1 convolutional layer to generate the final bottleneck layer fused features, maintaining the same dimensionality as the original ResUNet and ensuring seamless integration with the decoder. This gating mechanism can adaptively balance global semantic information with local spatial details, making it suitable for handling low-contrast regions and complex boundary conditions commonly encountered in lower limb vascular plaque segmentation.

[0050] In one feasible embodiment, the upsampling classification module includes a plurality of upsampling convolutional layers and activation classification units connected in sequence; After the bottom-most upsampling convolutional layer upsamples the fused feature sample, it performs a fusion convolution process with the downsampling convolutional feature sample of the corresponding downsampling convolutional layer to obtain the corresponding upsampled fused feature sample. Each of the remaining upsampling convolutional layers upsamples the upsampled fused feature sample output by the previous upsampling convolutional layer, and then performs a fused convolution with the downsampled convolutional feature sample of the corresponding downsampling convolutional layer to obtain the corresponding upsampled fused feature sample. The activation classification unit performs activation classification processing on the upsampled fusion feature samples output by the highest upsampled convolutional layer to obtain the sample vascular plaque segmentation result.

[0051] Please see Figure 8 The upsampling classification module is a ResUNet decoder. The decoder adopts a symmetrical upsampling architecture to gradually restore spatial resolution and refine segmentation boundaries. Each upsampling convolutional layer first enhances the feature map resolution through transposed convolution or bilinear interpolation combined with 1×1 convolution. Then, it is fused with the feature map saved in the corresponding stage of the encoder through skip connections. This design effectively restores the spatial details lost in the encoder. The fused features are further refined through multiple 3×3 convolutional layers and the LeakyReLU activation function to enhance feature representation capabilities.

[0052] Finally, the activation classifier is activated by applying the softmax activation function to generate a probability distribution map of the three categories. Each pixel position outputs three probability values, corresponding to blood vessels, calcified plaques, and soft plaques, respectively, with a total of 1.0.

[0053] In this application, the trained medical image segmentation model can output a three-channel feature map of the target lower limb CTA image. A pixel-level probability distribution of lower limb blood vessels (x_vascular_lumen), soft plaques (x_soft_plaque), and calcified plaques (x_calcified_plaque) is generated using the softmax function. The probability value at each pixel location satisfies: x_vascular_lumen + x_soft_plaque + x_calcified_plaque = 1.0, and each value ∈ [0,1]. Here, f_max = max(x_vascular_lumen, x_soft_plaque, x_calcified_plaque), and the index value corresponding to the maximum value is the corresponding three category value.

[0054] The above steps yielded gx values ​​for three different categories: lower limb blood vessels (value 1), soft plaques (value 2), and calcifications (value 3). This resulted in the segmentation results for lower limb blood vessels, soft plaques, and calcified plaques. The 3D reconstruction results are shown below. Figure 9 As shown. Therefore, after inputting the target lower limb CTA image into the trained medical image segmentation model to obtain the lower limb vascular plaque segmentation result of the target lower limb CTA image, the mask region area, volume, and three-dimensional volume index of the blood vessels or plaques in each slice can be obtained based on the lower limb vascular plaque segmentation result of each slice: Based on the number of segmentation mask pixels in the lower limb vascular plaque segmentation results, the mask region area of ​​the blood vessel or plaque in each slice is obtained:

[0055] in, Let be the area of ​​the mask region for blood vessels or plaques on the i-th slice. This represents the number of segmentation mask pixels in the masked region of the corresponding slice within the blood vessel or plaque. It is the pixel spacing of the slice along the x-axis. It is the pixel spacing of the slice along the y-axis; Based on the lower limb vascular plaque segmentation results, obtain the total volume of the blood vessel or plaque:

[0056] in, The total volume of the blood vessel or plaque. The thickness of the slice interval. The total number of slices covering the masked region corresponding to a blood vessel or plaque; Based on the total volume of the plaque and the total volume of the blood vessels, the three-dimensional volumetric parameters of the target lower limb CTA image are obtained:

[0057] in, For three-dimensional volume indicators, The total volume of the patch. This represents the total volume of the blood vessels.

[0058] In summary, the core innovations of this application focus on: ① Constructing a dual-path feature extraction architecture with clear division of labor and complementary collaboration—Path 1: LoRA fine-tuning + cross-slice temporal enhancement, specifically extracting spatiotemporal fine-tuning features of large models, focusing on capturing global semantics and temporal correlations; Path 2: ResUNet focuses on extracting local spatial details and boundary features (downsampled features) as a supplement to global spatiotemporal features; ② Using a pixel-level adaptive gating fusion mechanism, unlike existing fixed-weight fusion, it can dynamically adjust the weight ratio of global spatiotemporal features and downsampled features according to pixel organization type, achieving accurate adaptation of the two types of features. The synergy of dual-path feature extraction and fusion mechanism not only highlights the global temporal advantages of global spatiotemporal features but also compensates for the lack of local details, significantly improving segmentation accuracy and boundary refinement.

[0059] A second embodiment of this application provides a lower limb vascular plaque segmentation device, including a storage device, a processor, and a computer program stored in the storage device and executable by the processor. When the processor executes the computer program, it inputs a target lower limb CTA image into a medical image segmentation model that has been trained as described above, and obtains the lower limb vascular plaque segmentation result of the target lower limb CTA image.

[0060] It should be noted that the lower limb vascular plaque segmentation device provided in the second embodiment of this application is only illustrated by the above-described division of functional modules when performing the lower limb vascular plaque segmentation method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the lower limb vascular plaque segmentation device provided in the second embodiment of this application and the lower limb vascular plaque segmentation method of the first embodiment of this application belong to the same concept, and its implementation process is detailed in the method embodiment, which will not be repeated here.

[0061] The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0062] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0063] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function selected in one or more boxes.

[0064] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function selected in one or more boxes.

[0065] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0066] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0067] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0068] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0069] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for segmenting a blood vessel plaque of a lower limb, characterized by, include: Training samples of lower limb CTA images and a medical image segmentation model are obtained; the medical image segmentation model includes a global spatiotemporal feature extraction module, a downsampling feature extraction module, a feature fusion module, and an upsampling classification module; the global spatiotemporal feature extraction module includes a DINOv3-ViT large model and a temporal feature extraction model; The lower limb CTA image training samples are processed by global semantic extraction using the DINOv3-ViT large model of the spatiotemporal feature extraction module to obtain global semantic feature samples. Then, the global semantic feature samples are processed by temporal feature extraction using the temporal feature extraction model to obtain global spatiotemporal feature samples. The downsampling feature extraction module performs local feature extraction on the lower limb CTA image training samples to obtain several layers of downsampling convolutional feature samples; wherein, the bottom layer of downsampling convolutional feature samples has the same spatial dimension and channel dimension as the global spatiotemporal feature samples. The feature fusion module fuses the global spatiotemporal feature samples and the lowest-level downsampled convolutional feature samples to obtain fused feature samples. The upsampling classification module performs fusion upsampling and classification processing on the fused feature samples and several layers of downsampling convolutional feature samples to obtain the sample vascular plaque segmentation result. A loss function is constructed based on the standard vascular plaque segmentation results corresponding to the training samples of the lower limb CTA images and the vascular plaque segmentation results of the samples. The module parameters of the global spatiotemporal feature extraction module, downsampling feature extraction module, feature fusion module and upsampling classification module of the medical image segmentation model are trained through the loss function. During training, LoRA is used to fine-tune the parameters of the DINOv3-ViT large model. The target lower limb CTA image is input into the trained medical image segmentation model to obtain the lower limb vascular plaque segmentation result of the target lower limb CTA image.

2. The method of claim 1, wherein, The DINOv3-ViT large model includes a multi-layer Transformer Encoder, which includes a multi-head self-attention module, a feedforward neural network module, and a residual connection module. The multi-head self-attention module is used to receive image block embedding and add position-encoded sequence features, and generate query (Q), key (K) and value (V) vectors through three sets of parallel linear layers respectively; The Q, K, and V are uniformly divided into multiple independent attention heads along the feature dimension. The scaled dot product attention weight is calculated independently for each head. After normalization and softmax activation, it is multiplied with the corresponding V to obtain the feature output. The outputs of all attention heads are concatenated along the feature dimension and fused into a multidimensional sequence feature through a linear projection layer. The feedforward neural network module is used to extract the nonlinear features of the multidimensional sequence features; The residual connection module is used to add and fuse the multidimensional sequence features and the nonlinear features to obtain the global semantic feature sample.

3. The method of claim 2, wherein, During training, several target-independent attention heads of each Transformer Encoder layer are fine-tuned using LoRA.

4. The method of claim 3, wherein, The steps for fine-tuning parameters using LoRA include: Low-rank matrices A and B are embedded in parallel and concatenated to form a LoRA layer in the qkv linear layer of the target independent attention head; low-rank matrices A and B are bypasses composed of two linear layers. During training, the pre-trained weight matrix of the target independent attention head is frozen, and only the low-rank matrix A and the low-rank matrix B are trained. After training, the low-rank matrix A and the low-rank matrix B are multiplied and then combined to form the pre-trained weight matrix as the weight matrix parameters of the target independent attention head after fine-tuning.

5. The method of claim 1, wherein, The step of the temporal feature extraction model performing temporal feature extraction processing on the global semantic feature samples to obtain global spatiotemporal feature samples includes: The temporal feature extraction model calculates the attention weights of several adjacent slices before and after the current slice relative to the current slice using cosine similarity. Then, based on the attention weights, it fuses the features of the several adjacent slices with the features of the current slice to obtain the global spatiotemporal feature samples.

6. The method for segmenting lower limb vascular plaques according to claim 1, characterized in that, The downsampling feature extraction module includes an initial convolutional layer and several downsampling convolutional layers connected in sequence; The initial convolutional layer performs convolution processing on the lower limb CTA image training samples to obtain sample convolutional features; The highest-level downsampling convolutional layer performs downsampling convolution processing on the sample convolutional features to obtain the corresponding downsampling convolutional feature samples. Each of the remaining downsampling convolutional layers performs downsampling convolution on the downsampling convolutional feature samples output by the previous downsampling convolutional layer to obtain the corresponding downsampling convolutional feature samples.

7. The method for segmenting lower limb vascular plaques according to claim 1, characterized in that, The feature fusion module obtains the fused feature sample using the following formula: Fused_feature=gate_weight × DINO_proj + (1 - gate_weight) × bottleneck_feat; Wherein, Fused_feature is the fused feature sample, gate_weight is the gate weight map generated by the gated network, DINO_proj is the global spatiotemporal feature sample, and bottleneck_feat is the bottom-level downsampled convolutional feature sample.

8. The method for segmenting lower limb vascular plaques according to claim 1, characterized in that, The upsampling classification module includes several upsampling convolutional layers and activation classification units connected in sequence; After the bottom-most upsampling convolutional layer upsamples the fused feature sample, it performs a fusion convolution process with the downsampling convolutional feature sample of the corresponding downsampling convolutional layer to obtain the corresponding upsampled fused feature sample. Each of the remaining upsampling convolutional layers upsamples the upsampled fused feature sample output by the previous upsampling convolutional layer, and then performs a fused convolution with the downsampled convolutional feature sample of the corresponding downsampling convolutional layer to obtain the corresponding upsampled fused feature sample. The activation classification unit performs activation classification processing on the upsampled fusion feature samples output by the highest upsampled convolutional layer to obtain the sample vascular plaque segmentation result.

9. The method for segmenting lower limb vascular plaques according to claim 1, characterized in that, After inputting the target lower limb CTA image into the trained medical image segmentation model to obtain the lower limb vascular plaque segmentation result of the target lower limb CTA image, the method further includes: Based on the number of segmentation mask pixels in the lower limb vascular plaque segmentation results, the mask region area of ​​the blood vessel or plaque in each slice is obtained: in, Let be the area of ​​the mask region for blood vessels or plaques on the i-th slice. This represents the number of segmentation mask pixels in the masked region of the corresponding slice within the blood vessel or plaque. It is the pixel spacing of the slice along the x-axis. It is the pixel spacing of the slice along the y-axis; Based on the lower limb vascular plaque segmentation results, obtain the total volume of the blood vessel or plaque: in, The total volume of the blood vessel or plaque. The thickness of the slice interval. The total number of slices covering the masked region corresponding to a blood vessel or plaque; Based on the total volume of the plaque and the total volume of the blood vessels, the three-dimensional volumetric parameters of the target lower limb CTA image are obtained: in, For three-dimensional volume indicators, The total volume of the patch. This represents the total volume of the blood vessels.

10. A device for segmenting lower limb vascular plaques, characterized in that: The device includes a storage device, a processor, and a computer program stored in the storage device and executable by the processor. When the processor executes the computer program, it inputs a target lower limb CTA image into a trained medical image segmentation model as described in any one of claims 1-9 to obtain a lower limb vascular plaque segmentation result from the target lower limb CTA image.

Images