Vessel image segmentation method based on space-time collaboration and multi-scale snake convolution

Abstract

The application provides a blood vessel image segmentation method based on space-time cooperation and multi-scale snake convolution, which comprises the following steps: sequentially performing down-sampling processing on an image sequence by using a down-sampling operator to obtain a sequence of each scale; obtaining time sequence feature representation by processing the sequence of each scale based on a time sequence convolution network; performing time sequence feature shunt processing and aggregation processing on the time sequence feature representation to obtain a time sequence feature map of four scales; performing encoding processing on the time sequence feature map of four scales by using an encoder to obtain an original bottleneck feature map; sequentially processing the original bottleneck feature map by using a multi-scale snake convolution module to obtain a processed bottleneck feature map; and obtaining a final segmentation result map by processing the processed bottleneck feature map based on a decoder; and the application shows extremely high parameter efficiency by designing a time sequence extraction module based on a time convolution network to effectively avoid a huge calculation burden.

Description

Technical Field

[0001] This invention relates to the field of medical image analysis technology, and in particular to a method for segmenting blood vessel images based on spatiotemporal collaboration and multi-scale serpentine convolution. Background Technology

[0002] In the assessment of cerebrovascular diseases, digital subtraction angiography (DSA) is widely recognized as the "gold standard" due to its high spatiotemporal resolution in capturing the inflow and outflow of contrast agents and the hemodynamic processes. Unlike typical static medical images, DSA data is presented as a multi-frame sequence, containing dynamic temporal information of great diagnostic value. However, manually interpreting multi-frame DSA sequences frame by frame is extremely time-consuming and highly dependent on expert experience. Therefore, there is an urgent need for a sequence segmentation method that can automatically and robustly process dynamic multi-frame data. However, due to the spatiotemporal asynchrony of contrast agent perfusion, the low contrast of peripheral vessels, and extreme scale changes, accurately extracting and aggregating vascular features from high-dimensional dynamic sequences remains extremely challenging.

[0003] Most existing studies on vessel segmentation have failed to effectively utilize the dynamic advantages of multi-frame sequences. Early methods typically relied on minimum density projection (MinIP) to crudely compress the sequence into a single static 2D image, or used 2D convolutional neural networks, such as U-Net, for independent training based on a single frame. These static or single-frame segmentation methods completely strip away the temporal coherence in the angiographic sequence, making vessel rupture highly susceptible to local artifacts or insufficient perfusion. To capture the temporal dependencies of the sequence, researchers have explored segmentation architectures directly targeting multi-frame data, such as using 3D convolution, ConvGRU, and Transformer-based networks for temporal modeling. However, these approaches face bottlenecks such as limited local receptive fields, low computational efficiency for long sequences, and excessive secondary computational overhead for multi-frame self-attention mechanisms. In another core aspect of sequence segmentation—the spatiotemporal aggregation of multi-frame features—early temporal pooling strategies (such as average or max pooling) easily lose fine-grained evolution information of microvessels in different frames. Although subsequent studies have introduced global attention, it is still difficult to accurately account for the asynchronous propagation of contrast agents in different spatial locations, resulting in insufficient pixel-level fusion accuracy. In addition, when processing multi-frame sequence features, due to the extremely large scale span and high distortion of intracranial blood vessels, traditional fixed grid sampling (such as standard or dilated convolution) is difficult to adapt to the actual direction of blood vessels and deform accordingly. A single fixed convolution kernel is very likely to miss slender and tortuous blood vessels at extreme scales during multi-frame feature extraction. Summary of the Invention

[0004] In view of the above situation, the main objective of this invention is to propose a blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution to solve the above-mentioned technical problems.

[0005] This invention proposes a blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution, the method comprising the following steps:

[0006] Step 1: Use the original image sequence as the first scale sequence; use the downsampling operator to downsample the first scale sequence one by one to obtain the second scale sequence, the third scale sequence, and the fourth scale sequence respectively.

[0007] Based on sequences at the first, second, third, and fourth scales, temporal feature representations are obtained through processing by a temporal convolutional network.

[0008] Step 2: Perform time series feature splitting and aggregation processing on the time series feature representation to obtain time series feature maps at four scales;

[0009] Step 3: Encode the temporal feature maps at four scales using an encoder to obtain the original bottleneck feature maps;

[0010] Step 4: Use the multi-scale snake convolution module to perform parallel branch convolution processing on the original bottleneck feature map, and then sequentially pass it through attention-gated progressive fusion processing and residual weighting processing to obtain the processed bottleneck feature map.

[0011] Step 5: Input the processed bottleneck feature map into the decoder, and obtain the final segmentation result map through step-by-step upsampling and dense fusion processing.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0013] 1. This invention designs a temporal extraction module based on a temporal convolutional network, which is specifically designed to capture long-range hemodynamic features in DSA sequences (original image sequences) in a fully parallel manner. This effectively avoids a huge computational burden, exhibits extremely high parameter efficiency, and provides fast training and inference, while also presenting the best blood vessel segmentation results.

[0014] 2. This invention designs a pixel-level attention pooling module specifically to address the problem that microvessels are easily smoothed out during multi-frame fusion. After temporal feature extraction, an adaptive weight allocation mechanism is introduced to enhance the feature representation of key imaging frames. This mechanism can selectively emphasize key temporal components rich in contrast agent information, significantly improving the network's ability to retain low-contrast microvessel details and enhancing the sensitivity of vessel segmentation.

[0015] 3. This invention designs a bottleneck layer extraction method that combines multi-scale features with deformable dynamic serpentine convolution to guide the convolution kernel to adaptively deform along the blood vessel centerline. This aims to overcome the segmentation fragmentation problem caused by extreme scales and curved blood vessels, and establish a more robust solution with higher topological connectivity for cerebral blood vessel segmentation tasks. Attached Figure Description

[0016] Figure 1 This is a flowchart of the blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution proposed in this invention;

[0017] Figure 2 This is a diagram illustrating the overall framework of the blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution proposed in this invention.

[0018] Figure 3 The diagram shows the overall framework of the multi-scale serpentine convolution module and the attention gating mechanism of the blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution proposed in this invention; wherein, (a) is the framework diagram of the multi-scale serpentine convolution module and (b) is the framework diagram of the attention gating mechanism. Detailed Implementation

[0019] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0020] These and other aspects of the embodiments of the present invention will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to illustrate some ways of implementing the principles of the embodiments of the present invention; however, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0021] Please see Figure 1 and Figure 2 This invention proposes a blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution, which includes the following steps:

[0022] Step 1: Use the original image sequence as the first scale sequence; use the downsampling operator to downsample the first scale sequence one by one to obtain the second scale sequence, the third scale sequence, and the fourth scale sequence respectively.

[0023] Based on sequences at the first, second, third, and fourth scales, temporal feature representations are obtained through processing by a temporal convolutional network.

[0024] In step 1, based on the sequences at the first, second, third, and fourth scales, a temporal convolutional network is used to process the sequences to obtain temporal feature representations. The specific steps are as follows:

[0025] The temporal convolutional network consists of two serially cascaded temporal extraction modules, with the dilation rate of each module increasing exponentially. Each temporal extraction module contains two identical dilated causal convolutional layers, and the two dilated causal convolutional layers share the same dilation rate. The temporal extraction module consists of two convolutional layers connected in series, and after each convolutional operation, weight normalization, ReLU activation function, and random dropout processing are performed sequentially.

[0026] The sequence at the current scale is concatenated frame by frame with the temporal feature sequence at the previous scale, and then fused through 1×1 convolution to obtain the fused feature sequence.

[0027] The fused feature sequence is reshaped into a dimensionally reshaped feature sequence; the dimensionally reshaped feature sequence is then input into the first temporal extraction module for processing to obtain the output of the first temporal extraction module.

[0028] The output of the first time series extraction module is used as the input of the second time series extraction module. After stacking the two time series extraction modules, the time series feature representation is obtained.

[0029] Specifically, the original image sequence is used as the sequence at the first scale; the downsampling operator is used to successively downsample the sequence at the first scale to obtain the sequences at the second, third, and fourth scales, respectively. The corresponding process has the following relationship:

[0030] ;

[0031] in, Represents the sequence at the first scale; Represents the original image sequence, and , This represents the number of feature channels in the input image. Indicates batch size, This represents the height of each original image frame. Indicates the width of each frame of the original image; Indicates the first Sequences of various scales Indicates the first Sequences of various scales This indicates processing via a convolution operation with a stride of 2;

[0032] After concatenating the current scale sequence with the temporal feature sequence of the previous scale frame by frame, and then fusing them through 1×1 convolution to obtain the fused feature sequence, the following relationship exists:

[0033] ;

[0034] in, Indicates the first The first scale The fused feature sequence of frames, This represents the learnable weights of a 1×1 convolutional kernel. Indicates the first The first scale The temporal feature sequence of frames, Indicates the first The first scale A sequence of frames, Indicates the total number of frames. This indicates channel-by-channel splicing processing.

[0035] It should be noted that this method of fusing the original image sequence at the current scale with the temporal features passed from the previous stage frame by frame enables each temporal extraction module (ETEM block) to jointly mine multi-scale temporal context and obtain progressively refined dynamic representations.

[0036] Specifically, in the process of inputting the dimension-reshaped feature sequence into the first temporal extraction module for processing to obtain the output of the first temporal extraction module, the following relationship exists:

[0037] ;

[0038] in, This represents the intermediate output features after the first convolutional transformation. This represents the output of the first timing extraction module. This indicates weight normalization processing. This indicates activation function processing. This indicates that the convolution kernel is... void ratio Hollow causal convolution processing; Indicates the reshaped first Feature sequences at each scale, and ; This indicates processing via residual mapping. This indicates random discarding.

[0039] In the process of dilated causal convolution, the definition of causal convolution is:

[0040] ;

[0041] in, Indicates the first Causal convolution processing of frames The sliding index variable represents the convolution kernel. Indicates the total size of the convolution kernel. Indicates the convolution kernel at the index The weighting coefficient at the location, This indicates that the input feature sequence is at time [time]. The sampled value at that location.

[0042] It should be noted that the Temporal Convolutional Network (TCN) consists of two serially cascaded Temporal Blocks. The first Temporal Block has a dilation rate of 1, and the second Temporal Block has a dilation rate of 2. Each Temporal Block contains two identical dilated causal convolutional layers, and these two dilated causal convolutional layers share the same dilation rate. Inside each Temporal Block, there are two convolutional layers connected in series. After each convolutional operation, weight normalization (WN), ReLU activation function, and random dropout are performed sequentially to enhance the stability of training.

[0043] Step 2: Perform time series feature splitting and aggregation processing on the time series feature representation to obtain time series feature maps at four scales;

[0044] In step 2, the temporal feature representation is subjected to temporal feature splitting and aggregation processing to obtain temporal feature maps at four scales. The specific steps are as follows:

[0045] For each frame representing the temporal features at the current scale, spatial downsampling is performed using a 2×2 convolution with a stride of 2 to obtain the temporal feature sequence at the current scale; the temporal feature sequence at the current scale is then used to participate in the temporal convolutional network processing at the next scale.

[0046] All temporal feature sequences are accumulated to obtain an accumulated temporal feature sequence; the accumulated temporal feature sequence is processed by 1×1 convolution to obtain a pixel-level score map;

[0047] The pixel-level score map is normalized along the temporal dimension to obtain attention weights; the attention weights and temporal feature sequences are then aggregated to obtain temporal feature maps at four scales.

[0048] Specifically, in the process of spatial downsampling using a 2×2 convolution with a stride of 2 for each frame representing the temporal features at the current scale, to obtain the temporal feature sequence at the current scale, the following relationship exists:

[0049] ;

[0050] in; Indicates the first The first scale The temporal feature sequence of the frame, and ; This indicates processing using a 2×2 convolution with a stride of 2. Indicates the first The first scale Temporal characteristics of frames;

[0051] In the process of processing the accumulated temporal feature sequence through 1×1 convolution to obtain the pixel-level score map, the following relationship exists:

[0052] ;

[0053] in; Represents a pixel-level score image, and ; This indicates processing via 1×1 convolution. Indicates the first A time-series feature sequence accumulated at multiple scales;

[0054] In the process of using attention weights and temporal feature sequences, and then aggregating them to obtain temporal feature maps at four scales, the following relationship exists:

[0055] ;

[0056] in, Indicates the first Temporal feature maps at various scales; Indicates the first Attention weights for frames, and ; This indicates element-wise multiplication.

[0057] Furthermore, through attention-based temporal aggregation, this step yields a 2D feature map. , Selectively integrating the spatial responses with the highest information content throughout the entire sequence effectively summarizes the dynamic blood flow pattern into a compact representation; after extraction stages by four temporal extraction modules, a set of temporal priors is obtained. .

[0058] Step 3: Encode the temporal feature maps at four scales using an encoder to obtain the original bottleneck feature maps;

[0059] In step 3, the temporal feature maps at four scales are encoded using an encoder to obtain the original bottleneck feature map. The specific steps are as follows:

[0060] The encoder consists of four levels, each of which consists of two consecutive 3×3 convolutional layers and a stride convolution with a stride of 2. The coding levels are connected by dense skip connections.

[0061] The temporal feature map of the first scale is input into the first layer of the encoder, processed by two consecutive 3×3 convolutional layers, and then batch normalized and ReLU activated in sequence to obtain the intermediate feature map of the first layer. The intermediate feature map of the first layer is reduced in spatial resolution by using stride convolution with a stride of 2 to obtain the output feature map of the first layer.

[0062] The temporal feature map of the second scale is input into the second layer of the encoder. After being processed by two consecutive 3×3 convolutional layers, it is batch normalized and ReLU activated in sequence to obtain the intermediate feature map of the second layer. The intermediate feature map of the second layer is reduced in spatial resolution using stride convolution with a stride of 2 to obtain the feature map of the second layer. The output feature map of the first layer and the feature map of the second layer are channel adjusted and fused using 1×1 convolution to obtain the output feature map of the second layer.

[0063] The temporal feature map of the third scale is input into the third level of the encoder. After being processed by two consecutive 3×3 convolutional layers, it is batch normalized and ReLU activated in sequence to obtain the intermediate feature map of the third level. The spatial resolution of the intermediate feature map of the third level is reduced by using stride convolution with a stride of 2 to obtain the feature map of the third level. The output feature map of the second level and the feature map of the third level are channel adjusted and fused by 1×1 convolution to obtain the output feature map of the third level.

[0064] The temporal feature map of the fourth scale is input into the fourth level of the encoder. After processing through two consecutive 3×3 convolutional layers, it is batch normalized and ReLU activated sequentially to obtain the intermediate feature map of the fourth level. The intermediate feature map of the fourth level is then processed to reduce its spatial resolution using strided convolution with a stride of 2 to obtain the feature map of the fourth level. The output feature map of the third level and the feature map of the fourth level are then channel-adjusted and fused using 1×1 convolution to obtain the output feature map of the fourth level. The output feature map of the fourth level is then used as the original bottleneck feature map.

[0065] Furthermore, the encoder adopts a nested architecture of FR-Unet, consisting of four encoding levels. The encoder incorporates the multi-scale temporal priors generated during the temporal modeling stage. As input, each of Each feature is fed into its corresponding encoding level; this design ensures explicit alignment between temporal representations and spatial feature levels; each encoding level contains a downsampling module consisting of two consecutive 3×3 convolutional layers (each followed by batch normalization and ReLU activation) and a stride convolution with a stride of 2 to reduce spatial resolution; to promote multi-scale feature reuse and mitigate information loss caused by repeated downsampling, dense skip connections are used between encoder levels to progressively aggregate features from different depths in a nested manner.

[0066] Step 4: Use the multi-scale snake convolution module to perform parallel branch convolution processing on the original bottleneck feature map, and then sequentially pass it through attention-gated progressive fusion processing and residual weighting processing to obtain the processed bottleneck feature map.

[0067] Please see Figure 3 In step 4, the original bottleneck feature map is processed by parallel branch convolution using a multi-scale serpentine convolution module, followed by attention-gated progressive fusion and residual weighting to obtain the processed bottleneck feature map. The specific steps are as follows:

[0068] The original bottleneck feature map is input into the four parallel branches of the multi-scale snake convolution module. Each parallel branch processes the original bottleneck feature map using deformable dynamic snake convolution to obtain the output feature map of each parallel branch.

[0069] Starting from the first parallel branch, the output feature map of the current parallel branch is fused with the aggregated feature map of the previous parallel branch through attention gating operation to obtain the aggregated feature map of the current parallel branch.

[0070] After completing the fusion processing of all parallel branches, the final aggregated feature is obtained; through a learnable residual weighting mechanism, the final aggregated feature is combined with the original bottleneck feature map to obtain the processed bottleneck feature map.

[0071] Specifically, in the process of processing the original bottleneck feature map using deformable dynamic serpentine convolution in each parallel branch to obtain the output feature map of each parallel branch, the following relationship exists:

[0072] ;

[0073] in, Indicates the first Output feature maps of parallel branches This indicates batch normalization processing. This indicates that the convolution kernel size is... The initial direction of offset learning is Deformable dynamic serpentine convolution processing, This represents the original bottleneck feature map; Indicates the first The kernel size of deformable dynamic serpentine convolutions in each parallel branch, and ; Indicates the first The offset learning of the initial direction of deformable dynamic serpentine convolution in each parallel branch, and ;

[0074] In each parallel branch, the processing of the original bottleneck feature map by the deformable dynamic serpentine convolution is expressed by the following relationship:

[0075] ;

[0076] in, This represents the offset of the original bottleneck feature map in dynamic prediction. Indicates size is The learnable convolution weights for the sampling points corresponding to the convolution kernel. This indicates the current position on the original bottleneck feature map; Relative to The size is The kernel offset, and , ; This represents a convolution operation that calculates the offset. Describing deformable dynamic serpentine convolution in Convolution calculation at the point, Representation of feature map The offset position;

[0077] Starting from the first parallel branch, the process of fusing the output feature map of the current parallel branch with the aggregated feature map of the previous parallel branch through attention gating to obtain the aggregated feature map of the current parallel branch has the following relationship:

[0078] ;

[0079] in, This represents the output feature map of the first parallel branch. The aggregation features of the first parallel branch, Indicates the first Aggregation features of parallel branches Indicates the first Aggregation features of parallel branches Indicates the first Output feature maps of parallel branches This indicates processing via attention gating;

[0080] In the process of obtaining the aggregated features of the current parallel branch, the attention gating's processing of the input features is expressed by the following relation:

[0081] ;

[0082] in; Indicates the gating signal, i.e. ; This represents the features of the gated signal after a 1×1 convolution transformation. This represents the output feature map of the current parallel branch, i.e. ; This represents the features after performing a 1×1 convolution transformation on the output feature map of the current parallel branch. This represents the 1×1 convolution weight applied to the gated signal. This represents the 1×1 convolution weights applied to the output feature map of the current parallel branch. Representing a spatial attention map, This indicates that it is processed using the Sigmoid function. This indicates processing via 1×1 convolution;

[0083] In the process of combining the final aggregated features with the original bottleneck feature map through a learnable residual weighting mechanism to obtain the processed bottleneck feature map, the following relationship exists:

[0084] ;

[0085] in, This represents the processed bottleneck feature map; Represents learnable parameters, and ; This represents the final aggregated feature.

[0086] It should be noted that steps 3 and 4 together constitute the segmentation stage of this invention. The segmentation stage aims to generate accurate pixel-level vessel segmentation results by utilizing multi-scale temporal priors obtained from the temporal modeling stage. In order to effectively integrate features across different spatial resolutions and preserve fine-grained details, this invention adopts a densely connected encoder-decoder architecture that supports full-scale feature fusion. In addition, in order to explicitly address the elongated shape and drastic scale changes of intracranial vessels, this invention introduces a multi-scale serpentine convolution module (MSCM) in the bottleneck layer of the segmentation stage. The multi-scale serpentine convolution module enhances morphological adaptability and promotes vessel continuity across extreme vessel scales.

[0087] Furthermore, in the bottleneck layer of the segmentation stage, a multi-scale serpentine convolution module is introduced to enhance morphological adaptability and vessel continuity; the original bottleneck features generated by the encoder are denoted as... , which serves as the input to the multi-scale serpentine convolution module.

[0088] The multi-scale serpentine convolution module contains four parallel branches, each equipped with different kernel sizes and deformable dynamic serpentine convolution (DSConv). The deformable dynamic serpentine convolution adapts the kernel sampling points to the vascular morphology through learnable offsets.

[0089] Step 5: Input the processed bottleneck feature map into the decoder, and obtain the final segmentation result map through step-by-step upsampling and dense fusion processing;

[0090] In step 5, the decoder consists of four stages. Except for the first stage which includes dense fusion processing, each subsequent stage includes progressive upsampling processing and dense fusion processing.

[0091] The processed bottleneck feature map is used as the feature map of the first stage upsampling, and it is concatenated with the intermediate feature map of the first level along the channel dimension to obtain the decoding feature map of the first stage.

[0092] After refining the decoding feature map of the first stage through two consecutive 3×3 convolutional layers, the intermediate prediction values ​​of the first stage are obtained through batch normalization and ReLU activation.

[0093] The decoded feature map of the first stage is continuously upsampled using bilinear interpolation to obtain the upsampled feature map of the second stage.

[0094] The second-stage upsampled feature map, the intermediate feature map of the second level, and the first-stage decoded feature map are concatenated along the channel dimension to obtain the second-stage decoded feature map.

[0095] After refining the decoding feature map of the second stage through two consecutive 3×3 convolutional layers, the intermediate prediction values ​​of the second stage are obtained through batch normalization and ReLU activation.

[0096] The decoded feature map of the second stage is continuously upsampled using bilinear interpolation to obtain the feature map of the third stage.

[0097] The feature map of the third stage upsampling, the intermediate feature map of the third level, and the decoding feature map of the second stage are concatenated along the channel dimension to obtain the decoding feature map of the third stage.

[0098] After refining the decoding feature map of the third stage through two consecutive 3×3 convolutional layers, the intermediate prediction values ​​of the third stage are obtained through batch normalization and ReLU activation.

[0099] The decoded feature map of the third stage is continuously upsampled using bilinear interpolation to obtain the upsampled feature map of the fourth stage.

[0100] The feature map of the fourth stage upsampling, the intermediate feature map of the fourth level, and the decoding feature map of the third stage are concatenated along the channel dimension to obtain the decoding feature map of the fourth stage.

[0101] After refining the decoding feature map of the fourth stage through two consecutive 3×3 convolutional layers, the intermediate prediction values ​​of the fourth stage are obtained through batch normalization and ReLU activation.

[0102] The intermediate predicted values ​​from the first stage, the second stage, the third stage, and the fourth stage are averaged and then processed by the Sigmoid activation function to obtain the final segmentation result image.

[0103] Specifically, after averaging the intermediate predicted values ​​from the first, second, third, and fourth stages, and then processing them through the Sigmoid activation function to obtain the final segmentation result image, the following relationship exists:

[0104] ;

[0105] in, This represents the final segmentation result image. Indicates the first Intermediate forecast values ​​for the stage.

[0106] Furthermore, in the decoder's upsampling stage, spatial resolution is gradually restored and blood vessel boundaries are refined. After upsampling and channel alignment, the decoder aggregates feature maps from the corresponding encoder level and feature maps from higher-resolution decoder nodes, thus simultaneously utilizing shallow spatial details and deep semantic context. The fused representation is then refined by two consecutive 3×3 convolutional layers. The four nested decoder nodes with the highest spatial resolution generate four intermediate predicted logistic values. This facilitates deep supervision during training. During the inference phase, these predictions are aggregated to generate the final segmentation output.

[0107] During training, a loss is computed for all supervised outputs to jointly optimize shallow and deep representations. Specifically, this invention employs a hybrid loss function that combines Dice loss and binary cross-entropy (BCE) loss, and defines the overall training objective as:

[0108] ;

[0109] in, Represent the objective function; Represents the weighting coefficient, and ; Represents the weighting coefficient, and ; This indicates that the data has been processed using the Dice loss function; This indicates that the data has been processed using the binary cross-entropy loss function, which aims to emphasize structural overlap while maintaining pixel-level classification accuracy. This indicates the actual label.

[0110] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0111] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0112] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution, characterized in that, The method includes the following steps: Step 1: Use the original image sequence as the first scale sequence; use the downsampling operator to downsample the first scale sequence one by one to obtain the second scale sequence, the third scale sequence, and the fourth scale sequence respectively. Based on sequences at the first, second, third, and fourth scales, temporal feature representations are obtained through processing by a temporal convolutional network. Step 2: Perform time series feature splitting and aggregation processing on the time series feature representation to obtain time series feature maps at four scales; Step 3: Encode the temporal feature maps at four scales using an encoder to obtain the original bottleneck feature maps; Step 4: Perform parallel branch convolution processing on the original bottleneck feature map using a multi-scale serpentine convolution module, followed by attention-gated progressive fusion processing and residual weighting processing to obtain the processed bottleneck feature map. The specific steps are as follows: The original bottleneck feature map is input into the four parallel branches of the multi-scale snake convolution module. Each parallel branch processes the original bottleneck feature map using deformable dynamic snake convolution to obtain the output feature map of each parallel branch. Starting from the first parallel branch, the output feature map of the current parallel branch is fused with the aggregated feature map of the previous parallel branch through attention gating operation to obtain the aggregated feature map of the current parallel branch. After completing the fusion process of all parallel branches, the final aggregated feature is obtained; By using a learnable residual weighting mechanism, the final aggregated features are combined with the original bottleneck feature map to obtain the processed bottleneck feature map. Step 5: Input the processed bottleneck feature map into the decoder, and obtain the final segmentation result map through step-by-step upsampling and dense fusion processing.

2. The blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution according to claim 1, characterized in that, In step 1, the original image sequence is used as the sequence at the first scale. The first scale sequence is then downsampled sequentially using a downsampling operator to obtain the sequences at the second, third, and fourth scales, respectively. The corresponding process follows the following relationship: ； in, This represents the sequence at the first scale. Represents the original image sequence. Indicates the first Sequences of various scales Indicates the first Sequences of various scales This indicates processing via a convolution operation with a stride of 2; Based on the sequences at the first, second, third, and fourth scales, temporal feature representations are obtained through processing using a temporal convolutional network. The specific steps are as follows: The temporal convolutional network consists of two serially cascaded temporal extraction modules, with the dilation rate of each module increasing exponentially. Each temporal extraction module contains two identical dilated causal convolutional layers, and the two dilated causal convolutional layers share the same dilation rate. The temporal extraction module consists of two convolutional layers connected in series, and after each convolutional operation, weight normalization, ReLU activation function, and random dropout processing are performed sequentially. The sequence at the current scale is concatenated frame by frame with the temporal feature sequence at the previous scale, and then fused through 1×1 convolution to obtain the fused feature sequence. The fused feature sequence is reshaped into a dimensionally reshaped feature sequence; the dimensionally reshaped feature sequence is then input into the first temporal extraction module for processing to obtain the output of the first temporal extraction module. The output of the first time series extraction module is used as the input of the second time series extraction module. After stacking the two time series extraction modules, the time series feature representation is obtained.

3. The blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution according to claim 2, characterized in that, After concatenating the current scale sequence with the temporal feature sequence of the previous scale frame by frame, and then fusing them through 1×1 convolution to obtain the fused feature sequence, the following relationship exists: ； in, Indicates the first The first scale The fused feature sequence of frames, This represents the learnable weights of a 1×1 convolutional kernel. Indicates the first The first scale The temporal feature sequence of frames, Indicates the first The first scale A sequence of frames, Indicates the total number of frames. This indicates channel-by-channel splicing processing; In the process of inputting the dimension-reshaped feature sequence into the first temporal extraction module for processing to obtain the output of the first temporal extraction module, the following relationship exists: ； in, This represents the intermediate output features after the first convolutional transformation. This represents the output of the first timing extraction module. This indicates weight normalization processing. This indicates activation function processing. This indicates that the convolution kernel is... void ratio The processing of dilated causal convolution, Indicates the reshaped first Feature sequences at each scale This indicates processing via residual mapping. This indicates random discarding. In the process of dilated causal convolution, the definition of causal convolution is: ； in, Indicates the first Causal convolution processing of frames The sliding index variable represents the convolution kernel. Indicates the total size of the convolution kernel. Indicates the convolution kernel at the index The weighting coefficient at the location, This indicates that the input feature sequence is at time [time]. The sampled value at that location.

4. The blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution according to claim 3, characterized in that, In step 2, the temporal feature representation undergoes temporal feature splitting and aggregation processing to obtain temporal feature maps at four scales. The specific steps are as follows: For each frame representing the temporal features at the current scale, spatial downsampling is performed using a 2×2 convolution with a stride of 2 to obtain the temporal feature sequence at the current scale; the temporal feature sequence at the current scale is then used to participate in the temporal convolutional network processing at the next scale. All temporal feature sequences are accumulated to obtain an accumulated temporal feature sequence; the accumulated temporal feature sequence is processed by 1×1 convolution to obtain a pixel-level score map; The pixel-level score map is normalized along the temporal dimension to obtain attention weights; using the attention weights and temporal feature sequences, the data are aggregated to obtain temporal feature maps at four scales.

5. The blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution according to claim 4, characterized in that, In the process of spatial downsampling using a 2×2 convolution with a stride of 2 for each frame representing the temporal features at the current scale, to obtain the temporal feature sequence at the current scale, the following relationship exists: ； in, Indicates the first The first scale The temporal feature sequence of frames, This indicates processing using a 2×2 convolution with a stride of 2. Indicates the first The first scale Temporal characteristics of frames; In the process of processing the accumulated temporal feature sequence through 1×1 convolution to obtain the pixel-level score map, the following relationship exists: ； in, Represents a pixel-level score image. This indicates processing via 1×1 convolution. Indicates the first A time-series feature sequence accumulated at multiple scales; In the process of using attention weights and temporal feature sequences, and then aggregating them to obtain temporal feature maps at four scales, the following relationship exists: ； in, Indicates the first Temporal feature maps at various scales, Indicates the first Attention weights of frames This indicates element-wise multiplication.

6. The blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution according to claim 5, characterized in that, In step 3, the temporal feature maps at four scales are encoded using an encoder to obtain the original bottleneck feature map. The specific steps are as follows: The encoder consists of four levels, each of which consists of two consecutive 3×3 convolutional layers and a stride convolution with a stride of 2. The coding levels are connected by dense skip connections. The temporal feature map of the first scale is input into the first layer of the encoder, processed by two consecutive 3×3 convolutional layers, and then batch normalized and ReLU activated in sequence to obtain the intermediate feature map of the first layer. By using strided convolution with a stride of 2, the spatial resolution of the intermediate feature map of the first layer is reduced to obtain the output feature map of the first layer. The temporal feature map of the second scale is input into the second layer of the encoder. After being processed by two consecutive 3×3 convolutional layers, it is batch normalized and ReLU activated in sequence to obtain the intermediate feature map of the second layer. The intermediate feature map of the second layer is reduced in spatial resolution using stride convolution with a stride of 2 to obtain the feature map of the second layer. The output feature map of the first layer and the feature map of the second layer are channel adjusted and fused using 1×1 convolution to obtain the output feature map of the second layer. The temporal feature map of the third scale is input into the third level of the encoder. After being processed by two consecutive 3×3 convolutional layers, it is batch normalized and ReLU activated in sequence to obtain the intermediate feature map of the third level. The spatial resolution of the intermediate feature map of the third level is reduced by using stride convolution with a stride of 2 to obtain the feature map of the third level. The output feature map of the second level and the feature map of the third level are channel adjusted and fused by 1×1 convolution to obtain the output feature map of the third level. The temporal feature map of the fourth scale is input into the fourth level of the encoder, processed by two consecutive 3×3 convolutional layers, and then batch normalized and ReLU activated in sequence to obtain the intermediate feature map of the fourth level. The spatial resolution of the intermediate feature map of the fourth level is reduced by using stride convolution with a stride of 2 to obtain the feature map of the fourth level. Using 1×1 convolution, the output feature map of the third layer and the feature map of the fourth layer are processed by channel adjustment and fusion to obtain the output feature map of the fourth layer; the output feature map of the fourth layer is used as the original bottleneck feature map.

7. The blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution according to claim 6, characterized in that, In the process of processing the original bottleneck feature map using deformable dynamic serpentine convolution in each parallel branch to obtain the output feature map of each parallel branch, the following relationship exists: ； in, Indicates the first Output feature maps of parallel branches This indicates batch normalization processing. This indicates that the convolution kernel size is... The initial direction of offset learning is Deformable dynamic serpentine convolution processing, This represents the original bottleneck feature map. Indicates the first The kernel size of deformable dynamic serpentine convolutions in each parallel branch. Indicates the first The offset learning of the initial direction of deformable dynamic serpentine convolution in each parallel branch; In each parallel branch, the processing of the original bottleneck feature map by the deformable dynamic serpentine convolution is expressed by the following relationship: ； in, This represents the offset of the original bottleneck feature map in dynamic prediction. Indicates size is The learnable convolution weights for the sampling points corresponding to the convolution kernel. This represents the current position on the original bottleneck feature map. Relative to The size is The kernel offset, This represents a convolution operation that calculates the offset. Describing deformable dynamic serpentine convolution in Convolution calculation at the point, Representation of feature map The offset position; Starting from the first parallel branch, the process of fusing the output feature map of the current parallel branch with the aggregated feature map of the previous parallel branch through attention gating to obtain the aggregated feature map of the current parallel branch has the following relationship: ； in, This represents the output feature map of the first parallel branch. The aggregation features of the first parallel branch, Indicates the first Aggregation features of parallel branches Indicates the first Aggregation features of parallel branches Indicates the first Output feature maps of parallel branches This indicates processing via attention gating; In the process of obtaining the aggregated features of the current parallel branch, the attention gating's processing of the input features is expressed by the following relation: ； in, Indicates a gating signal. This represents the features after performing a 1×1 convolution transformation on the gated signal. This represents the output feature map of the current parallel branch. This represents the features after performing a 1×1 convolution transformation on the output feature map of the current parallel branch. This represents the 1×1 convolution weight applied to the gated signal. This represents the 1×1 convolution weights applied to the output feature map of the current parallel branch. Representing a spatial attention map, This indicates that it is processed using the Sigmoid function. This indicates processing via 1×1 convolution; In the process of combining the final aggregated features with the original bottleneck feature map through a learnable residual weighting mechanism to obtain the processed bottleneck feature map, the following relationship exists: ； in, This represents the processed bottleneck feature map. Indicates learnable parameters, This represents the final aggregated feature.

8. The blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution according to claim 7, characterized in that, In step 5, the processed bottleneck feature map is input to the decoder, and the final segmentation result map is obtained through step-by-step upsampling and dense fusion processing. The specific steps are as follows: The decoder consists of four stages. Except for the first stage, which includes dense fusion processing, each subsequent stage includes progressive upsampling processing and dense fusion processing. The processed bottleneck feature map is used as the feature map of the first stage upsampling, and it is concatenated with the intermediate feature map of the first level along the channel dimension to obtain the decoding feature map of the first stage. After refining the decoding feature map of the first stage through two consecutive 3×3 convolutional layers, the intermediate prediction values ​​of the first stage are obtained through batch normalization and ReLU activation. The decoded feature map of the first stage is continuously upsampled using bilinear interpolation to obtain the upsampled feature map of the second stage. The second-stage upsampled feature map, the intermediate feature map of the second level, and the first-stage decoded feature map are concatenated along the channel dimension to obtain the second-stage decoded feature map. After refining the decoding feature map of the second stage through two consecutive 3×3 convolutional layers, the intermediate prediction values ​​of the second stage are obtained through batch normalization and ReLU activation. The decoded feature map of the second stage is continuously upsampled using bilinear interpolation to obtain the feature map of the third stage. The feature map of the third stage upsampling, the intermediate feature map of the third level, and the decoding feature map of the second stage are concatenated along the channel dimension to obtain the decoding feature map of the third stage. After refining the decoding feature map of the third stage through two consecutive 3×3 convolutional layers, the intermediate prediction values ​​of the third stage are obtained through batch normalization and ReLU activation. The decoded feature map of the third stage is continuously upsampled using bilinear interpolation to obtain the upsampled feature map of the fourth stage. The feature map of the fourth stage upsampling, the intermediate feature map of the fourth level, and the decoding feature map of the third stage are concatenated along the channel dimension to obtain the decoding feature map of the fourth stage. After refining the decoding feature map of the fourth stage through two consecutive 3×3 convolutional layers, the intermediate prediction values ​​of the fourth stage are obtained through batch normalization and ReLU activation. The intermediate predicted values ​​from the first stage, the second stage, the third stage, and the fourth stage are averaged and then processed by the Sigmoid activation function to obtain the final segmentation result image.

9. The blood vessel image segmentation method based on spatiotemporal collaboration and multi-scale serpentine convolution according to claim 8, characterized in that, After averaging the intermediate predicted values ​​from the first, second, third, and fourth stages, and then processing them through the Sigmoid activation function to obtain the final segmentation result image, the following relationship exists: ； in, This represents the final segmentation result image. Indicates the first Intermediate forecast values ​​for the stage.

Images