Microvessel cross-section segmentation method

Abstract

This invention relates to a method for cross-slice segmentation of microvessels, comprising: segmenting a three-dimensional microvessel image block into sequential two-dimensional slices, using the batch dimension in a deep learning framework as the slice dimension; based on the sequential two-dimensional slices, performing depth extraction of local fine features on the two-dimensional plane using a fully convolutional densely connected network, and downsampling through a transition layer; inserting a cross-slice multi-dimensional attention module at the skip connection between the encoder and decoder to perform multi-dimensional attention feature enhancement; and based on feature concatenation decoding and microvessel reconstruction, gradually mapping and restoring the deep, low-resolution abstract semantic features into a high-precision segmentation mask with the same resolution as the original input. This invention can significantly reduce computational power consumption and improve the three-dimensional spatial continuity of microvessels.

Description

Technical Field

[0001] This invention relates to a method for cross-slice segmentation of microvessels. Background Technology

[0002] Currently, in the process of extracting three-dimensional microvessels or similar topological structures, various imaging techniques (such as optical microscopy and magnetic resonance imaging) inevitably encounter a series of spatial and optical interference problems. Taking "two-photon microscopy (2PM)," which is widely used in this field and has the most representative interference characteristics, as an example, it mainly faces the following three typical interferences when acquiring the three-dimensional capillary network of the mouse brain: first, "anisotropy" with a significantly lower Z-axis resolution than the XY plane; second, "background noise" caused by deep imaging; and third, "imaging artifacts" caused by surrounding tissues (such as the meninges).

[0003] The aforementioned problems are highly prevalent in 3D medical imaging. Therefore, how to extract the foreground (such as microvascular networks) with a realistic 3D spatial topology completely, accurately, and with low computational consumption while overcoming complex environmental interferences such as anisotropy, background noise, and imaging artifacts has long been a core technical challenge in this field.

[0004] For the task of extracting the foreground of three-dimensional microvessels, the existing technologies mainly include the following three types: (1) A segmentation extraction scheme using two-dimensional (2D) fully convolutional networks as the backbone network is proposed. This scheme performs pixel-level microvascular foreground segmentation on two-dimensional slices and achieves good results in 2D planar segmentation by combining the classic U-shaped network framework (UNet). Among them, densely connected fully convolutional networks (such as 2D FC-DenseNet) have always been a typical and commonly used scheme in 2D fully convolutional segmentation by taking advantage of the highly reusable features of densely connected modules (Dense Block).

[0005] (2) Extraction scheme based on three-dimensional (3D) fully convolutional network. This scheme extracts the spatial structure of blood vessels directly from three-dimensional volume data through 3D convolutional kernels and trains with complete 3D voxels. Among them, 3D-VNet is the most common and mature model for three-dimensional medical image segmentation.

[0006] (3) Segmentation and extraction scheme based on simple stacking (2.5D) of multi-layer 2D planar input. This scheme splices adjacent slices in the channel dimension and inputs them into a 2D network for foreground extraction. Combining the advantages of 2D networks having lower computational requirements and 3D networks preserving spatial topological features, the 2D network model usually selected is the one described in Scheme 1.

[0007] When extracting microvessels from complex optical interference environments generated by two-photon imaging, the aforementioned classical methods all have significant limitations: (1) 2D FC-DenseNet (lacking three-dimensional spatial information): Although this scheme can extract microvascular features in detail on a single 2D slice, it completely loses the topological continuity of microvessels in three-dimensional space because it is processed independently layer by layer. This makes it very easy for the extracted foreground to break or produce incorrect vascular structure adhesions on the Z-axis.

[0008] (2) 3D V-Net (Introducing anisotropic interference and requiring high computational power): This scheme aims to preserve three-dimensional spatial information, but two-photon imaging generally suffers from anisotropy (Z-axis stretching). Directly using 3D convolutional kernels to forcibly extract three-dimensional features will cause severe interference to the network in the Z-axis direction, resulting in blurred foreground edges and introducing a large number of convolution artifacts. At the same time, during the computation process, the number of network parameters, the number of floating-point operations (FLOPs), and the three-dimensional feature maps generated during forward propagation will all lead to an exponential increase in memory consumption. When faced with high-resolution, large-scale microvascular network data such as the whole mouse brain, preserving the complete tissue as input is extremely computationally expensive.

[0009] (3) Simple 2.5D scheme (introducing noise and artifacts from adjacent layers): Based on the problems of pure 2D convolution and 3D convolution in 3D vessel segmentation tasks, the classic improved scheme attempts to find a balance between preserving the complete 3D structure of the vessel (through adjacent slices) and avoiding 3D convolution artifacts. However, since it simply aggregates and extracts multiple slices equally, it lacks the ability to dynamically filter slice features. This leads to background noise and highly reflective artifacts (such as meninges) in adjacent slices being indiscriminately introduced into the foreground extraction of the current layer, which correspondingly reduces the correct extraction rate of the foreground. Summary of the Invention

[0010] In view of this, it is necessary to provide a method for segmenting microvessels across slices.

[0011] This invention provides a method for cross-slice segmentation of microvessels, comprising: step S1, dividing a three-dimensional microvessel image block into sequential two-dimensional slices, using the batch dimension in a deep learning framework as the slice dimension; step S2, based on the sequential two-dimensional slices, performing depth extraction of local fine features on the two-dimensional plane using a fully convolutional densely connected network, and performing downsampling through a transition layer; step S3, inserting a cross-slice multi-dimensional attention module at the jump connection between the encoder and decoder to perform multi-dimensional attention feature enhancement; and step S4, based on feature concatenation decoding and microvessel reconstruction, gradually mapping and restoring the deep, low-resolution abstract semantic features into a high-precision segmentation mask with the same resolution as the original input.

[0012] Step S1 includes: The 3D microvascular image patch is segmented along the Z-axis into sequential 2D slices, and the batch dimension in the deep learning framework is used as the slice dimension. That is, the shape of the input tensor is defined as [S, C, H, W], where Number of slices For the number of channels, and The height and width of the space.

[0013] Step S1 includes: Step S11: Slide to capture 3D data blocks: Read the raw three-dimensional two-photon microvascular data; slice along the depth direction according to the set number of slices. By using a sliding window to dynamically extract a continuous sequence of slices with a specified overlap step, a shape of [shape not specified] is obtained. The three-dimensional data block; wherein, the dimensional format of the original three-dimensional two-photon microvascular data is as follows: That is, height, width, and total depth; Step S12: Reconstruct the physical dimension: When converting to a network input tensor, the depth dimension is converted using a matrix transpose operation. Move it to the first position to transform the data block into Subsequently, a dimensionality-up operation was used to increase the single-channel dimension. , forming a shape The four-dimensional tensor; Step S13: Perform dimension unpacking on the data for which batch dimensions were automatically added: In the typical data loading pipeline of deep learning frameworks, data is usually automatically batch-dimensioned. Before entering the network, a dimension unpacking operation is performed to forcibly remove the batch dimensions allocated by the deep learning framework. This allows the tensor input to the network to be restored to its original state. .

[0014] Step S2 includes: Using a fully convolutional densely connected network as the backbone network, each layer in the downsampling path of the backbone network performs depth extraction of local fine features in the XY two-dimensional plane through dense blocks, and performs downsampling through transition layers, so that the microvascular cross-sectional features of each local slice are efficiently captured.

[0015] Step S2 includes: Step S21: Initial feature mapping: Sequence tensor First, the initial two-dimensional convolutional layer is used to perform preliminary spatial feature extraction, generating a shallow feature tensor. Step S22: High-frequency feature reuse in dense blocks: The feature tensor passes through multiple downsampling stages in sequence; in each stage, the core mechanism is a dense block; inside the dense block, there are multiple convolutional units connected in sequence; feature concatenation: the input of each convolutional unit in the dense block is the concatenation of the output feature maps of all the preceding layers in the channel dimension. Step S23: Transitional downsampling and feature transfer: After each dense block, a transition downsampling layer is connected to reduce the spatial resolution of the feature map, expand the planar receptive field of the neuron, and thus aggregate a larger range of two-dimensional spatial context. Step S24: Direct the deep two-dimensional feature maps extracted from each dense block to the CSAM module.

[0016] The convolutional unit is a cascaded structure of an instance normalization layer, an activation layer, a two-dimensional convolutional layer, and a feature discarding layer.

[0017] Step S3 includes: The cross-slice multidimensional attention module sequentially reweights the feature map of the current dimension through three sequential semantic / channel attention modules, position / spatial attention modules, and slice attention modules.

[0018] Step S3 includes: The semantic / channel attention module is used to compress slices and spatial dimensions through global max pooling and average pooling, and to learn the importance between channels using a multilayer perceptron, thereby focusing on the core semantic features of microvessels at the global level. The location / spatial attention module is used to compress features along the channel and slice dimensions, and generate a spatial attention map using a two-dimensional convolutional layer to locate the high-frequency response region of microvessels in the XY plane. The slice attention module is used to model the three-dimensional continuity between slices and suppress interference; after feature extraction, MLP is used to predict the mean vector of a low-rank multivariate Gaussian distribution. Diagonal variance vector and low-rank factor matrix The covariance matrix is ​​constructed; during the training phase, a reparameterization technique is used for sampling, and during the inference phase, the mean is directly output; finally, dynamic slice weights are generated through activation functions to suppress slice features containing highly reflective meninges or strong noise.

[0019] Step S4 includes: In the decoder path, deep features are first upsampled through deconvolution to restore spatial resolution. Then, the upsampled features are concatenated with the corresponding layer's skip connection features, which have already been enhanced and filtered by the CSAM module, along the channel dimension. The concatenated features are further fused through dense blocks and upsampled layer by layer, ultimately resulting in a final output. The convolutional layer outputs the classification probability of each pixel, thereby generating a complete microvascular segmentation mask with three-dimensional connectivity.

[0020] Step S4 includes: Step S41: Transitional upsampling to restore spatial resolution: For the deepest feature tensor at the bottom of the U-shaped network, or the feature tensor output by the previous decoding layer, a transition upsampling layer is used to process it; this layer mainly includes transposed convolution operation; data flow: while the spatial resolution of the feature map is multiplied by a set stride, the number of feature channels is compressed and the dimension is reduced, providing a basis for physical size alignment for subsequent feature fusion. Step S42: Introducing cross-layer stitching of CSAM high-fidelity features: Align the deep feature tensor obtained after upsampling in step S41 with the feature tensor from the same layer of the network, which comes from the skip connection; Data flow: Strictly concatenate the above two sets of feature tensors in the channel dimension to form a composite feature tensor containing multi-scale semantics and high-fidelity local structural information. Step S43: Nonlinear deep fusion based on decoder dense blocks: Simple concatenation of channel dimensions cannot directly guide pixel classification. The concatenated composite feature tensor is then input into the dense block of the current level of the decoder, thereby accurately reconstructing the true boundary of microvessels while reducing noise. Step S44: Layer-by-layer spatial reconstruction and foreground probability output: Repeat the above "upsampling" process CSAM feature splicing The decoding process of "dense block fusion" continues until the spatial resolution of the feature tensor is completely restored to the size of the original input data block; final mapping: the highest resolution feature tensor after fusion is mapped through a... The output convolutional layer; the final output tensor represents the predicted probability that each pixel in the spatial sequence belongs to the microvessel foreground. Based on this, a high-precision microvessel mask with complete 3D topological connectivity is generated through binarization, completing the entire segmentation process. The beneficial effects of this application include: (1) The integrity of the three-dimensional spatial structure is preserved: Compared with 2D-FC-DenseNet, this invention effectively aggregates the contextual information between slices by organically combining 2.5D sequence input with CSAM mechanism. The extracted microvascular foreground maintains extremely high connectivity on the Z-axis, overcoming the problem of blood vessel breakage caused by pure 2D methods.

[0021] (2) Avoids blurring and artifacts caused by anisotropy: Compared with the 3D V-Net scheme, this invention avoids direct 3D convolution calculation. By combining the 2.5D serialization input strategy with the CSAM module, it processes three-dimensional relationships in the dimensionality reduction space, which not only greatly reduces the consumption of video memory, but also fundamentally avoids the blurring of the foreground boundary caused by two-photon anisotropy.

[0022] (3) Achieved high-purity foreground extraction and noise reduction: Compared with simple 2.5D indiscriminate aggregation, this invention achieves dynamic perception of slice quality through a specific probability distribution sampling mechanism. While utilizing three-dimensional spatial information, it successfully blocks the penetration of background noise from adjacent slices and high-reflectivity artifacts into the central slice, significantly improving the purity of the extracted vascular foreground.

[0023] (4) Overcoming the computational bottleneck of large-scale data expansion: This invention abandons the energy-intensive 3D convolution operation and utilizes a strategy of "2.5D serialized input + multi-dimensional attention enhancement". Features are extracted in the XY plane using a 2D densely connected network with extremely low computational consumption, and context association is completed in the Z-axis using an attention mechanism. Compared with the 3D solution, the memory consumption of this architecture is reduced by several times, and it can process larger-scale image patches on conventional computing devices, greatly avoiding the microvascular rupture problem caused by whole-brain data slicing, and has extremely high engineering application value. Attached Figure Description

[0024] Figure 1 This is a flowchart of the microvascular cross-slice segmentation method of the present invention; Figure 2 This is a schematic diagram illustrating the data flow of serialized data input reconstruction according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the architecture of a fully convolutional densely connected network according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the semantic / channel attention module structure according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the location / spatial attention module structure according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the slice attention module structure in an embodiment of the present invention. Detailed Implementation

[0025] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0026] See Figure 1 The diagram shown is a flowchart of a preferred embodiment of the microvascular cross-slice segmentation method of the present invention.

[0027] Step S1 involves segmenting the 3D microvascular image chunk into sequential 2D slices, using the batch dimension from the deep learning framework as the slice dimension. Specifically, the 3D microvascular image chunk is segmented along the Z-axis into sequential 2D slices, and the batch dimension from the deep learning framework is innovatively used as the slice dimension. The shape of the input tensor is defined as [S, C, H, W] (where...). Number of slices For the number of channels, and (Height and width in space). This step constructs a data format that can be efficiently processed in parallel by 2D networks while preserving the physical adjacency sequence relationship, and avoids energy-intensive 3D convolution operations. Please participate in this step. Figure 2 Specifically: Step S11: Slide to extract a 3D data chunk: Read raw three-dimensional two-photon microvascular data (its dimension format is usually ). (i.e., height, width, and total depth). Along the depth ( (Axis) direction, according to the set number of slices (i.e., data block depth, such as) The specified slice overlap step is used to dynamically extract a continuous sequence of slices using a sliding window, resulting in a shape of... 3D data blocks.

[0028] Step S12: Reconstructing the physical dimension (core data flow technology): When converting it into a network input tensor, this embodiment abandons the conventional approach of "stacking along the channel dimension". Instead, it uses a matrix transpose operation to convert the depth dimension... Move it to the first position to transform the data block into Then, a dimensionality-up operation was used to increase the single-channel dimension. , forming a shape The four-dimensional tensor.

[0029] Step S13: Perform dimension unpacking on the data for which batch dimensions were automatically added: In the typical data loading pipeline of deep learning frameworks (such as PyTorch), data is usually automatically loaded with batch dimensions to form a batch data structure. (Typically, one chunk is loaded at a time during validation / training, i.e.) Before entering the network, a dimension unpacking operation is performed to forcibly remove the batch dimensions allocated by the deep learning framework. This allows the tensor input to the network to be restored to its original state. Technical effect: The role of this dimensional reconstruction is to improve the depth dimension for subsequent 2D backbone networks. Logically, it was "disguised" as batch size. The network will simultaneously and independently... Parallel two-dimensional convolution of slices greatly improves computational efficiency; simultaneously, the... The slices strictly maintain the original three-dimensional physical adjacency order in memory, providing a sequence basis for subsequent modules to extract inter-layer context.

[0030] Step S2 involves performing depth extraction of local fine features on the two-dimensional plane based on the sequenced two-dimensional slices using a fully convolutional densely connected network (FC-DenseNet), followed by downsampling through a transition layer. Specifically, a FC-DenseNet is used as the backbone network. In the downsampling path of the backbone network, each layer performs depth extraction of local fine features on the XY two-dimensional plane using dense blocks, followed by downsampling through a transition layer. This efficiently captures the microvascular cross-sectional features of each local slice. This step enables extremely fine capture of very subtle capillary edges and high-frequency morphological features within a single slice plane. Specifically, it includes: Step S21: Initial feature mapping: Sequence tensor First, the slices pass through an initial 2D convolutional layer (First Conv) for preliminary spatial feature extraction, generating a shallow feature tensor with a certain number of channels. At this stage, the operations are completely independent between slices.

[0031] Step S22: High-frequency feature multiplexing of dense blocks: The feature tensor passes through multiple downsampling stages sequentially. The core mechanism in each stage is a dense block. Within the dense block, multiple convolutional units are connected in series (typically a cascade of batch normalization layers (BatchNorm), ReLU activation layers (ReLU), and 2D convolutional layers (Conv2D). In this embodiment, an innovative cascade of instance normalization layers (InstanceNorm), ReLU activation layers (ReLU), 2D convolutional layers (Conv2D), and dropout layers (Dropout) is used to accommodate cross-slice input processing). Feature concatenation: The input to each convolutional unit within the dense block is a concatenation of the output feature maps from all preceding layers along the channel dimension. Technical effect: Capillary foregrounds in two-photon microscopy images are extremely small and have low contrast. In traditional networks, these high-frequency edge details are easily smoothed or lost as the number of layers increases. The dense connection mechanism forces the network to reuse high-resolution edge features from extremely shallow layers in each layer, thereby maximizing the preservation of the planar morphological features of microvessels.

[0032] Step S23: Transition Down and Feature Transfer: After each dense block, a transition downsampling layer (containing) is connected. Convolutional dimensionality reduction and max pooling operations are used to reduce the spatial resolution of feature maps. This expands the planar receptive field of neurons, thereby aggregating a larger range of two-dimensional spatial context. See the appendix for the specific structure of the encoder path. Figure 3 Left-side downsampling path.

[0033] Step S24: Direct the deep 2D feature maps extracted from each dense block to the CSAM module: The deep two-dimensional feature map extracted from each dense block (at this point, the shape still remains as shown) Instead of being directly fed to the decoder via a skip connection, it is diverted as input to the CSAM in step S3, in order to further aggregate three-dimensional spatial connectivity and eliminate optical noise caused by independent layers based on two-dimensional high-frequency features.

[0034] Step S3: Enhancement of multi-dimensional attention features based on CSAM (skip connection part). That is, at the skip connection between the encoder and decoder, a cross-slice attention module (CSAM) is inserted (see appendix for details of the CSAM internal structure). Figure 4 ~Appendix Figure 6This cross-slice multidimensional attention module reweights the feature map of the current dimension sequentially through three serial sub-attention branches: (1) Semantic / Channel Attention Module: By compressing slices and spatial dimensions through global max pooling and average pooling, the importance between channels is learned by multilayer perceptron (MLP), thereby focusing on the core semantic features of microvessels at the global level.

[0035] (2) Positional / Spatial Attention Module: Compress features along the channel and slice dimensions, generate spatial attention maps using two-dimensional convolutional layers, and locate the high-frequency response regions (ROIs) of microvessels in the XY plane.

[0036] (3) Slice Attention Module: Used to model the three-dimensional continuity between slices and suppress interference. After feature extraction, the mean vector of a low-rank multivariate Gaussian distribution is predicted using MLP. Diagonal variance vector and low-rank factor matrix And construct the covariance matrix During the training phase, a reparameterization technique is used for sampling, while the mean is directly output during the inference phase. Finally, a dynamic slice weight is generated using an activation function (Sigmoid) to suppress slice features containing highly reflective dura mater or strong noise. Specifically: Suppose the original feature tensor input to CSAM is (in This represents the slice depth. For the number of channels, (for spatial resolution). The original feature tensor undergoes feature recalibration sequentially through the following three sub-modules: Step S31: Specific implementation of the Semantic / Channel Attention Module: This semantic / channel attention module is used to extract the feature channels most sensitive to the microvascular foreground. The structure of this semantic / channel attention module is shown in the attached figure. Figure 4 As shown.

[0037] Step S311: Multidimensional feature compression: Along the slice ( ) and space ( Three dimensions, simultaneously processing the input tensor Global max pooling (GMP) and global average pooling (GAP) are performed to compress the features into two shapes. The channel descriptor vector.

[0038] Step S312: Channel weight generation: These two vectors are input separately into a multilayer perceptron (MLP) with shared weights (including dimensionality reduction and expansion operations) for feature fusion. The two outputs of the MLP are then element-wise summed, and a channel weight vector is generated by passing the sigmoid activation function. .

[0039] Step S313: Feature recalibration: Will With the original input Perform a broadcast multiply to obtain the channel-enhanced feature tensor. .

[0040] Step S32: Specific implementation of the Positional / Spatial Attention module: After acquiring the key channels, this location / spatial attention module is used to pinpoint the specific spatial distribution of microvessels on a two-dimensional plane. The structure of this location / spatial attention module is shown in the attached figure. Figure 5 As shown.

[0041] Step S321: Cross-channel and slice compression: Receive the output from the previous step This time, along the slice ( ) and channels ( ) dimension, respectively perform global max pooling and global average pooling, to obtain two shapes. A two-dimensional spatial distribution map.

[0042] Step S322: Spatial weight generation: These two 2D images are concatenated along the channel dimension to form a shape of... The feature map. Then, it is passed through a two-dimensional convolutional layer with a large receptive field (e.g., The convolution kernel reduces the dimension to a single channel, and then passes it through a sigmoid activation function to generate a spatial weight matrix. .

[0043] Step S323: Feature recalibration: Will With features Perform broadcast multiplication to obtain the spatially augmented feature tensor. .

[0044] Step S33: Specific implementation of the Slice Attention Module based on probability distribution sampling: To accurately assess the importance of different slices in the 3D context and dynamically suppress low-quality slices containing background noise or imaging artifacts, a probability distribution weight generation strategy based on variational inference was designed in the Slice Attention Module of CSAM. This strategy does not directly predict deterministic weight values, but instead models the probability distribution of the potential importance of slices and introduces uncertainty awareness through a sampling mechanism. The probability distribution calculation and reparameterized sampling process of this branch are attached. Figure 6 As shown.

[0045] Step S331: Slice feature compression and preliminary fusion: Continuing from the previous step's output (This is recorded as) (to simplify subsequent formula derivation), the input contains The feature tensor of each slice First, along the passage ( ) and space ( Global Max Pooling (GMP) and Global Average Pooling (GAP) are performed on each dimension to compress the complex information of each slice into a global feature descriptor, resulting in two shapes. eigenvectors.

[0046] Subsequently, the two feature vectors are input into a shared multilayer perceptron (MLP) for nonlinear transformation, and the transformed results are added together and fused to obtain an intermediate feature vector that aggregates global information of the slice. , here will It is treated as a feature dimension.

[0047] Step S332: Prediction of multivariate Gaussian distribution parameters: Using fused feature vectors This method infers the multivariate Gaussian distribution parameters of the potential importance of slices. This embodiment employs a low-rank approximation to construct the covariance matrix, reducing computational complexity and capturing the correlations between slices. Specifically, it includes three parallel branches: Mean prediction branch: Input the fully connected layer and predict the mean vector of the distribution. .

[0048] Independent variance prediction branch: The logarithm of the diagonal covariance predicted by the fully connected layer is input, then passed through the Softplus activation function and a minimum constant is added. (For example To ensure the stability and positive definiteness of the values, a vector of diagonal elements is obtained. .

[0049] Low-rank factor prediction branch: Input a fully connected layer and reshape the output to its desired shape. Low-rank factor matrix ,in The set rank (e.g., 5) is used to model the covariance relationship between different slices.

[0050] A low-rank multivariate Gaussian distribution is constructed using the above parameters. The covariance matrix .

[0051] Step S333: Reparameterization Sampling and Weight Generation: Generate latent weight variables based on the constructed probability distribution. .

[0052] During the model training phase: a reparameterization trick is used for sampling. Specifically, noise vectors are sampled from a standard normal distribution. and Through formula Calculate the latent weight variables (in (This indicates element-wise multiplication). This mechanism not only ensures the differentiability of the gradient during backpropagation, but also enables the model to reduce its dependence on high-variance (i.e., high uncertainty, low quality) slices during training.

[0053] In the model inference phase: to ensure the determinism of the result, the mean vector of the distribution is directly used as the latent variable, i.e. .

[0054] Finally, the latent variables Input the Sigmoid activation function and map it to The interval is used to obtain the final slice attention weight vector. .

[0055] Step S334: Feature reweighting: The generated weight vector Dimensional reshaping It is then multiplied element-wise by the original input feature tensor via a broadcasting mechanism. The algorithm dynamically adjusts the features of different slices and outputs an enhanced feature tensor. This enhanced feature tensor is then directly passed to the corresponding level of the decoder via a skip connection for feature stitching and image reconstruction in step S4.

[0056] Step S4, based on feature concatenation decoding and microvascular reconstruction, gradually maps and restores the deep, low-resolution abstract semantic features to a high-precision segmentation mask with the same resolution as the original input. That is, in the decoder path, deep features are first upsampled (transition up) through deconvolution to restore spatial resolution; then, the upsampled features are concatenated with the corresponding layer's skip connection features, which have been enhanced and filtered by the CSAM module, along the channel dimension. The concatenated features are further fused through dense blocks and upsampled layer by layer, ultimately resulting in a final image. The convolutional layer outputs the classification probability for each pixel, thereby generating a complete microvessel segmentation mask with 3D connectivity. Specifically: Step S41: Transition Up Sampling to Restore Spatial Resolution: For the deepest feature tensor at the bottom of the U-shaped network, or the feature tensor output by the previous decoding layer, a transition upsampling layer is used to process it. This layer mainly includes a transposed convolution operation. Data flow: This operation reduces the spatial resolution of the feature map ( While scaling up by a set step size, the number of feature channels is compressed and reduced in dimensionality to provide a basis for physical size alignment for subsequent feature fusion.

[0057] Step S42: Introduce cross-layer concatenation of CSAM high-fidelity features: This is a crucial interactive step for achieving precise microvascular reconstruction. The deep feature tensor obtained after upsampling in step S41 is aligned with the feature tensor from the same layer of the network, which comes from the skip connection. Key technology: The skip connection feature tensor here is the output result after recalibration by the CSAM module in step S3. Because this feature has undergone precise semantic channel and spatial location localization, as well as variational probability sampling filtering at the slice dimension, it not only preserves extremely high-resolution capillary edge details but also completely eliminates the strong reflection artifacts between adjacent slices caused by two-photon anisotropy, and includes the prior knowledge of three-dimensional spatial continuity between slices. Data flow: The above two sets of feature tensors are rigorously concatenated at the channel dimension (dim=1) to form a composite feature tensor containing multi-scale semantics and high-fidelity local structural information.

[0058] Step S43: Nonlinear deep fusion based on decoder dense blocks: Simple concatenation of channel dimensions cannot directly guide pixel classification. The concatenated composite feature tensor is then input into the dense block of the current level of the decoder. Technical effect: Relying on the feature reuse mechanism of multiple convolutional units within the dense block, the network is forced to perform nonlinear mapping and deep fusion of "deep global abstract features" and "3D local continuous features filtered by CSAM", thereby accurately reconstructing the true boundary of microvessels while reducing noise.

[0059] Step S44: Layer-by-layer spatial reconstruction and foreground probability output: The network performs the above "upsampling" in a loop. CSAM feature splicing The decoding process of "dense block fusion" continues until the spatial resolution of the feature tensor is completely restored to the size of the original input data block (i.e., the shape is restored to its original size). Final mapping: The fused highest-resolution feature tensor is mapped through a... The output convolutional layer. This layer will reduce the channel dimension. Dimensionality is reduced to the target number of categories (in this embodiment, the number of channels is reduced to 1 for the microvessel foreground extraction task). The final output tensor represents the predicted probability that each pixel in the spatial sequence belongs to the microvessel foreground. Based on this, a high-precision microvessel mask with complete three-dimensional topological connectivity is generated by binarization, completing the entire segmentation process.

[0060] exist Figure 2First, from the original 3D two-photon microvascular data (size H × W × total depth), a continuous sequence of slices is dynamically extracted using a sliding window, resulting in slice blocks of shape [H, W, S]. The extracted data blocks are then subjected to matrix transpose, moving the depth dimension S to the front, reconstructing it to [S, H, W]. After adding a single-channel dimension C through framework operations, the default batch dimension assigned by the deep learning framework is unpacked and forcibly removed. The original depth dimension S is logically disguised as the batch size, thus forming the final input tensor [S, C, H, W], allowing subsequent networks to independently and parallelly process S slices using 2D convolutions. The rightmost part of the diagram shows the stacking of input tensors of the same batch (i.e., slices in the same heap) across different channel dimensions (C).

[0061] exist Figure 3 The architecture adopts a U-shaped (U-Net) backbone, which mainly consists of four parts: encoder (downsampling path), cross-slice multidimensional attention module (CSAM), bottleneck layer (Bottleneck), and decoder (upsampling path).

[0062] Model Input: The input is a 3D microvascular image slice that has been reconstructed through 2.5D serialization.

[0063] The encoder consists of alternating cascaded dense blocks and downsampling blocks. Each dense block contains multiple sequentially connected feature extraction units: instance normalization (IN) - activation layer (ReLU) - convolutional layer (Conv) - drop layer (Drop). Dense blocks utilize feature concatenation (circles marked with "C" in the diagram) to achieve high reuse of local high-frequency, subtle features. The downsampling block contains: IN - ReLU - The “Conv-Dropout-Max Pool” structure is used to progressively reduce the spatial resolution of the feature map and expand the receptive field.

[0064] CSAM is forcibly embedded in the skip connect (shown by the dashed line in the diagram) between the encoder and decoder.

[0065] The CSAM module contains three sequential sub-attention branches for reweighting and enhancing the deep feature maps output by the encoder: Channel Attention (CA): Focuses on the core semantic channels of microvessels by sequentially using global max pooling and average pooling, MLP, and sigmoid activation function.

[0066] Spatial Attention (SPA): This involves sequentially performing global max pooling and average pooling, followed by feature concatenation. Large receptive field 2D convolution and sigmoid activation function are used to locate the high-frequency response region of microvessels in the XY plane.

[0067] Slice Attention (SA): It dynamically suppresses slice features containing highly reflective dura mater or strong noise by sequentially using global max pooling, multilayer perceptron (MLP), variational sampling (including reparameterization techniques), and sigmoid activation function.

[0068] The weight matrices output from the three branches are broadcast multiplied with the original features (as shown in the figure). (Symbols), to complete feature filtering.

[0069] The bottleneck layer (bottom) is located at the deepest part of the U-shaped network. It completes the transformation of the deepest abstract features through dense blocks, CSAM modules, and upsampling blocks.

[0070] The decoder (right-hand path) consists of alternating upsampling blocks and dense blocks. The upsampling blocks utilize... Transposed convolution ( Conv performs layer-by-layer spatial resolution recovery of deep features. Concatenate (C) merges the upsampled features with high-fidelity features from same-level skip connections, filtered and enhanced by the CSAM module, along the channel dimension. The merged features then undergo further non-linear deep fusion in the decoder's dense blocks to accurately reconstruct the true boundaries of microvessels. Model Output: The network ends through a... Convolutional layer ( Conv) maps the final high-resolution composite feature map to a microvessel binarization segmentation prediction mask for each pixel, thus completing the foreground extraction.

[0071] exist Figure 4 In the middle: the feature tensor that receives the original input. F Its dimensions are [S, C, H, W]. Feature tensor FGlobal max pooling (GMP) and global average pooling (GAP) operations are performed simultaneously along the slice (S) and spatial (H, W) dimensions, compressing the channel descriptor vectors into two vectors of shape [1, C, 1, 1]. These two channel descriptor vectors are then fed into a multilayer perceptron (MLP) with shared weights, where dimensionality reduction and expansion operations are performed to learn the importance of channels. The two outputs of the MLP are then element-wise added and passed through a sigmoid activation function to generate the final channel weight vector. (The shape is [1, C, 1, 1]). Finally, the generated channel weight vector... Compared with the original input features F Perform broadcast multiplication to output the feature tensor with enhanced channel / semantic features. F’ .

[0072] exist Figure 5 In the middle: Receive the feature tensor from the output of the previous step (semantic attention module). The input tensor has a shape of [S,C,H,W]. It undergoes parallel global max pooling (GMP) and global average pooling (GAP) operations along the slice (S) and channel (C) dimensions. These two pooling operations generate two two-dimensional spatial distribution maps with shapes [1, 1, H, W]. These two two-dimensional distribution maps are then concatenated along the channel dimension to form a combined feature map with a shape of [1,2, H, W]. The concatenated feature map is then reduced to a single channel through a large two-dimensional convolutional layer with a receptive field of 7×7, and then processed by the sigmoid activation function to generate a spatial weight matrix. The shape is [1, 1, H, W]. Finally, the spatial weight matrix is ​​multiplied using a broadcast multiply mechanism. Input features Element-wise multiplication yields the spatially augmented feature tensor. It can accurately locate the high-frequency response area of ​​microvessels on a two-dimensional plane.

[0073] exist Figure 6 In the middle: Receives an input feature tensor containing S slices. F’’ (or marked in the image) Global max pooling and average pooling are performed along the channel (C) and spatial (H, W) dimensions to compress the feature vectors into slices. These compressed feature vectors are then fed into a fully connected layer, where three parallel branches predict the parameters of the underlying low-rank multivariate Gaussian distribution, including the mean vector. Diagonal variance vector (i.e., the vector of diagonal elements) and low-rank factor matrix During the training phase, standard normal noise is introduced for reparameterization sampling, using the formula... Calculate the latent weight variables Latent variables After mapping using the Sigmoid activation function, the final slice attention weight vector is generated. The weight vector is then multiplied with the input feature tensor through a broadcast mechanism to perform dynamic reweighting of the slice features and obtain the final output, effectively suppressing slice features containing strong noise.

[0074] In other embodiments of the present invention: (1) Modified design of dense connection mechanism in backbone network: This invention relies on dense short connections within blocks in FC-DenseNet to ensure that microvascular features are not attenuated or lost in deep networks. While maintaining the design mechanism of "high feature reuse," the specific structural parameters of the backbone network can be equivalently replaced: Alternative designs: The feature growth rate (k) within the dense block can be dynamically adjusted based on computational resources and vessel resolution; the layer depth of the dense block, the feature compression factor in the transition layer, and the size of the bottleneck layer at the bottom of the U-shaped structure can all be configured with different specifications. The aforementioned network hyperparameter adjustments or local module variations based on the dense connection mechanism are all within the scope of protection of this invention.

[0075] (2) Changes in the internal implementation and serial / parallel order of the CSAM module: The innovative essence of the CSAM module lies in explicitly decoupling the attention mechanism into a joint weighted sum of three dimensions: "semantic channel," "spatial location," and "slice sequence." As long as this three-dimensional decoupling and joint mechanism is followed, changes in its internal technical details do not produce essential differences. Alternative to serial and parallel order: The preferred embodiment of the present invention adopts a serial computation order of "channel → space → slice", but in practical applications, the order of these three attention modules can be arbitrarily replaced (e.g., slice → channel → space), or a fusion method of multiplying / adding the weight matrices after parallel computation of the three branches can be adopted.

[0076] Replacement of internal feature extraction operators: The specific implementation techniques within each attention branch can be replaced in a conventional way. For example, replace "global max pooling and average pooling" in semantic channel attention with other pooling dimensionality reduction methods; replace the multilayer perceptron (MLP) for extracting spatial / slice weights with 1×1 convolutional layers, etc.

[0077] (Exclusive Explanation): It should be noted that the core feature of the CSAM module of this invention lies in the explicit decoupling and dimensional weighting of the channel, spatial, and slice dimensions. Global self-attention mechanisms that do not explicitly distinguish the above dimensions (such as the self-attention of pure Transformer) are fundamentally different from the technical approach of this invention.

[0078] (3) Other applications (cross-domain application expansion): The preferred embodiment of the present invention is for "extraction of three-dimensional microvascular network of mouse brain under two-photon microscopy", but this model aims to solve the technical problems of "anisotropy (low Z-axis resolution)", "fracture of microtubular structures" and "interlayer noise interference", and has wide applicability in other medical images with similar pain points: Extended Application 1: Other Microvascular Imaging and Extraction. It can be used for retinal microvascular network segmentation in optical coherence tomography (OCTA) and three-dimensional vessel extraction in photoacoustic imaging, etc.

[0079] Extended Application 2: Segmentation of Anisotropic Medical Body Data in Clinical Practice. The 2.5D dimensionality reduction strategy and cross-slice noise reduction mechanism of this invention can be directly applied to the processing of anisotropic MRI or CT images in clinical practice, such as prostate zone segmentation and cancerous lesion segmentation, placental MRI 3D reconstruction, and medical segmentation tasks involving multiple organs / lesions, including the heart. In these tasks, this solution also possesses significant technical value in reducing computational consumption and improving the continuity of 3D space.

[0080] This invention leverages the powerful feature extraction capabilities of Fully Convolutional Densely Connected Networks (FC-DenseNet) (ensuring microscopic details) and integrates a multi-dimensional attention enhancement mechanism from the Cross-Slice Attention Module (CSAM) (completing high-frequency details and 3D continuity across semantic, spatial, and slice dimensions). Through innovative data loading techniques, it fully utilizes the batch dimension in the PyTorch framework as the slice dimension to achieve cross-slice segmentation. When aggregating 3D spatial information, it accurately eliminates noise and artifacts introduced by adjacent slices, thereby achieving high-fidelity, high-purity 3D microvascular foreground extraction while ensuring efficient data utilization and significantly reducing computational consumption.

[0081] Although the present invention has been described with reference to the present preferred embodiments, those skilled in the art should understand that the above preferred embodiments are only used to illustrate the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for segmenting microvessels across slices, characterized in that, The method includes: Step S1: Divide the three-dimensional microvascular image block into sequential two-dimensional slices, using the batch dimension in the deep learning framework as the slice dimension. Step S2: Based on the serialized two-dimensional slices, perform depth extraction of local fine features on the two-dimensional plane using a fully convolutional dense connection network, and perform downsampling through a transition layer; Step S3: Insert a cross-slice multidimensional attention module at the skip connection between the encoder and decoder to perform multidimensional attention feature enhancement; Step S4, based on feature splicing decoding and microvascular reconstruction, gradually maps and restores the deep, low-resolution abstract semantic features into a high-precision segmentation mask with the same resolution as the original input.

2. The method as described in claim 1, characterized in that, Step S1 includes: The 3D microvascular image patch is segmented along the Z-axis into sequential 2D slices, and the batch dimension in the deep learning framework is used as the slice dimension. That is, the shape of the input tensor is defined as [S, C, H, W], where... Number of slices For the number of channels, and The height and width of the space.

3. The method as described in claim 2, characterized in that, Step S1 includes: Step S11: Slide to capture 3D data blocks: Read the raw 3D two-photon microvascular data; slice along the depth direction according to the set number of slices. By using a sliding window to dynamically extract a continuous sequence of slices with a specified overlap step, a shape of [shape not specified] is obtained. The three-dimensional data block; wherein, the dimensional format of the original three-dimensional two-photon microvascular data is as follows: That is, height, width, and total depth; Step S12: Reconstruct the physical dimension: When converting to a network input tensor, the depth dimension is converted using a matrix transpose operation. Move it to the first position to transform the data block into Subsequently, a dimensionality-up operation was used to increase the single-channel dimension. , forming a shape The four-dimensional tensor; Step S13: Perform dimension unpacking on the data for which batch dimensions were automatically added: In the typical data loading pipeline of deep learning frameworks, data is usually automatically batch-dimensioned. Before entering the network, a dimension unpacking operation is performed to forcibly remove the batch dimensions allocated by the deep learning framework. This allows the tensor input to the network to be restored to its original state. .

4. The method as described in claim 3, characterized in that, Step S2 includes: Using a fully convolutional densely connected network as the backbone network, each layer in the downsampling path of the backbone network performs depth extraction of local fine features in the XY two-dimensional plane through dense blocks, and performs downsampling through transition layers, so that the microvascular cross-sectional features of each local slice are efficiently captured.

5. The method as described in claim 4, characterized in that, Step S2 includes: Step S21: Initial feature mapping: Sequence tensor First, the initial two-dimensional convolutional layer is used to perform preliminary spatial feature extraction, generating a shallow feature tensor. Step S22: High-frequency feature reuse in dense blocks: The feature tensor passes through multiple downsampling stages in sequence; in each stage, the core mechanism is a dense block; inside the dense block, there are multiple convolutional units connected in sequence; feature concatenation: the input of each convolutional unit in the dense block is the concatenation of the output feature maps of all the preceding layers in the channel dimension. Step S23: Transitional downsampling and feature transfer: After each dense block, a transition downsampling layer is connected to reduce the spatial resolution of the feature map, expand the planar receptive field of the neuron, and thus aggregate a larger range of two-dimensional spatial context. Step S24: Direct the deep two-dimensional feature maps extracted from each dense block to the CSAM module.

6. The method as described in claim 5, characterized in that, The convolutional unit is a cascaded structure of an instance normalization layer, an activation layer, a two-dimensional convolutional layer, and a feature discarding layer.

7. The method as described in claim 6, characterized in that, Step S3 includes: The cross-slice multidimensional attention module sequentially reweights the feature map of the current dimension through three sequential semantic / channel attention modules, position / spatial attention modules, and slice attention modules.

8. The method as described in claim 7, characterized in that, Step S3 includes: The semantic / channel attention module is used to compress slices and spatial dimensions through global max pooling and average pooling, and to learn the importance between channels using a multilayer perceptron, thereby focusing on the core semantic features of microvessels at the global level. The location / spatial attention module is used to compress features along the channel and slice dimensions, and to generate a spatial attention map using a two-dimensional convolutional layer to locate the high-frequency response region of microvessels in the XY plane. The slice attention module is used to model the three-dimensional continuity between slices and suppress interference; after feature extraction, MLP is used to predict the mean vector of a low-rank multivariate Gaussian distribution. Diagonal variance vector and low-rank factor matrix The covariance matrix is ​​constructed; during the training phase, a reparameterization technique is used for sampling, and during the inference phase, the mean is directly output; finally, dynamic slice weights are generated through activation functions to suppress slice features containing highly reflective meninges or strong noise.

9. The method as described in claim 8, characterized in that, Step S4 includes: In the decoder path, deep features are first upsampled through deconvolution to restore spatial resolution. Then, the upsampled features are concatenated with the corresponding layer's skip connection features, which have already been enhanced and filtered by the CSAM module, along the channel dimension. The concatenated features are further fused through dense blocks and upsampled layer by layer, ultimately resulting in a final output. The convolutional layer outputs the classification probability of each pixel, thereby generating a complete microvascular segmentation mask with three-dimensional connectivity.

10. The method as described in claim 9, characterized in that, Step S4 includes: Step S41: Transitional upsampling to restore spatial resolution: For the deepest feature tensor at the bottom of the U-shaped network, or the feature tensor output by the previous decoding layer, a transition upsampling layer is used to process it; this layer mainly includes transposed convolution operation; data flow: while the spatial resolution of the feature map is multiplied by a set stride, the number of feature channels is compressed and the dimension is reduced, providing a basis for physical size alignment for subsequent feature fusion. Step S42: Introducing cross-layer stitching of CSAM high-fidelity features: Align the deep feature tensor obtained after upsampling in step S41 with the feature tensor from the same layer of the network, which comes from the skip connection; Data flow: Strictly concatenate the above two sets of feature tensors in the channel dimension to form a composite feature tensor containing multi-scale semantics and high-fidelity local structural information. Step S43: Nonlinear deep fusion based on decoder dense blocks: Simple concatenation of channel dimensions cannot directly guide pixel classification. The concatenated composite feature tensor is then input into the dense block of the current level of the decoder, thereby accurately reconstructing the true boundary of microvessels while reducing noise. Step S44: Layer-by-layer spatial reconstruction and foreground probability output: The decoding process is executed iteratively until the spatial resolution of the feature tensor is completely restored to the size of the original input data block; final mapping: the fused highest-resolution feature tensor is mapped through a... The output convolutional layer; the final output tensor represents the predicted probability that each pixel in the spatial sequence belongs to the microvascular foreground. Based on this, a high-precision microvascular mask with complete three-dimensional topological connectivity is generated by binarization, thus completing the entire segmentation process.

Images