A medical image segmentation method and system

Abstract

The present invention discloses a medical image segmentation method and system, the method comprising: designing a dynamic strip convolution block capable of learning an extension direction offset and a vertical direction offset of an input feature map; constructing a multi-head convolution block based on the designed dynamic strip convolution block, a deformable convolution block and a first standard convolution block, and using a local-macro attention mechanism to adaptively fuse local-macro features and features from the multi-head convolution block to obtain fused features with various morphological structures; using the multi-head convolution block and the local-macro attention mechanism to construct a U-shaped segmentation network, inputting a medical image into the U-shaped segmentation network, and outputting a medical image segmentation result; the present invention has the advantages of high sampling accuracy, simple strategy and strong robustness.

Description

Technical Field

[0001] The present invention relates to the field of deep learning, and in particular to a medical image segmentation method and system. Background Art

[0002] Medical image segmentation is a key technology in the field of computer vision and medical image processing, which aims to segment medical images into multiple regions or parts with clinical significance. Medical image segmentation is crucial in computer-aided diagnosis and lesion localization. Existing methods usually use the same network architecture to segment anatomical structures with different morphologies, ignoring the problem of morphological heterogeneity, resulting in unsatisfactory segmentation results (especially in medical image modalities such as chest X-rays where heterogeneity is very strong). Although some shape-aware operators such as deformable convolution and snake convolution have been introduced, they still have limitations in processing strip structures with changing directions (ribs, clavicles). Therefore, there is an urgent need for a segmentation method that can dynamically adapt to different anatomical morphologies to improve the accuracy of medical image segmentation. Existing morphologically aware operators are mainly divided into two categories: isotropic deformable convolution operators and anisotropic snake convolution operators.

[0003] The first type of method uses an additional set of learnable parameters to record the offset of the sampling range, thereby achieving a deformable convolution range. For example, in the prior art 1, Jifeng Dai et al., Deformable convolution networks. Proceedings of the IEEE international conference on computer vision. 2017: 764-773. Deformable convolution introduces a learnable offset so that the sampling position of the convolution kernel can be dynamically adjusted, thereby more flexibly adapting to geometric deformations and complex shapes in the image. Compared with standard convolution, it more effectively captures the deformation details of the target object and significantly improves the performance in tasks such as target detection and semantic segmentation. This technology performs well in enhancing feature extraction capabilities, and is particularly suitable for processing visual data with irregular structures.

[0004] The second type of method adds some constraints for specific structures on the basis of deformable convolution. For example, the prior art 2 Yaolei Qi et al., Dynamic snake convolution based on topological geometric constraints for tubular structure segmentation. "Proceedings of the IEEE international conference on computer vision". 2023: 6070-6079. Unlike deformable convolutions that freely learn geometric changes, snake convolutions impose additional continuity constraints to prevent the perception area from deviating from the segmentation target area, especially for thin tubular structures. Snake convolution enhances the perception of thin and curved tubular structures.

[0005] Although the first type of method mentioned above can learn the geometric changes of the segmentation target, it is easy to fail to sample the slender structure area for anisotropic structures such as ribs and clavicles, thereby losing geometric features and having low sampling accuracy. The second type of method mentioned above is specially designed to solve the segmentation of slender structures. The convolution range of the serpentine convolution operator increases continuously in the vertical or horizontal direction. However, for strip structures with different directions, a variety of complex feature fusion strategies are required to make up for the deficiency of the single extension direction (vertical, horizontal) of the convolution range, resulting in complex strategies and poor robustness. Summary of the invention

[0006] The technical problem to be solved by the present invention is that the existing medical image segmentation method has low sampling accuracy, complex strategy and poor robustness.

[0007] The present invention solves the above technical problems through the following technical means: a medical image segmentation method, comprising the following steps:

[0008] Step 1: Design a dynamic strip convolution block that can learn the extension direction offset and vertical direction offset of the input feature map;

[0009] Step 2: construct a multi-head convolution block based on the designed dynamic strip convolution block, deformable convolution block and the first standard convolution block, and use the local-macro attention mechanism to adaptively fuse the local-macro features and the features from the multi-head convolution block to obtain fused features with various morphological structures;

[0010] Step 3: Use multi-head convolution blocks and local-macro attention mechanism to build a U-shaped segmentation network, input the medical image into the U-shaped segmentation network, and output the medical image segmentation result.

[0011] Furthermore, the design process of the dynamic strip convolution block is:

[0012] The input feature map is group normalized after passing through the large kernel convolution block, and then outputs the extension direction offset through the first activation function. The input feature map is group normalized after passing through the second standard convolution block, and then outputs the vertical direction offset through the second activation function. The extension direction offset and the vertical direction offset are used to perform feature interpolation on the input feature map to obtain the deformed feature map.

[0013] Furthermore, the first activation function is an Arctan activation function, and the second activation function is a Tanh activation function.

[0014] Furthermore, the process of performing a feature interpolation operation on the input feature map using the extension direction offset and the vertical direction offset to obtain the deformed feature map is:

[0015] Flatten the convolution kernel of the second standard convolution block, and calculate the coordinates of each sampling point in the original horizontal coordinate system after the offset transformation according to the axis formula, expressed as

[0016] C i+d =(x i+d ,y i+d )=(dcosθ-δ i+d sinθ,dsinθ+δ i+d cosθ)

[0017] Among them, d = {0, 1, 2, 3, 4} is the distance from each sampling point to the convolution center in the flattened state, C i+d is the coordinate of the i+dth sampling point, x i+d is the horizontal coordinate of the i+dth sampling point, y i+d is the coordinate of the i+dth sampling point, θ is the offset in the extension direction, δ i+d is the vertical offset of the i+dth sampling point position; the coordinates of each sampling point in the original horizontal coordinate system constitute the offset sampling range, and the learned sampling range is bilinearly interpolated, which is expressed by the formula C=∑ C″ bilinear_interpolation(C″,C)·C″, where bilinear_interpolation represents a bilinear interpolation operation, C is the offset sampling range, and C″ is an integer position. A convolution operation is performed on the bilinearly interpolated sampling range to obtain a deformed feature map.

[0018] Furthermore, the step 2 includes:

[0019] A certain part of the input feature map is magnified by a preset multiple with the convolution kernel as the center, and the local geometric feature is obtained as the local-macro feature. The input feature map is input into the multi-head convolution block, the output results of the multi-head convolution block are spliced, and then the local-macro attention mechanism is used to adaptively fuse the local-macro features and the features from the multi-head convolution block.

[0020] Furthermore, the local-macro attention mechanism is used to adaptively fuse local-macro features and features from multi-head convolution blocks, including:

[0021] The local-macro features are used as the query vector Q in the attention calculation, and the concatenated features of the output results of the multi-head convolution block are used as the key vector K and value vector V in the attention calculation. After the attention is weighted to the concatenated features, the features from the multi-head convolution blocks are adaptively fused to obtain fused features with various morphological structures.

[0022] Furthermore, the formula for calculating the attention is:

[0023]

[0024] Among them, Attention represents attention calculation, d k Represents the dimension of the key vector K.

[0025] Furthermore, the U-shaped segmentation network is constructed by using multi-head convolution blocks and local-macro attention mechanism, including:

[0026] The input feature map passes through a multi-head convolution block, and the local-macro attention mechanism is used to adaptively fuse local-macro features and features from the multi-head convolution block. This is a feature extraction operation, and then downsampling is performed, and then another feature extraction operation and downsampling operation are performed. This operation is repeated multiple times to complete the downsampling process. Then, the feature map output by the downsampling process is subjected to a feature extraction operation and then upsampled. After upsampling, another feature extraction operation and upsampling operation are performed again. This operation is repeated multiple times to complete the upsampling process. Finally, the output feature map is passed through the segmentation head to output the medical image segmentation result.

[0027] The present invention also provides a medical image segmentation system, comprising:

[0028] The dynamic strip convolution part is used to design a dynamic strip convolution block that can learn the extension direction offset and vertical direction offset of the input feature map;

[0029] The adaptive morphological perception part is used to construct a multi-head convolution block based on the designed dynamic strip convolution block, deformable convolution block and the first standard convolution block, and adopt the local-macro attention mechanism to adaptively fuse the local-macro features and the features from the multi-head convolution block to obtain fused features with various morphological structures;

[0030] The segmentation network building part is used to build a U-shaped segmentation network using multi-head convolution blocks and a local-macro attention mechanism, input the medical image into the U-shaped segmentation network, and output the medical image segmentation result.

[0031] Furthermore, the design process of the dynamic strip convolution block is:

[0032] The input feature map is group normalized after passing through the large kernel convolution block, and then outputs the extension direction offset through the first activation function. The input feature map is group normalized after passing through the second standard convolution block, and then outputs the vertical direction offset through the second activation function. The extension direction offset and the vertical direction offset are used to perform feature interpolation on the input feature map to obtain the deformed feature map.

[0033] Furthermore, the first activation function is an Arctan activation function, and the second activation function is a Tanh activation function.

[0034] Furthermore, the process of performing a feature interpolation operation on the input feature map using the extension direction offset and the vertical direction offset to obtain the deformed feature map is:

[0035] Flatten the convolution kernel of the second standard convolution block, and calculate the coordinates of each sampling point in the original horizontal coordinate system after the offset transformation according to the axis formula, expressed as

[0036] C i+d =(x i+d ,y i+d )=(dcosθ-δ i+d sinθ,dsinθ+δ i+d cosθ)

[0037] Among them, d = {0, 1, 2, 3, 4} is the distance from each sampling point to the convolution center in the flattened state, C i+d is the coordinate of the i+dth sampling point, x i+d is the horizontal coordinate of the i+dth sampling point, y i+d is the coordinate of the i+dth sampling point, θ is the offset in the extension direction, δ i+d is the vertical offset of the i+dth sampling point position; the coordinates of each sampling point in the original horizontal coordinate system constitute the offset sampling range, and the learned sampling range is bilinearly interpolated, which is expressed by the formula C=∑C″ bilinear_interpolation(C″,C)·C″, where bilinear_interpolation represents a bilinear interpolation operation, C is the offset sampling range, and C″ is an integer position. A convolution operation is performed on the bilinearly interpolated sampling range to obtain a deformed feature map.

[0038] Furthermore, the adaptive morphology sensing part is also used for:

[0039] A certain part of the input feature map is magnified by a preset multiple with the convolution kernel as the center, and the local geometric feature is obtained as the local-macro feature. The input feature map is input into the multi-head convolution block, the output results of the multi-head convolution block are spliced, and then the local-macro attention mechanism is used to adaptively fuse the local-macro features and the features from the multi-head convolution block.

[0040] Furthermore, the local-macro attention mechanism is used to adaptively fuse local-macro features and features from multi-head convolution blocks, including:

[0041] The local-macro features are used as the query vector Q in the attention calculation, and the concatenated features of the output results of the multi-head convolution block are used as the key vector K and value vector V in the attention calculation. After the attention is weighted to the concatenated features, the features from the multi-head convolution blocks are adaptively fused to obtain fused features with various morphological structures.

[0042] Furthermore, the formula for calculating the attention is:

[0043]

[0044] Among them, Attention represents attention calculation, d k Represents the dimension of the key vector K.

[0045] Furthermore, the U-shaped segmentation network is constructed by using multi-head convolution blocks and local-macro attention mechanism, including:

[0046] The input feature map passes through a multi-head convolution block, and the local-macro attention mechanism is used to adaptively fuse local-macro features and features from the multi-head convolution block. This is a feature extraction operation, and then downsampling is performed, and then another feature extraction operation and downsampling operation are performed. This operation is repeated multiple times to complete the downsampling process. Then, the feature map output by the downsampling process is subjected to a feature extraction operation and then upsampled. After upsampling, another feature extraction operation and upsampling operation are performed again. This operation is repeated multiple times to complete the upsampling process. Finally, the output feature map is passed through the segmentation head to output the medical image segmentation result.

[0047] The advantages of the present invention are:

[0048] (1) The present invention designs a dynamic strip convolution block that can learn the extension direction offset and the vertical direction offset of the input feature map. The sampling range of each convolution center is adaptively constrained by these two learnable parameters, so as to better obtain the morphological features of the thin strip segmentation target with continuously changing extension direction and aspect ratio, avoid the loss of geometric features, and achieve accurate segmentation of challenging strip anatomical structures with high sampling accuracy. In addition, a variety of types of convolution kernels are used, such as deformable convolution, dynamic strip convolution, standard convolution, etc. Based on the cross-attention technology, a local-macro attention is designed to adaptively fuse features from different convolution kernels, thereby obtaining important features of various morphological structures. The strategy is simple and significantly improves the diversity and robustness of feature representation.

[0049] (2) The present invention uses the proposed dynamic strip convolution block and the adaptive morphological perception strategy of step 2 to replace the standard convolution operator and feature splicing operation in the standard U-Net, thereby obtaining a medical image segmentation network framework with morphological perception capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of the overall design process of a medical image segmentation method disclosed in an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of the design process of a dynamic strip convolution block in a medical image segmentation method disclosed in an embodiment of the present invention;

[0052] Figure 3 A schematic diagram of an adaptive morphological perception strategy in a medical image segmentation method disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Example 1

[0055] like Figure 1As shown, Embodiment 1 of the present invention provides a medical image segmentation method, which solves the problem that the existing medical segmentation method ignores the large morphological heterogeneity of the segmentation target by designing a novel deformable convolution operator for complex strip structures and an adaptive feature fusion strategy, and achieves high-precision segmentation effect on a variety of medical segmentation data sets. The present invention is implemented using the PyTorch framework, applies deep learning technology to medical image segmentation, and proposes a morphology-aware medical image segmentation network (MaP), which pursues adaptive morphological perception, so that the segmentation model can perform high-precision segmentation on various target structures with large morphological differences, such as Figure 1 As shown. Specifically, this application first designs a dynamic strip convolution operator based on deformable convolution, and introduces two types of learnable offsets to make the convolution sampling range more suitable for capturing the morphological features of complex and changeable strip structures (such as ribs and clavicles). Then, multiple types of convolution kernels are combined to form a multi-head convolution block, and then the features of the multi-head convolution are fused through the proposed adaptive morphological perception fusion strategy. Finally, the segmentation network is built using the basic U-shaped structure, which specifically includes the following steps:

[0056] S1. Design a dynamic strip convolution block that can learn the extension direction offset and vertical direction offset of the input feature map; the specific process is:

[0057] like Figure 2 As shown, the input feature map is subjected to a group normalization operation after passing through a large kernel convolution block, and then the extension direction offset is output through a first activation function. The input feature map is subjected to a group normalization operation after passing through a second standard convolution block, and then the vertical direction offset is output through a second activation function. The extension direction offset and the vertical direction offset are used to perform a feature interpolation operation on the input feature map to obtain a deformed feature map. In this embodiment, the first activation function is an Arctan activation function, and the second activation function is a Tanh activation function. The process of performing a feature interpolation operation on the input feature map using the extension direction offset and the vertical direction offset to obtain a deformed feature map is as follows:

[0058] For a 3×3 standard convolution, the convolution range is a square. In order to make the convolution sampling range better capture the complex strip structure, inspired by deformable convolution, two additional groups of convolutions (one group of large kernel convolutions and one group of standard convolutions) are used to learn two types of offsets θ∈[-π / 2,π / 2] and Δ∈[-1,1] respectively. Among them, θ represents the approximate extension direction of the overall strip structure, and Δ represents the vertical offset of each sampling point in the extension direction. In implementation, the standard convolution kernel is first flattened, and the coordinates of each sampling point after the offset transformation in the original horizontal coordinate system are calculated according to the rotation axis formula. Taking the positive sampling point as an example, its coordinates can be expressed as

[0059] Ci+d =(x i+d ,y i+d )=(dcosθ-δ i+d sinθ,dsinθ+δ i+d cosθ)

[0060] Among them, d = {0, 1, 2, 3, 4} is the distance from each sampling point to the convolution center in the flattened state, C i+d is the coordinate of the i+dth sampling point, x i+d is the horizontal coordinate of the i+dth sampling point, y i+d is the ordinate of the i+dth sampling point, θ is the offset in the extension direction, δ i+d is the vertical offset of the i+dth sampling point; the coordinates of each sampling point in the original horizontal coordinate system constitute the sampling range, and the learned sampling range is bilinearly interpolated, which is expressed by the formula C=∑ C′′ bilinear_interpolation(C″,C)·C″, where bilinear_interpolation represents a bilinear interpolation operation, C is the offset sampling range, and C″ is an integer position. A convolution operation is performed on the bilinearly interpolated sampling range to obtain a deformed feature map.

[0061] S2. Based on the designed dynamic strip convolution block, deformable convolution block and the first standard convolution block, a multi-head convolution block is constructed, and the local-macro attention mechanism is used to adaptively fuse the local-macro features and the features from the multi-head convolution block to obtain fused features with various morphological structures; the specific process is:

[0062] like Figure 3 As shown in Figure 2, for the same feature map, three different types of convolutions are used for feature extraction: standard convolution, deformable convolution, and dynamic strip convolution. The feature maps obtained are f 1 =standard_conv(x),f 2 = deformable_conv(x),f 3 = dynamic_strip_conv(x). Multiple convolution kernels in parallel help capture important morphological features of heterogeneous targets and improve segmentation accuracy. Wherein, x represents the input feature map, standard_conv, deformable_conv, and dynamic_strip_conv represent standard convolution, deformable convolution, and dynamic strip convolution, respectively. Standard convolution and deformable convolution are prior arts, and dynamic strip convolution is designed by the present invention. The working process thereof is specifically referred to step S1.

[0063] Continue reading Figure 3, a certain part of the input feature map is magnified by a preset multiple with the convolution kernel as the center, and the local geometric features are obtained as local-macro features. At the same time, the input feature map is input into the multi-head convolution block, and the output results of the multi-head convolution block are spliced. Then the local-macro features are used as the query vector Q in the attention calculation, and the spliced ​​features of the output results of the multi-head convolution block are used as the key vector K and value vector V in the attention calculation. After the attention is weighted to the spliced ​​features, the features from the multi-head convolution block are adaptively fused to obtain fused features with various morphological structures. The formula for the attention calculation is:

[0064]

[0065] Among them, Attention represents attention calculation, d k Represents the dimension of the key vector K.

[0066] The above process can be expressed as the following process using the formula: After each multi-head convolution block, the adaptive morphological perception strategy is applied to fuse the features, and the fused features f MA =LmA(linear(flatten(expand(f′,λ))),linear(flatten(concat(f 1 ,f 2 ,f 3 )))). Wherein, LmA is the local-macro attention mechanism proposed in the present invention, and the expand function represents the magnification of a certain part of the feature map before multi-head convolution with the convolution kernel as the center, so as to obtain more significant geometric features of this part, where f′ is the local-macro feature and λ is the magnification factor. At the same time, the concat function represents the splicing of the feature maps after multi-head convolution. Based on the operation rules of cross attention, the spliced ​​features are weighted according to the significance of the morphological features, so as to obtain adaptive morphological perception, and Q in the attention calculation is defined as linear(flatten(expanfd(f′,λ))), K and V are defined as linear(flatten(concat(f 1 ,f 2 ,f 2 ), flatten represents the feature flattening operation, and linear represents the linear layer, so that a certain part of the feature map before the multi-head convolution is enlarged with the convolution kernel as the center, and then the feature is flattened and passed through a linear layer as the query vector Q. Similarly, the concatenated features of the output results of the multi-head convolution block are flattened and passed through another linear layer as the key vector K and value vector V. Finally, the attention is calculated as The morphologically aware features can be obtained by weighting the calculated attention to the concatenated features.

[0067] S3, using multi-head convolution blocks and local-macro attention mechanism to build a U-shaped segmentation network, input the medical image into the U-shaped segmentation network, and output the medical image segmentation result. The specific process is:

[0068] The present invention builds the U-shaped segmentation network of the present invention based on the classic UNet segmentation network structure, wherein the standard convolution block of the UNet segmentation network structure is replaced by a multi-head convolution block, and the feature splicing operation of the UNet segmentation network structure is replaced by a local-macro attention mechanism. The model consists of an encoder part and a decoder part. The encoder part contains three convolution layers for feature extraction, and the decoder part contains three convolution layers for recovering segmentation details. The corresponding layers on both sides are skipped. The final segmentation head performs prediction. For details, see Figure 1 The input feature map passes through the multi-head convolution block, and the local-macro attention mechanism is used to adaptively fuse the local-macro features and the features from the multi-head convolution block. This is a feature extraction operation, and then downsampling is performed, and then another feature extraction operation and downsampling operation are performed. This operation is repeated multiple times to complete the downsampling process, and then the feature map output by the downsampling process is subjected to a feature extraction operation and then upsampled. After upsampling, another feature extraction operation and upsampling operation are performed again. This operation is repeated multiple times to complete the upsampling process, and finally the output feature map is passed through the segmentation head to output the medical image segmentation result.

[0069] After the U-shaped segmentation network is constructed, the network needs to be trained and tested in practical applications. In this embodiment, the classic binary cross entropy loss is used for optimization. During the test, a medical image is given, and the model automatically segments a binary image of the target structure.

[0070] Through the above technical solutions, the present invention proposes a segmentation framework based on dynamic strip convolution and adaptive morphological perception. First, a dynamic strip convolution operator is designed. By learning additional offset parameters, the convolution sampling range is adaptively changed, and the slender structure with gradually changing extension direction and aspect ratio is adaptively focused on, so as to capture the morphological characteristics of complex strip structures, and achieve accurate segmentation of challenging strip anatomical structures (such as 24 ribs and 2 clavicles), which greatly improves the segmentation effect of thin strip structures. In addition, an adaptive morphological perception strategy is proposed. By using appropriate convolution kernels (standard convolution, deformable convolution, dynamic strip convolution, etc.) to match different local morphological structures, and adaptively fusing these diversified features, the important features of target structures with different morphologies are obtained, which significantly improves the diversity and robustness of feature representation. Finally, the proposed dynamic strip convolution operator and adaptive morphological perception strategy are used to replace the standard convolution operator and feature splicing operation in the standard U-Net, and a medical image segmentation network framework with morphological perception can be obtained.

[0071] Example 2

[0072] Based on embodiment 1, embodiment 2 of the present invention further provides a medical image segmentation system, including:

[0073] The dynamic strip convolution part is used to design a dynamic strip convolution block that can learn the extension direction offset and vertical direction offset of the input feature map;

[0074] The adaptive morphological perception part is used to construct a multi-head convolution block based on the designed dynamic strip convolution block, deformable convolution block and the first standard convolution block, and adopt the local-macro attention mechanism to adaptively fuse the local-macro features and the features from the multi-head convolution block to obtain fused features with various morphological structures;

[0075] The segmentation network building part is used to build a U-shaped segmentation network using multi-head convolution blocks and a local-macro attention mechanism, input the medical image into the U-shaped segmentation network, and output the medical image segmentation result.

[0076] Specifically, the design process of the dynamic strip convolution block is:

[0077] The input feature map is group normalized after passing through the large kernel convolution block, and then outputs the extension direction offset through the first activation function. The input feature map is group normalized after passing through the second standard convolution block, and then outputs the vertical direction offset through the second activation function. The extension direction offset and the vertical direction offset are used to perform feature interpolation on the input feature map to obtain the deformed feature map.

[0078] More specifically, the first activation function is the Arctan activation function, and the second activation function is the Tanh activation function.

[0079] More specifically, the process of performing a feature interpolation operation on the input feature map using the extension direction offset and the vertical direction offset to obtain the deformed feature map is as follows:

[0080] Flatten the convolution kernel of the second standard convolution block, and calculate the coordinates of each sampling point in the original horizontal coordinate system after the offset transformation according to the axis formula, expressed as

[0081] C i+d =(x i+d ,y i+d )=(dcosθ-δ i+d sinθ,dsinθ+δ i+d cosθ)

[0082] Among them, d = {0, 1, 2, 3, 4} is the distance from each sampling point to the convolution center in the flattened state, C i+d is the coordinate of the i+dth sampling point, x i+d is the horizontal coordinate of the i+dth sampling point, y i+d is the coordinate of the i+dth sampling point, θ is the offset in the extension direction, δ i+d is the vertical offset of the i+dth sampling point position; the coordinates of each sampling point in the original horizontal coordinate system constitute the offset sampling range, and the learned sampling range is bilinearly interpolated, which is expressed by the formula C=∑ c″ bilinear_interpolation(C″,C)·C′′, where bilinear_interpolation represents a bilinear interpolation operation, C is the offset sampling range, and C″ is an integer position. A convolution operation is performed on the bilinearly interpolated sampling range to obtain a deformed feature map.

[0083] Specifically, the adaptive morphology sensing part is also used for:

[0084] A certain part of the input feature map is magnified by a preset multiple with the convolution kernel as the center, and the local geometric feature is obtained as the local-macro feature. The input feature map is input into the multi-head convolution block, the output results of the multi-head convolution block are spliced, and then the local-macro attention mechanism is used to adaptively fuse the local-macro features and the features from the multi-head convolution block.

[0085] More specifically, the local-macro attention mechanism is used to adaptively fuse local-macro features and features from multi-head convolutional blocks, including:

[0086] The local-macro features are used as the query vector Q in the attention calculation, and the concatenated features of the output results of the multi-head convolution block are used as the key vector K and value vector V in the attention calculation. After the attention is weighted to the concatenated features, the features from the multi-head convolution blocks are adaptively fused to obtain fused features with various morphological structures.

[0087] More specifically, the attention calculation formula is:

[0088]

[0089] Among them, Attrntion represents the attention calculation, d k Represents the dimension of the key vector K.

[0090] Specifically, the U-shaped segmentation network is constructed by using multi-head convolution blocks and local-macro attention mechanism, including:

[0091] The input feature map passes through a multi-head convolution block, and the local-macro attention mechanism is used to adaptively fuse local-macro features and features from the multi-head convolution block. This is a feature extraction operation, and then downsampling is performed, and then another feature extraction operation and downsampling operation are performed. This operation is repeated multiple times to complete the downsampling process. Then, the feature map output by the downsampling process is subjected to a feature extraction operation and then upsampled. After upsampling, another feature extraction operation and upsampling operation are performed again. This operation is repeated multiple times to complete the upsampling process. Finally, the output feature map is passed through the segmentation head to output the medical image segmentation result.

[0092] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A medical image segmentation method, characterized in that: The following steps are involved: Step 1, design a dynamic strip convolution block that can learn the extension direction offset and vertical direction offset of the input feature map; the design process of the dynamic strip convolution block is: the input feature map is subjected to group normalization operation after passing through the large kernel convolution block, and then the extension direction offset is output through the first activation function, the input feature map is subjected to group normalization operation after passing through the second standard convolution block, and then the vertical direction offset is output through the second activation function, and the extension direction offset and the vertical direction offset are used to perform feature interpolation operation on the input feature map to obtain the deformed feature map: Flatten the convolution kernel of the second standard convolution block, and calculate the coordinates of each sampling point in the original horizontal coordinate system after the offset transformation according to the axis formula, expressed as in, is the distance from each sampling point to the convolution center in the flattened state, is the coordinate of the i+dth sampling point, is the horizontal coordinate of the i+dth sampling point position, is the ordinate of the i+dth sampling point, is the extension direction offset, is the vertical offset of the position of the i+dth sampling point; the coordinates of each sampling point in the original horizontal coordinate system form the offset sampling range, bilinear interpolation is performed on the learned sampling range, and a convolution operation is performed on the bilinearly interpolated sampling range to obtain a deformed feature map; Step 2: construct a multi-head convolution block based on the designed dynamic strip convolution block, deformable convolution block and the first standard convolution block, and use the local-macro attention mechanism to adaptively fuse the local-macro features and the features from the multi-head convolution block to obtain fused features with various morphological structures; Step 3: Use multi-head convolution blocks and local-macro attention mechanism to build a U-shaped segmentation network, input the medical image into the U-shaped segmentation network, and output the medical image segmentation result.

2. A medical image segmentation method according to claim 1, characterized in that: The first activation function is the Arctan activation function, and the second activation function is the Tanh activation function.

3. A medical image segmentation method according to claim 1, characterized in that: The second step comprises: A certain part of the input feature map is magnified by a preset multiple with the convolution kernel as the center, and the local geometric feature is obtained as the local-macro feature. The input feature map is input into the multi-head convolution block, the output results of the multi-head convolution block are spliced, and then the local-macro attention mechanism is used to adaptively fuse the local-macro features and the features from the multi-head convolution block.

4. A medical image segmentation method according to claim 3, characterized in that: The local-macro attention mechanism is used to adaptively fuse local-macro features and features from multi-head convolution blocks, including: The local-macro features are used as the query vector Q in the attention calculation, and the concatenated features of the output results of the multi-head convolution block are used as the key vector K and value vector V in the attention calculation. After the attention is weighted to the concatenated features, the features from the multi-head convolution blocks are adaptively fused to obtain fused features with various morphological structures.

5. A medical image segmentation method according to claim 4, characterized in that: The formula for attention calculation is: in, represents the attention calculation, Represents the dimension of the key vector K.

6. A medical image segmentation method according to claim 1, characterized in that: The U-shaped segmentation network is constructed by using multi-head convolution blocks and local-macro attention mechanism, including: The input feature map passes through the multi-head convolution block, and the local-macro attention mechanism is used to adaptively fuse the local-macro features and the features from the multi-head convolution block. This is a feature extraction operation, and then downsampling is performed, and then another feature extraction operation and downsampling operation are performed. The above operations are performed multiple times to complete the downsampling process, and then the feature map output by the downsampling process is subjected to a feature extraction operation and then upsampled. After upsampling, another feature extraction operation and upsampling operation are performed again. The above operations are performed multiple times to complete the upsampling process, and finally the output feature map is passed through the segmentation head to output the medical image segmentation result.

7. A medical image segmentation system, characterized in that: include: The dynamic strip convolution part is used to design a dynamic strip convolution block that can learn the extension direction offset and vertical direction offset of the input feature map; the design process of the dynamic strip convolution block is as follows: the input feature map is subjected to a group normalization operation after passing through a large kernel convolution block, and then the extension direction offset is output through a first activation function, the input feature map is subjected to a group normalization operation after passing through a second standard convolution block, and then the vertical direction offset is output through a second activation function, and the input feature map is subjected to a feature interpolation operation using the extension direction offset and the vertical direction offset to obtain a deformed feature map: Flatten the convolution kernel of the second standard convolution block, and calculate the coordinates of each sampling point in the original horizontal coordinate system after the offset transformation according to the axis formula, expressed as in, is the distance from each sampling point to the convolution center in the flattened state, is the coordinate of the i+dth sampling point, is the horizontal coordinate of the i+dth sampling point position, is the ordinate of the i+dth sampling point, is the extension direction offset, is the vertical offset of the position of the i+dth sampling point; the coordinates of each sampling point in the original horizontal coordinate system form the offset sampling range, bilinear interpolation is performed on the learned sampling range, and a convolution operation is performed on the bilinearly interpolated sampling range to obtain a deformed feature map; The adaptive morphological perception part is used to construct a multi-head convolution block based on the designed dynamic strip convolution block, deformable convolution block and the first standard convolution block, and adopt the local-macro attention mechanism to adaptively fuse the local-macro features and the features from the multi-head convolution block to obtain fused features with various morphological structures; The segmentation network building part is used to build a U-shaped segmentation network using multi-head convolution blocks and a local-macro attention mechanism, input the medical image into the U-shaped segmentation network, and output the medical image segmentation result.

Images