Non-voiding and geometry-aware attention based multi-modal 3d mri segmentation method, system and medium

Abstract

The application discloses a kind of multi-modal three-dimensional MRI segmentation methods, systems and media based on non-empty voxelization and geometric perception attention, the method includes the following steps: in input stage, by non-empty voxelization to multi-modal MRI body data is screened, and sparse voxel is obtained;In the coding stage, based on three-way dynamic non-empty voxel and geometric perception cross-attention downsampling, the feature of sparse voxel is extracted;In the decoding stage, based on geometric perception cross-attention upsampling, the sparse voxel feature obtained by extraction is realized cross-scale alignment and fusion gradually;In the output stage, the segmentation result is output by convolution. The application realizes while guaranteeing segmentation accuracy, reduces the consumption of computing resources, enhances geometric perception ability and improves the generality of model.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and artificial intelligence, and in particular to a multimodal three-dimensional MRI segmentation method, system and medium based on non-empty voxelization and geometric perception attention. Background Technology

[0002] In clinical medical image processing, three-dimensional medical image segmentation technology has been widely applied in disease diagnosis, preoperative planning, radiotherapy, and disease monitoring. Multimodal magnetic resonance imaging (MRI) has become an important foundation for current medical image analysis because it can provide complementary tissue feature information under different imaging sequences. To achieve automatic segmentation of anatomical structures in MRI images, researchers generally use deep learning methods to build end-to-end segmentation networks.

[0003] Currently, most mainstream 3D medical image segmentation methods are based on convolutional neural network (CNN) structures, especially encoder-decoder networks represented by U-Net. U-Net achieves high segmentation accuracy under limited sample conditions through its symmetrical structure of downsampling and upsampling paths and its cross-layer skip connection mechanism. Subsequent variants such as 3D U-Net, Attention U-Net, and nnU-Net have further improved feature extraction and cross-layer information fusion methods, and have been widely used in multimodal MRI segmentation tasks.

[0004] In recent years, with the introduction of the Transformer architecture, medical image segmentation methods have gradually developed hybrid models combining convolution and self-attention mechanisms. For example, methods such as UNETR, SwinUNETR, TransBTS, and nnFormer utilize self-attention mechanisms for global dependency modeling, enabling them to outperform traditional CNN models in some tasks. Furthermore, some research has attempted to improve 3D segmentation efficiency through lightweight or sparse computational methods, such as CoTr, MedNeXt, and 3D UX-Net. These methods have improved the computational cost and generalization performance of the models to varying degrees.

[0005] Overall, existing technologies have formed a deep learning segmentation system based on the U-Net series and combined with improvements to the Transformer architecture, which can achieve high segmentation accuracy in multimodal MRI tasks and provides an important foundation for subsequent research.

[0006] Despite significant progress in accuracy achieved by existing deep learning-based multimodal 3D MRI segmentation methods, several shortcomings and limitations remain. First, traditional convolutional neural networks primarily rely on local convolution operations in 3D volumetric data, resulting in a limited receptive field and difficulty in fully capturing global spatial dependencies, leading to insufficient segmentation accuracy for complex structures or cross-regional anatomical tissues. While the introduction of the Transformer architecture has improved global modeling capabilities to some extent through its self-attention mechanism, 3D Transformers exhibit extremely high computational complexity and consume enormous amounts of memory and computing power when processing voxel-level data, hindering their implementation in resource-constrained clinical settings. Furthermore, Transformer models typically rely on large-scale labeled data to achieve strong generalization capabilities, but medical imaging data is limited by privacy and labeling costs, resulting in a limited number of training samples, further restricting the practical effectiveness of such methods.

[0007] Secondly, existing technologies still have significant shortcomings in modeling the spatial geometric information of medical images. Most deep learning methods only introduce spatial location information through sinusoidal positional encoding or implicit convolutional structures, but these methods cannot fully express the geometric morphology and topological features of anatomical structures. This results in unclear segmentation results in regions with complex boundaries or rich details, making it difficult to accurately depict organ edges and tumor contours. In addition, to reduce computational costs, some studies have adopted sparse computation or lightweight network structures, but this often comes at the cost of sacrificing feature representation capabilities, leading to a significant decrease in segmentation accuracy and failing to achieve an ideal balance between efficiency and accuracy.

[0008] In general, existing multimodal MRI segmentation methods suffer from the following core problems: First, it is difficult to balance global dependency modeling with computational efficiency; second, insufficient utilization of geometric spatial information limits boundary and structure recognition capabilities; and third, excessively large model size and data dependence restrict the feasibility of clinical applications. How to reduce computational resource consumption, enhance geometric perception capabilities, and improve model versatility while ensuring segmentation accuracy has become a critical technical problem that urgently needs to be solved in this field. Summary of the Invention

[0009] The main objective of this invention is to propose a multimodal 3D MRI segmentation method, system, and medium based on non-empty voxelization and geometric perception attention, aiming to reduce computational resource consumption, enhance geometric perception capabilities, and improve the versatility of the model while ensuring segmentation accuracy.

[0010] To achieve the above objectives, this invention provides a multimodal 3D MRI segmentation method based on non-empty voxelization and geometrical attention, the method comprising the following steps:

[0011] Step S10: In the input stage, multimodal MRI volume data are screened by non-empty voxelization to obtain sparse voxels;

[0012] Step S20: In the encoding stage, sparse voxel features are extracted based on three-dimensional dynamic non-empty voxels and geometric perception cross-attention downsampling.

[0013] Step S30: In the decoding stage, the refined sparse voxel features are progressively aligned and fused across scales based on geometric perception cross-attention upsampling;

[0014] Step S40: In the output stage, the segmentation result is output through convolution.

[0015] A further technical solution of the present invention is that step S10 includes:

[0016] Step S101: Embed the multimodal input using 3D convolution to obtain the initial feature map;

[0017] Step S102, based on the feature vector of each voxel Norm and threshold The comparison results generate an occupancy map, filter out the set of non-empty voxel coordinates, and extract the corresponding embedded features from the feature map;

[0018] Step S103: Based on the embedded features, a sparse voxel set with geometric coordinates is formed.

[0019] A further technical solution of the present invention is that step S101 includes:

[0020] Using 3D convolution for multimodal input Embedding:

[0021] (1)

[0022] in, The differences are the height, width, and depth of the data. The number of modes is and the kernel size is . Step size is The output channel is The initial feature map is obtained. .

[0023] A further technical solution of the present invention is that step S102 includes:

[0024] Based on the feature vector of each voxel Norm and threshold The comparison results generate an occupancy map:

[0025] (2)

[0026] in, Represents the occupancy map. The threshold is used to filter out the set of non-empty voxel coordinates:

[0027] (3)

[0028] in, The number of non-empty voxels; and the corresponding embedded features are extracted from the feature map:

[0029] (4).

[0030] A further technical solution of the present invention is that step S103 includes:

[0031] Forming a sparse voxel set with absolute geometric coordinates:

[0032] (5)

[0033] in, Represents the absolute coordinates of a voxel. For the corresponding feature vector, For voxel indexing.

[0034] A further technical solution of the present invention is that step S20 includes:

[0035] Step S201: Adaptively divide the window into subsets based on voxel density, wherein the number of subsets is determined by the number of voxels. With upper limit parameter Decide:

[0036] (6)

[0037] in, For window The number of voxels in the middle The maximum number of voxels in a single subset. For indicator functions;

[0038] Step S202, along Sequence segmentation is performed in three orthogonal directions.

[0039] A further technical solution of the present invention is that step S30 includes:

[0040] Step S301: In the downsampling stage, firstly, a coarse-scale query vector is obtained by max pooling. Then, the high-resolution voxel features and their geometric coordinates are used as key values ​​in the cross-attention process.

[0041] (8)

[0042] in, This indicates the attention of the bulls. For position encoding;

[0043] Step S302: In the upsampling stage, reverse alignment is performed using the encoded residual as the query and the coarse features at the decoding end as the key.

[0044] (9)

[0045] The updated sparse voxel set is finally obtained.

[0046] To achieve the above objectives, the present invention also proposes a multimodal three-dimensional MRI segmentation system based on non-empty voxelization and geometric awareness attention. The system includes a memory, a processor, and a multimodal three-dimensional MRI segmentation program based on non-empty voxelization and geometric awareness attention stored on the processor. The multimodal three-dimensional MRI segmentation program based on non-empty voxelization and geometric awareness attention is executed by the processor to perform the steps of the method described above.

[0047] To achieve the above objectives, the present invention provides a computer-readable storage medium storing a multimodal three-dimensional MRI segmentation program based on non-empty voxelization and geometric awareness attention. When the multimodal three-dimensional MRI segmentation program based on non-empty voxelization and geometric awareness attention is run by a processor, the steps of the method described above are executed.

[0048] The beneficial effects of this invention, based on a multimodal 3D MRI segmentation method, system, and medium using non-empty voxelization and geometrically perceptual attention, are as follows:

[0049] This invention, through the above-described technical solution, filters multimodal MRI volumetric data through non-empty voxelization in the input stage to obtain sparse voxels; in the encoding stage, it extracts sparse voxel features based on three-dimensional dynamic non-empty voxels and geometrically perceptive cross-attention downsampling; in the decoding stage, it progressively aligns and fuses the extracted sparse voxel features across scales based on geometrically perceptive cross-attention upsampling; and in the output stage, it outputs the segmentation results through convolution, thereby achieving reduced computational resource consumption, enhanced geometric perception capabilities, and improved model versatility while ensuring segmentation accuracy. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0051] Figure 1 This is the overall architecture diagram of the multimodal 3D MRI segmentation method based on non-empty voxelization and geometric perception attention of the present invention;

[0052] Figure 2 This is a schematic diagram of the feature extraction backbone used in each stage;

[0053] Figure 3 This is a schematic diagram of the overall structure of a preferred embodiment of the multimodal 3D MRI segmentation method based on non-empty voxelization and geometric perception attention of the present invention;

[0054] Figure 4 This is a schematic diagram of the architecture of the three-dimensional dynamic non-empty voxel converter module;

[0055] Figure 5 This is a schematic diagram of the geometric perception cross-attention fusion module;

[0056] Figure 6 This is a visualization of the segmentation results on the BraTS2021, ACDC, and MSD Prostate datasets;

[0057] Figure 7 This is a schematic diagram showing the non-empty percentage (%) and the percentage of embedded voxels preserved (%) during non-empty embedding.

[0058] Figure 8 This is a schematic diagram showing the average Dice score (%), crossover ratio (%), FLOPs, and number of parameters for different 3D segmentation methods on the BraTS2021, ACDC, and MSD prostate datasets.

[0059] Figure 9 This is a system architecture diagram of the multimodal 3D MRI segmentation system based on non-empty voxelization and geometrically perceptual attention, as presented in this invention.

[0060] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0062] This invention proposes a multimodal 3D MRI segmentation method based on non-empty voxelization and geometrically perceptual attention. The overall process is as follows: Figure 1As shown: The input multimodal volume data is first embedded through "non-empty voxelization", retaining only the set of voxels with anatomical significance to enter the subsequent backbone network; the encoding path is composed of four-level layered modules, each level containing "three-dimensional dynamic non-empty voxel Transformer (3DNVT) + geometrically aware cross-attention downsampling (GCA-Down)" to further refine sparse voxel features at each scale; then, in the decoding path, "geometrically aware cross-attention upsampling (GCA-Up)" is used to achieve cross-scale alignment and fusion step by step, and finally the segmentation result is output through convolution.

[0063] Figure 1 This is the overall architecture diagram of the multimodal 3D MRI segmentation method based on non-empty voxelization and geometric perception attention of the present invention. Figure 1 The overall segmentation network structure of this invention is shown. Its core process includes input multimodal MRI volume data, which is filtered by a non-empty voxelization module to obtain sparse voxels, and then sequentially enters a multi-level encoder structure. Each level includes a three-dimensional dynamic non-empty voxel Transformer and a geometrically perceptual cross-attention downsampling module. Finally, in the decoding stage, feature fusion is achieved through geometrically perceptual cross-attention upsampling, and the segmentation result is generated by the output convolutional layer. Figure 1 It intuitively reflects the overall path of "input filtering - hierarchical modeling - cross-scale alignment - output prediction".

[0064] To adapt to the differences between sparse and dense signals, the network uses sparse residual convolution and regular residual convolution as the feature extraction backbone in different branches (e.g., Figure 2 As shown, the whole process revolves around three main lines: "selective computation + geometric alignment + orientation-sensitive context modeling," which enables fine-grained segmentation under controlled computational complexity.

[0065] Figure 2 This section outlines the backbone of feature extraction used in each stage. It demonstrates the specific forms of sparse residual convolutional blocks and regular residual convolutional blocks. Sparse residual convolutional blocks are used to directly operate on non-empty voxel sets to efficiently preserve geometric information; regular residual convolutional blocks, on the other hand, operate on dense feature maps to supplement texture and detail information. Figure 2 The invention clearly distinguishes the different implementation methods of the two paths of sparse processing and dense processing.

[0066] like Figure 3 As shown, a preferred embodiment of the multimodal 3D MRI segmentation method based on non-empty voxelization and geometrically perceptual attention of the present invention includes the following steps:

[0067] Step S10: In the input stage, multimodal MRI volume data are screened by non-empty voxelization to obtain sparse voxels.

[0068] Step S20: In the encoding stage, sparse voxel features are extracted based on three-way dynamic non-empty voxels and geometric perception cross-attention downsampling.

[0069] In step S30, during the decoding stage, the refined sparse voxel features are progressively aligned and fused across scales based on geometric perception cross-attention upsampling.

[0070] Step S40: In the output stage, the segmentation result is output through convolution.

[0071] Specifically, step S10 includes:

[0072] Step S101: Embed the multimodal input using 3D convolution to obtain the initial feature map.

[0073] Step S101 includes:

[0074] Regarding the "non-empty voxelization" strategy in the input stage, firstly, 3D convolution is used to process the multimodal input. Embedding:

[0075] (1)

[0076] in, The differences are the height, width, and depth of the data. The number of modes is and the kernel size is . Step size is The output channel is The initial feature map is obtained. .

[0077] Step S102, based on the feature vector of each voxel Norm and threshold The comparison results generate an occupancy map, filter out the set of non-empty voxel coordinates, and extract the corresponding embedded features from the feature map.

[0078] Step S102 includes:

[0079] Based on the feature vector of each voxel Norm and threshold The comparison results generate an occupancy map:

[0080] (2)

[0081] in, Represents the occupancy map. The threshold is used to filter out the set of non-empty voxel coordinates:

[0082] (3)

[0083] in, The number of non-empty voxels; and the corresponding embedded features are extracted from the feature map:

[0084] (4).

[0085] Step S103: Based on the embedded features, a sparse voxel set with geometric coordinates is formed.

[0086] Step S103 specifically includes:

[0087] Forming a sparse voxel set with absolute geometric coordinates:

[0088] (5)

[0089] in, Represents the absolute coordinates of a voxel. For the corresponding feature vector, For voxel indexing.

[0090] Unlike traditional methods that only use relative position encoding, preserving absolute coordinates ensures that subsequent sparse processing steps can maintain a geometrically consistent reference system, reducing spatial mismatches from the source.

[0091] Figure 4 The Transformer (3DNVT) is a three-dimensional dynamic non-empty system. Its core feature is that it first adaptively divides the window into subsets according to the voxel density, and the number of subsets is determined by the number of voxels. With upper limit parameter Decide.

[0092] Figure 4 It is the architecture of a three-dimensional dynamic non-empty voxel converter module. Figure 4 This demonstrates the dynamic partitioning of the voxel set within a window, as well as the process of sorting and segmenting along the three orthogonal directions (X, Y, and Z). This structure significantly reduces the length of the attention sequence and fully models spatial dependencies in different directions, intuitively reflecting the orientation-sensitive context modeling concept of this invention.

[0093] In this embodiment, step S20 specifically includes:

[0094] Step S201: Adaptively divide the window into subsets based on voxel density, wherein the number of subsets is determined by the number of voxels. With upper limit parameter Decide:

[0095] (6)

[0096] in, For window The number of voxels in the middle The maximum number of voxels in a single subset. This is an indicator function.

[0097] Step S202, along Sequence segmentation is performed in three orthogonal directions. For example... The direction of the first The subset is:

[0098] (7)

[0099] in, voxels exist The sorting index is oriented. This "density adaptive + three-way divide and conquer" strategy reduces the sequence size from... Down to This reduces the complexity of the 3D Transformer from Downgraded to Experiments have shown that 3DNVT alone results in an approximately 1.42% increase in Dice.

[0100] In this embodiment, cross-scale fusion is performed by geometrically perceptive cross-attention (GCA).

[0101] Figure 5 This is a schematic diagram of the geometric perception cross-attention fusion module. Figure 5 The structure comprises two sub-modules: downsampling and upsampling. In the downsampling stage, pooling features are used as queries, and high-resolution voxel features are used as keys and values, achieving cross-scale alignment through the introduction of geometric coordinates. In the upsampling stage, encoded residual features are used as queries, and decoded features are used as keys and values, achieving cross-layer fusion and boundary refinement. This figure visually illustrates the core role of the geometry-aware mechanism in the transformation process across different scales.

[0102] Step S30 specifically includes:

[0103] Step S301: In the downsampling (GCA-Down) stage, the coarse-scale query vector is first obtained by max pooling. Then, the high-resolution voxel features and their geometric coordinates are used as key values ​​in the cross-attention process.

[0104] (8)

[0105] in, This indicates the attention of the bulls. For position encoding;

[0106] Step S302: In the upsampling (GCA-Up) stage, reverse alignment is performed using the encoded residual as the query and the coarse features at the decoding end as the key.

[0107] (9)

[0108] The resulting updated sparse voxel set was obtained. Ablation experiments showed that removing the GCA resulted in an average accuracy decrease of 2.28%, with a maximum decrease of up to 5.37% in complex regions.

[0109] To accommodate parallel extraction of sparse and dense representations, Figure 2 The two types of residual backbones serve sparse voxel data and original dense volume data, respectively: the sparse branch uses sparse residual convolution blocks (SP-Conv k3s1 / k1s1+InstanceNorm+activation), which are directly operated on the sparse index; the dense branch uses regular residual convolution blocks (Conv k3s1 / k1s1+InstanceNorm+activation), which preserve the texture and details of the original volume data. Both types work alternately at various scales with 3DNVT and GCA, achieving a balance between "geometric consistency" and "texture integrity," thus obtaining clearer contours and fewer spurious segments in challenging regions such as tumor boundaries, thin layers of myocardium, and the prostate transition zone.

[0110] This invention demonstrates that the proposed scheme can significantly reduce the number of embedded voxels on different datasets: nearly 80% reduction in embedded voxels on low non-empty ratio datasets (such as BraTS2021 and ISLES2022), and still 30%–40% reduction on high non-empty ratio datasets (such as MSD Prostate and ACDC) (see details). Figure 7 This resulted in a 6.74% reduction in FLOPs and a 51.22% reduction in inference latency, while maintaining stable or even improved segmentation accuracy. Figure 7 The percentage of non-empty voxels preserved during the non-empty embedding method proposed in this invention is (%).

[0111] From end-to-end results, this invention achieves superior overall performance compared to existing methods on three major multimodal 3D MRI benchmarks (BraTS2021, ACDC, and MSDProstate): the average Dice and IoU are 92.05% / 86.90%, 92.69% / 86.17%, and 73.50% / 60.41%, respectively, setting new baselines on multiple structural segmentation tasks (see...). Figure 8 Meanwhile, with an input size of 1×128×128×128, only 572.8 GFLOPs and 62.5M parameters are required, significantly reducing computational costs compared to a strong baseline. Figure 6 The visualization results further show that the present invention generates more coherent masks and contours that fit the real anatomy in fragmented and complex regions, verifying the unified benefits of "high precision and high efficiency" brought about by the synergy of non-empty voxelization, 3DNVT and GCA. Figure 8These are the average Dice score (%), crossover ratio (%), FLOPs, and number of parameters for different 3D segmentation methods on the BraTS2021, ACDC, and MSD prostate datasets. These efficiency metrics are evaluated based on an input volume of 1×128×128×128.

[0112] Figure 6 This is a visualization of the segmentation results on the BraTS2021, ACDC, and MSD Prostate datasets. It demonstrates the segmentation effect of this invention on multiple typical datasets, especially in fragmented regions and areas with complex boundaries, where it more closely matches the actual anatomical contours compared to comparative methods. Figure 6 The results verified the actual performance improvement brought about by the core module of the present invention.

[0113] Furthermore, regarding the implementation methods and alternative paths of this invention: non-empty voxelization can be used in conjunction with volumetric cropping, image registration, and intensity normalization to enhance robustness; the window and threshold of 3DNVT can be dynamically adjusted according to the target organ volume and resolution, and a cross-window overlap strategy can also be introduced; the "query-key" configuration of GCA can be interchanged with different "decoding-encoding" residual combinations, and upsampling alignment can be replaced by learnable convolution instead of interpolation; in resource-constrained environments, the inference end can crop channels and layers, or further optimize using half-precision / quantized inference, all while maintaining the core idea of ​​"first screening—direction modeling—geometric alignment". In summary, this invention... Figure 1 Based on this approach, selective computation of "non-empty voxelization", orientation-sensitive modeling of "three-dimensional dynamic Transformer", and cross-scale spatial alignment of "geometric perception cross attention" are organically integrated to form a closed-loop system from input screening to decoding and reconstruction. Its accuracy and efficiency advantages have been verified on multiple public benchmarks, and its flexibility and robustness in implementation are ensured through various alternative implementations, meeting the requirements of clinical scenarios for reliability, real-time performance and transferability.

[0114] The overall design concept of this invention demonstrates significant advantages in multimodal 3D MRI segmentation tasks. However, there are still several technical implementation paths that can be modified or altered in the overall architecture and core modules. These alternative solutions can serve as extended implementations of this invention, or they may become potential circumvention design directions for competitors. Therefore, it is necessary to describe and protect them in this specification.

[0115] From the perspective of the overall framework, this invention adopts an encoder-decoder structure based on U-Net, combined with non-empty voxelization, a three-way dynamic non-empty voxel Transformer, and a geometry-aware cross-attention module to achieve segmentation. However, the core idea of ​​this framework is to achieve efficient sparse modeling and geometric alignment, rather than being limited to a specific network form. Therefore, in alternative implementations, other types of deep network structures can be used to achieve the same technical goals. For example, fully convolutional networks based on sparse convolution, Mamba structures based on sequence modeling, or graph neural networks can be used to perform graph structure modeling of non-empty voxels to capture spatial contextual relationships. Although these alternative methods differ in specific forms, their essence is still efficient modeling of sparse representation of three-dimensional voxels and global spatial dependencies, and all should be considered equivalent variations of this invention.

[0116] Regarding non-empty voxelization strategies, although this invention achieves invalid voxel removal through threshold comparison, other schemes can also achieve similar functions through learning-based sparse selection networks, region-proposal-based pruning algorithms, or attention-guided dynamic sampling strategies. Although these methods have different implementation mechanisms, they all share the core idea of ​​"reducing the computational cost of invalid voxels and concentrating computational resources on dissecting key regions," which is consistent with the design purpose of this invention.

[0117] In implementing the three-dimensional dynamic non-empty voxel Transformer module, possible alternatives include using non-orthogonal directional partitioning (such as principal component directions or learned feature directions), neighborhood partitioning based on hash indices, or using a hybrid approach of local convolution and sparse attention for orientation-sensitive modeling. Although these schemes differ in the details of data partitioning and attention implementation, they all share the common feature of "density-adaptive and orientation-sensitive spatial dependency modeling," which is essentially the same as the technology of this invention.

[0118] For geometrically perceptual cross-attention mechanisms, there are also several alternative implementation paths. Besides explicit voxel coordinate embedding, learnable position embedding, multi-scale spatial registration, or spatial transformation networks (STNs) can be used to achieve geometric alignment, thereby establishing correspondences between features at different resolutions. Although these alternative methods do not explicitly use coordinate information, their core function remains to maintain the spatial consistency of features across scales, and therefore they also fall within the scope of this invention's technical equivalent solutions.

[0119] From a broader perspective, if competitors abandon voxel-based methods in segmentation tasks and instead adopt sparse modeling methods based on point clouds or surface meshes, achieving 3D structure segmentation through point sets or surface representations, the underlying principle remains consistent with the technical concept of this invention. Regardless of whether voxels, point clouds, or surface structures are used, as long as the core objective is to improve the accuracy and efficiency of 3D medical image segmentation through sparse computation and spatial geometric alignment, it can be considered an extension of the concept of this invention.

[0120] In summary, the core technical idea of ​​this invention lies in achieving efficient and high-precision 3D medical image segmentation through sparse screening, orientation-sensitive modeling, and geometric consistency alignment. Any alternative solution that follows this overall design concept but differs only in implementation path or model structure should be considered as an alternative technical solution or equivalent variation of this invention.

[0121] The beneficial effects of this invention's multimodal 3D MRI segmentation method based on non-empty voxelization and geometrically perceptual attention are:

[0122] This invention, through the above-described technical solution, filters multimodal MRI volumetric data through non-empty voxelization in the input stage to obtain sparse voxels; in the encoding stage, it extracts sparse voxel features based on three-dimensional dynamic non-empty voxels and geometrically perceptive cross-attention downsampling; in the decoding stage, it progressively aligns and fuses the extracted sparse voxel features across scales based on geometrically perceptive cross-attention upsampling; and in the output stage, it outputs the segmentation results through convolution, thereby achieving reduced computational resource consumption, enhanced geometric perception capabilities, and improved model versatility while ensuring segmentation accuracy.

[0123] To achieve the above objectives, this invention also proposes a multimodal 3D MRI segmentation system based on non-empty voxelization and geometrically perceptual attention, such as... Figure 9 As shown, the system includes a processor 1001, a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002, and a multimodal 3D MRI segmentation program based on non-empty voxelization and geometrical attention stored on the processor. The communication bus 1002 is used to establish communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM or a non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0124] Those skilled in the art will understand that Figure 9 The system structure shown does not constitute a limitation on the system and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0125] like Figure 9As shown, the memory 1005, which serves as a computer storage medium, may include an operating device, a network communication module, a user interface module, and a multimodal 3D MRI segmentation program based on non-empty voxelization and geometric perception attention.

[0126] exist Figure 9 In the system shown, the network interface 1004 is mainly used to connect to the network server and communicate with the network server; the user interface 1003 is mainly used to interact with the user terminal and receive user input instructions; and the processor 1001 can be used to call the multimodal three-dimensional MRI segmentation program based on non-empty voxelization and geometric perception attention stored in the memory 1005.

[0127] To achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a multimodal three-dimensional MRI segmentation program based on non-empty voxelization and geometric awareness attention. When the multimodal three-dimensional MRI segmentation program based on non-empty voxelization and geometric awareness attention is run by a processor, the steps of the method described above are executed, which will not be repeated here.

[0128] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made under the concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A multimodal 3D MRI segmentation method based on non-empty voxelization and geometrical attention, characterized in that, The method includes the following steps: Step S10: In the input stage, multimodal MRI volume data are screened by non-empty voxelization to obtain sparse voxels; Step S20: In the encoding stage, sparse voxel features are extracted based on three-dimensional dynamic non-empty voxels and geometric perception cross-attention downsampling. Step S30: In the decoding stage, the refined sparse voxel features are progressively aligned and fused across scales based on geometric perception cross-attention upsampling; Step S40: In the output stage, the segmentation result is output after convolution; Step S10 includes: Step S101: Embed the multimodal input using 3D convolution to obtain the initial feature map; Step S102, based on the feature vector of each voxel Norm and threshold The comparison results generate an occupancy map, filter out the set of non-empty voxel coordinates, and extract the corresponding embedded features from the feature map; Step S103: Form a sparse voxel set with geometric coordinates based on the embedded features; Step S20 includes: Step S201: Adaptively divide the window into subsets based on voxel density, wherein the number of subsets is determined by the number of voxels. With upper limit parameter Decide: （6） in, For window The number of voxels in the middle The maximum number of voxels in a single subset. For indicator functions; Step S202, along Sequence segmentation is performed in three orthogonal directions; Step S30 includes: Step S301: In the downsampling stage, firstly, a coarse-scale query vector is obtained by max pooling. Then, the high-resolution voxel features and their geometric coordinates are used as key values ​​in the cross-attention process. （8） in, This indicates the attention of the bulls. For position encoding; Step S302: In the upsampling stage, reverse alignment is performed using the encoded residual as the query and the coarse features at the decoding end as the key. （9） The updated sparse voxel set is finally obtained.

2. The multimodal 3D MRI segmentation method based on non-empty voxelization and geometrical awareness attention as described in claim 1, characterized in that, Step S101 includes: Using 3D convolution for multimodal input Embedding: （1） in, The differences are the height, width, and depth of the data. The number of modes is and the kernel size is . Step size is The output channel is The initial feature map is obtained. .

3. The multimodal 3D MRI segmentation method based on non-empty voxelization and geometrical awareness attention as described in claim 2, characterized in that, Step S102 includes: Based on the feature vector of each voxel Norm and threshold The comparison results generate an occupancy map: （2） in, Represents the occupancy map. The threshold is used to filter out the set of non-empty voxel coordinates: （3） in, The number of non-empty voxels; and the corresponding embedded features are extracted from the feature map: （4）。 4. The multimodal 3D MRI segmentation method based on non-empty voxelization and geometrical awareness attention as described in claim 3, characterized in that, Step S103 includes: Forming a sparse voxel set with absolute geometric coordinates: （5） in, Represents the absolute coordinates of a voxel. For the corresponding feature vector, For voxel indexing.

5. A multimodal three-dimensional MRI segmentation system based on non-empty voxelization and geometrical attention, characterized in that, The system includes a memory, a processor, and a multimodal 3D MRI segmentation program based on non-empty voxelization and geometric awareness attention stored on the processor, wherein the multimodal 3D MRI segmentation program based on non-empty voxelization and geometric awareness attention is executed by the processor to perform the steps of the method as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a multimodal 3D MRI segmentation program based on non-empty voxelization and geometrically aware attention, which, when run by a processor, executes the steps of the method as described in any one of claims 1 to 4.

Images