Brain tumor segmentation-oriented symmetric sensing Transform method
By introducing prior knowledge of brain symmetry into the Transformer brain tumor segmentation model and employing input label symmetry and attention map symmetry methods, the accuracy and generalization ability of brain tumor segmentation are improved, thus addressing the challenges posed by data scarcity.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-10
- Publication Date
- 2026-03-31
AI Technical Summary
Existing Transformer-based brain tumor segmentation methods lack generalization ability under conditions of data scarcity, making it difficult to accurately identify and segment lesions and rare tumors in small regions, and their segmentation accuracy is limited.
By incorporating prior knowledge of brain symmetry, the model enhances its focus on lesion regions through two methods: input label symmetry and attention map symmetry. Input label symmetry extracts the differences in features between the whole brain and the left and right hemispheres through a two-way parallel processing structure, while attention map symmetry optimizes the attention distribution by calculating the similarity of symmetric labels.
Without adding extra labeled data, the model's segmentation accuracy and generalization ability were improved, effectively solving the performance degradation problem caused by data scarcity.
Smart Images
Figure CN121767722A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a symmetric perception Transformer method and device for brain tumor segmentation. Background Technology
[0002] Brain tumors are among the most prevalent diseases threatening human health, characterized by their difficulty in diagnosis, long diagnostic time, and low margin for error. Manual diagnosis and segmentation of the lesion area are inefficient and hinder early detection and treatment, potentially missing the optimal window for intervention. Therefore, developing an efficient, fully automated brain tumor segmentation method is of great significance for clinical practice.
[0003] Brain tumor segmentation falls under the specific task of medical image segmentation, and therefore its segmentation methods are closely related to the development of medical image segmentation. Medical images, compared to natural images, typically exhibit low contrast, blurred boundaries, and inaccurate visual recognition. Commonly used medical imaging techniques include computed tomography (CT), ultrasound, magnetic resonance imaging (MRI), and X-rays. The goal of medical image segmentation is to separate different organs or lesion regions within an image. This task involves automatically or semi-automatically performing dense prediction of voxels in the medical image, thereby segmenting the image into different meaningful regions. These regions are independent of each other and each possesses some degree of similarity. In medical imaging, these regions often correspond to different tissue types, pathologies, organs, or other biological structures. Early traditional brain tumor segmentation methods often relied on manually designed features based on prior medical knowledge. The advantage of this method is that it allows for the design of different feature templates for different organs under the guidance of physicians, leading to more accurate segmentation. However, this approach is time-consuming and carries a degree of uncertainty. With the emergence and continuous development of deep learning, Convolutional Neural Networks (CNNs) have demonstrated a superior ability to extract image features. They can adaptively learn image features based on input data without human intervention, and their segmentation performance can surpass that of traditional segmentation methods. Therefore, later segmentation methods increasingly tend to design fully automatic, end-to-end models based on deep learning.
[0004] Deep learning has gradually become the best solution in the field of brain tumor segmentation in recent years. It can adaptively learn features at different levels based on the data itself. Therefore, designing brain tumor segmentation models no longer requires specialized medical knowledge; only continuous adjustment and optimization of the model architecture are needed. Recently, inspired by the success of the Transformer model in natural language processing, the Transformer has been further extended to perform visual recognition tasks. The ViT model addresses the problem of insufficient receptive field in CNNs, which prevents them from effectively extracting long-distance dependencies, by introducing a Transformer module, thus achieving more accurate image classification. The Transformer is also applied in the field of medical image segmentation, mainly in two types: supplementing CNNs with Transformers to improve segmentation results and pure Transformer architectures. 2D segmentation models require inputting and outputting slices one by one when processing 3D datasets. This is time-consuming and treats each slice as independent data, which is not conducive to discovering the intrinsic relationships between different slices of the same case. To compensate for this deficiency, a large number of 3D segmentation methods have been proposed, but these methods are all affected by the scarcity of data in medical scenarios, resulting in insufficient generalization.
[0005] Data scarcity is a significant challenge for Transformer models in brain tumor segmentation. Brain tumor segmentation is a complex task requiring a large amount of labeled data to train the model for accurate identification and segmentation of tumor regions. However, due to the time-consuming and expensive nature of brain tumor annotation and the need to protect patient privacy, available labeled data is often very limited, directly impacting the performance and generalization ability of the trained model. The model struggles to learn sufficient features and contextual information from limited data, leading to performance degradation, especially for lesions with small regional proportions and rare tumor types or variants. Due to the lack of sufficient samples, the model may be unable to accurately identify and segment these special types of tumors and regions. Furthermore, the limited data volume may result in a lack of diverse and representative training samples, causing the model to be unable to adapt well to brain tumor images with different distributions, thus reducing its generalization ability.
[0006] Therefore, there is an urgent need for a brain tumor segmentation framework that can adapt to the scarcity of data in medical scenarios and achieve improved generalization performance in a plug-and-play manner while remaining compatible with existing Transformer-based 3D segmentation methods. This invention is proposed against this backdrop, aiming to address the key issue of efficient, fully automated brain tumor segmentation. Summary of the Invention
[0007] This invention provides a symmetric sensing Transformer method and device for brain tumor segmentation, which addresses the shortcomings of existing technologies.
[0008] In a first aspect, the present invention provides a symmetry-aware Transformer method for brain tumor segmentation, comprising: enhancing the segmentation effect of lesion regions in a Transformer-based brain tumor segmentation model through symmetry processing; wherein the symmetry processing is based on prior knowledge of the symmetry of the brain, and guides the input or processing stage of the brain tumor segmentation model by extracting the difference information of the features of the left and right hemispheres, so as to enhance the model's focus on lesion regions.
[0009] According to the symmetry-sensing Transformer method for brain tumor segmentation provided by the present invention, the symmetry processing includes input label symmetry processing, which includes: dividing whole-brain feature labels into left-brain and right-brain parts; adopting a two-way parallel processing structure, wherein the first path processes whole-brain feature labels based on Transformer, and the second path processes left-brain and right-brain feature labels separately based on different Transformers; fusing the left-right brain feature difference features output by the second path with the whole-brain features from the first path, and outputting the fused features.
[0010] The symmetric perception Transformer method for brain tumor segmentation provided by the present invention adopts a two-way parallel processing structure, including: adding learnable positional encoding to the first path to extract whole-brain features; and using fixed sine and cosine positional encoding based on the symmetry axis offset to extract left-brain and right-brain features to generate left-right brain difference features. Both paths use a multi-head self-attention mechanism with half the number of heads, and the left-brain and right-brain feature labels in the second path share Transformer parameters.
[0011] According to the symmetric perception Transformer method for brain tumor segmentation provided by the present invention, the left and right brain feature difference features output by the second path are fused with the whole brain features output by the first path, and the fused features are output. The method includes: expanding the left and right brain feature difference features to the size of the whole brain features; and fusing the expanded left and right brain feature difference features with the whole brain features by adding pixels one by one.
[0012] According to the symmetry-aware Transformer method for brain tumor segmentation provided by the present invention, the symmetry processing includes attention map symmetry processing. The attention map symmetry processing method includes: adopting a two-way parallel processing structure, where both the first and second paths employ a multi-head self-attention mechanism with half the number of heads; the first path extracts whole-brain features through a Transformer; and performing attention map symmetry processing on the second path's Transformer, including: linearly mapping the input whole-brain feature labels into a query, key, and value matrix, and calculating the attention map; identifying symmetric label pairs, calculating cosine similarity, and ranking them; wherein, the symmetric label pairs are determined based on symmetrical labels in the left and right hemispheres; determining a specific proportion of target symmetric labels with the lowest similarity, and modifying the attention map according to the target symmetric labels so that high-similarity labels only focus on themselves.
[0013] According to the symmetric perception Transformer method for brain tumor segmentation provided by the present invention, the types of brain tumor segmentation models include: TransBTS, BiTr-Unet, mmFormer, UNETR, and SegTransVAE models.
[0014] The symmetric perception Transformer method for brain tumor segmentation provided by the present invention further includes: applying the method to a 3D brain tumor segmentation task.
[0015] In a second aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the symmetric perception Transformer method for brain tumor segmentation as described above.
[0016] Thirdly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the symmetric perception Transformer method for brain tumor segmentation as described above.
[0017] The symmetric sensing Transformer method and device for brain tumor segmentation provided by this invention have the following advantages compared with the prior art: (1) This invention innovatively introduces prior knowledge of brain symmetry and improves model performance without adding additional labeled data by using two methods: input label symmetry and attention map symmetry. The document emphasizes that "under the premise of not being able to obtain additional training samples, this project attempts to make up for the lack of data by using the inherent symmetry of the brain from the perspective of medical prior information".
[0018] (2) The input label symmetry processing method provided by the present invention extracts whole brain features and left and right brain difference features through a two-way parallel processing architecture, guides the model to focus on asymmetrical lesion areas, and improves segmentation accuracy.
[0019] (3) The attention map symmetry processing method provided by the present invention introduces symmetric markers to calculate similarity in the attention mechanism, optimizes the attention distribution, makes the model focus on the lesion area, and improves the segmentation accuracy.
[0020] (4) The symmetric perception Transformer method provided by the present invention effectively solves the problems of data scarcity, insufficient generalization ability and limited segmentation accuracy in the prior art through two innovative mechanisms: input label symmetry and attention map symmetry. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0022] Figure 1 This is a framework diagram of the input mark symmetry processing provided by the present invention; Figure 2 This is a framework diagram of the attention map symmetry processing provided by the present invention; Figure 3 These are schematic diagrams illustrating the visual experimental results of different models provided by this invention; Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0024] It should be noted that, in the description of the embodiments of the present invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.
[0025] This invention provides a symmetry-aware Transformer method for brain tumor segmentation, comprising: enhancing the segmentation effect of lesion regions in a Transformer-based brain tumor segmentation model through symmetry processing; the symmetry processing is based on prior knowledge of the symmetry of the brain, and guides the input or processing stage of the brain tumor segmentation model by extracting the difference information of the features of the left and right hemispheres, so as to enhance the model's focus on lesion regions.
[0026] Optionally, the types of brain tumor segmentation models include: TransBTS, BiTr-Unet, mmFormer, UNETR, and SegTransVAE models.
[0027] Considering the natural symmetry of the human brain, which is disrupted by lesions such as brain tumors, it can be abstractly assumed that asymmetrical areas are more likely to contain the tumor. Therefore, artificially highlighting the differences between the left and right hemispheres in the model can theoretically help the model focus more on the lesion area, thus linking the lesion with the asymmetrical feature. This invention incorporates prior knowledge of brain symmetry into a brain tumor segmentation model, explores different methods for guiding brain tumor symmetry, and designs reasonable improvement measures from multiple perspectives, including the input stage and the model processing stage. Two Transformer symmetry enhancement methods are proposed: input label symmetry and attention map symmetry.
[0028] Method 1: Input tag symmetry Previous work has thoroughly explored the effect of adding symmetrical input to the original input, which can achieve faster convergence speed, but the improvement in accuracy is not significant. This may be because, in the original input, the shallow network extracts low-level features such as texture and details. These features are usually greatly affected by noise, resulting in some noise-induced differences in high-level features. In order to eliminate this difference, this invention does not add extra input in the early stage of the network, but directly processes high-level features. By extracting the differences in high-level semantic features between the left and right hemispheres, the network is guided to learn the potential relationship between semantic differences caused by asymmetry and lesions.
[0029] As an optional embodiment, the symmetric perception Transformer method for brain tumor segmentation provided by the present invention includes the following input label symmetry processing method: dividing the whole-brain feature labels into left-brain and right-brain parts; adopting a two-way parallel processing structure, wherein the first path processes the whole-brain feature labels based on Transformer, and the second path processes the left-brain and right-brain feature labels separately based on different Transformers; fusing the left-brain and right-brain feature difference features output by the second path with the whole-brain features from the first path, and outputting the fused features.
[0030] The system employs a two-way parallel processing structure, which includes: adding learnable positional codes to the first path to extract whole-brain features; and using fixed sine and cosine positional codes based on the symmetry axis offset to extract left-brain and right-brain features to generate left-brain and right-brain difference features. Both paths use a multi-head self-attention mechanism with half the number of heads, and the left-brain and right-brain feature labels in the second path share Transformer parameters.
[0031] As an optional embodiment, the symmetric perception Transformer method for brain tumor segmentation provided by the present invention fuses the left-right brain feature difference features output by the second path with the whole-brain features output by the first path, and outputs the fused features, including: expanding the left-right brain feature difference features to the size of the whole-brain features; and fusing the expanded left-right brain feature difference features with the whole-brain features by adding pixels one by one.
[0032] Method 2: Attention Map Symmetry The input label symmetry method treats the Transformer module as a black box model, highlighting symmetrical position information by modifying the input and operating on the output. Similarly, the internal computational mechanism of the Transformer can be designed to enhance symmetry information while mitigating attention redundancy.
[0033] As an optional embodiment, the symmetric perception Transformer method for brain tumor segmentation provided by the present invention includes the following methods for attention map symmetry processing: A two-way parallel processing structure is adopted, with both the first and second paths employing a multi-head self-attention mechanism that halves the number of heads. The first approach extracts whole-brain features using a Transformer; The attention graph of the second-path Transformer is symmetricized, including: The input whole-brain feature labels are linearly mapped to a query, key, and value matrix, and an attention map is calculated. Identify symmetrical label pairs, calculate cosine similarity, and sort them; where symmetrical label pairs are determined based on symmetrical labels in the left and right hemispheres of the brain; Identify a specific proportion of target symmetric labels with the lowest similarity, and modify the attention map based on the target symmetric labels so that highly similar labels focus only on themselves.
[0034] The method provided by this invention can be used in 3D brain tumor segmentation tasks.
[0035] In the absence of additional training samples, this invention attempts to leverage the inherent symmetry of the brain by incorporating a symmetry perception module into the Transformer model from the perspective of prior medical information. It proposes two symmetry enhancement schemes: input token symmetry and attention map symmetry, to compensate for the performance degradation caused by insufficient training data and to fully utilize the global modeling capabilities of the Transformer module.
[0036] In summary, the symmetric sensing Transformer method and device for brain tumor segmentation provided by this invention have the following advantages compared with the prior art: (1) This invention innovatively introduces prior knowledge of brain symmetry and improves model performance without adding additional labeled data by using two methods: input label symmetry and attention map symmetry. The document emphasizes that "under the premise of not being able to obtain additional training samples, this project attempts to make up for the lack of data by using the inherent symmetry of the brain from the perspective of medical prior information".
[0037] (2) The input label symmetry processing method provided by the present invention extracts whole brain features and left and right brain difference features through a two-way parallel processing architecture, guides the model to focus on asymmetrical lesion areas, and improves segmentation accuracy.
[0038] (3) The attention map symmetry processing method provided by the present invention introduces symmetric markers to calculate similarity in the attention mechanism, optimizes the attention distribution, makes the model focus on the lesion area, and improves the segmentation accuracy.
[0039] (4) The symmetric perception Transformer method provided by the present invention effectively solves the problems of data scarcity, insufficient generalization ability and limited segmentation accuracy in the prior art through two innovative mechanisms: input label symmetry and attention map symmetry.
[0040] To make the purpose, technical solution, and advantages of this invention patent clearer, the invention will be further described in detail below with reference to the figures and embodiments.
[0041] Method 1: Input tag symmetry To fully perceive the long-distance dependencies between left and right brain features, this invention considers pruning the feature tokens input to the Transformer module. Since the downsampling portion of the encoder is primarily composed of convolutional layers, and convolution has its unique inherent inductive bias—namely, locality, spatial invariance, and translation equivalence—this invention allows us to assume that at different layers, the left and right halves of the features correspond to the patient's left and right brain features, respectively. Thus, this invention can separate and process the high-level features of the left and right brains.
[0042] Figure 1 This is a framework diagram of the input tag symmetry processing provided by the present invention, such as... Figure 1As shown, the N cascaded standard Transformer modules in the model can be replaced by the structure on the right side of the diagram. Yellow represents left-brain tokens, and blue represents right-brain tokens. The input is processed in parallel through two paths. Position embeddings (PEs) need to be added before inputting into the Transformer modules. The upper path (first path) adds learnable PEs that change continuously as the model is trained. The lower path (second path) may have variable token sizes due to symmetry axis offset issues, so fixed-value sine and cosine position embeddings are used based on the specific size. The structure of the two Transformer modules is completely consistent with the standard Transformer, but to maintain relative consistency in the number of parameters, the number of heads in the multi-head attention of the two Transformers is reduced to half that of the standard Transformer. The top path achieves the same effect as the standard Transformer, extracting long-range features from the whole brain. The bottom path inputs the tokens from the left and right hemispheres into a different Transformer than the top path. It's important to note that the left and right hemisphere tokens share the Transformer's parameters; that is, the bottom path parameters are not updated until the left and right hemispheres have passed through the Transformer and obtained their outputs. This ensures that the extracted feature differences remain stable. The output features from the left and right hemispheres are subtracted. Since the Transformer module does not change the feature size, the feature difference map size is half that of the whole-brain features from the top path. To maintain size consistency, the feature difference map is mirrored and expanded to its original size, and then fused with the whole-brain features before output. Experiments have shown that the fusion method—using linear layers to restore the size after stacking along the channel dimension—is not significantly different from direct element-wise addition. For simplicity, subsequent experiments all use the direct element-wise addition method.
[0043] Method 2: Attention Map Symmetry Attention mechanisms are key to the Transformer's success across multiple tasks, but their high computational complexity imposes unavoidable limitations on their applications. The high computational complexity stems from the fact that each token input to the Transformer requires calculating its attention value with all other tokens (including itself). Often, calculating attention between two tokens may be unnecessary. For example, in a brain tumor image, there are many pure black backgrounds. Distant black backgrounds clearly have low correlation, and calculating their attention would be highly redundant.
[0044] To address this, the present invention designs a method that focuses attention on tokens in asymmetrical regions of the left and right hemispheres: attention map symmetry. Figure 2This is a framework diagram of attention map symmetry processing provided by the present invention, such as... Figure 2 As shown, similar to the symmetric input tokenization, the module designed in this invention can be used plug-and-play to replace the standard Transformer module. The input token is also divided into two parallel computation paths. Both the upper and lower paths employ a multi-head self-attention mechanism with half the number of heads. The upper path is completely identical to the standard Transformer, used to extract whole-brain features. The multi-head self-attention design of the lower path is the innovation of this method. First, the input token (as a whole-brain feature token) is linearly mapped to a query, key-value, and value matrix (Q, K, V). Then, an attention map of size N×N×h is calculated using the following formula, where h is the number of heads.
[0045]
[0046] in It is a normalization factor. This refers to the Q / K / V dimension, used to avoid gradient vanishing during training. Input sequence N is the total number of tokens, and k is the dimension of the encoded token.
[0047] Next, we find symmetrical markers i and j in the left and right hemispheres of the brain. We can find N / 2 such pairs of markers. We then extract the rows (row i and row j) corresponding to these N / 2 groups of tokens. Each row represents the attention of the corresponding token to all other tokens. The more similar two tokens are, the more similar their attention to other tokens are considered to be. To reduce computational complexity, we prune the values at coordinates i,j,j,i,i,i,j,j in the attention of rows i and j, making each row of length N. The vector is 2, meaning it only contains attention to tokens other than the symmetric token pair. Cosine similarity is calculated pairwise between symmetric token pairs, resulting in a similarity matrix of size N / 2×h. The similarity between token pairs in each head is sorted, and the K% of tokens with the lowest similarity are recorded. Here, K is determined by the maximum foreground proportion of a single case in the training data, ensuring that theoretically all lesion tokens can be addressed without excessive redundancy. Next, the original attention map is modified based on the recorded token numbers, so that tokens with high similarity no longer focus on other tokens but only on themselves, while attention to tokens with low similarity is retained. Finally, the outputs from both paths are fused.
[0048] Furthermore, this invention selected five different Transformer-based brain tumor segmentation models as baseline models and introduced the method proposed in this invention for comparison.
[0049] The experimental results are shown in Table 1. The baseline model and its improvements are placed in adjacent rows, where ST represents input label symmetry and SA represents attention map symmetry. The first four columns after the method list show the Dice metrics, namely: whole tumor WT, tumor core TC, enhanced tumor ET, and average Dice metrics; the last four columns show the 95% Hausdorff distance metrics, which are consistent with the Dice metrics.
[0050] Table 1 Comparison of Experimental Results
[0051] Figure 3 This is a schematic diagram of the visualization experimental results of different models provided by the present invention.
[0052] Based on the experimental results, the segmentation performance of each benchmark model is ranked as follows: mmFormer > SegTransVAE > TransBTS > BiTr-Unet > UNETR.
[0053] The main contribution of this invention is as follows: 1) This paper introduces symmetric prior information into the Transformer model for brain tumor segmentation, building upon previous methods. Two Transformer-based symmetry perception methods for brain tumor segmentation—input label symmetryization and attention map symmetryization—are proposed, with detailed implementation processes and design principles described.
[0054] 2) The effectiveness of the proposed Transformer symmetric enhancement method was verified. It effectively improves the segmentation performance of the original model while maintaining similar running speed and resource consumption, thus achieving better assistance in brain tumor diagnosis.
[0055] 3) The universality of the proposed Transformer symmetry enhancement method is demonstrated. Extending the proposed method to multiple benchmark segmentation models, it achieves improved segmentation performance on a specific dataset (BraTS2020), indicating that introducing symmetry information into brain tumor segmentation methods can help the model better learn lesion features.
[0056] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440. The processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a symmetry-aware Transformer method for brain tumor segmentation. This method includes enhancing the segmentation effect on lesion regions in a Transformer-based brain tumor segmentation model through symmetry processing. The symmetry processing is based on prior knowledge of brain symmetry and guides the input or processing stages of the brain tumor segmentation model by extracting differences in features between the left and right hemispheres, thereby enhancing the model's focus on lesion regions.
[0057] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0058] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the symmetry-aware Transformer method for brain tumor segmentation provided in the above embodiments, the method comprising: enhancing the segmentation effect of lesion regions by symmetry processing in a Transformer-based brain tumor segmentation model; the symmetry processing is based on prior knowledge of symmetry in the brain, and guides the input or processing stage of the brain tumor segmentation model by extracting the difference information of left and right brain features, so as to enhance the model's focus on lesion regions.
[0059] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the symmetry-aware Transformer method for brain tumor segmentation provided in the above embodiments. The method includes: enhancing the segmentation effect on lesion regions in a Transformer-based brain tumor segmentation model through symmetry processing; the symmetry processing is based on prior knowledge of symmetry in the brain, and guides the input or processing stage of the brain tumor segmentation model by extracting the difference information of features between the left and right hemispheres, so as to enhance the model's focus on lesion regions.
[0060] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A symmetric perception Transformer method for brain tumor segmentation, characterized in that, include: In the Transformer-based brain tumor segmentation model, symmetry processing is used to enhance the segmentation effect on the lesion region; The symmetry processing is based on prior knowledge of the brain's symmetry. By extracting the difference information of the features of the left and right hemispheres, it guides the input or processing stages of the brain tumor segmentation model to enhance the model's focus on the lesion area.
2. The symmetric perception Transformer method for brain tumor segmentation according to claim 1, characterized in that, The symmetry processing method includes input tag symmetry processing, and the input tag symmetry processing method includes: The whole-brain feature markers are divided into left-brain and right-brain regions; A two-way parallel processing structure is adopted. The first path processes whole-brain feature labels based on Transformer, and the second path processes left-brain and right-brain feature labels separately based on different Transformers. The left and right brain feature differences from the second output are fused with the whole brain features from the first output to output the fused features.
3. The symmetric perception Transformer method for brain tumor segmentation according to claim 2, characterized in that, It adopts a two-way parallel processing structure, including: Learnable location codes are added to the first path to extract whole-brain features; For the second path, fixed sine and cosine position coding is used based on the symmetry axis offset to extract left brain and right brain features to generate left and right brain difference features; Both pathways utilize a multi-head self-attention mechanism with half the number of heads, and in the second pathway, the feature labels of the left and right hemispheres share Transformer parameters.
4. The symmetric perception Transformer method for brain tumor segmentation according to claim 2, characterized in that, The left-right brain feature differences from the second output are fused with the whole-brain features from the first output, and the fused features are output, including: Expand the differences in features between the left and right hemispheres to the size of features across the entire brain; By adding pixels one by one, the expanded left and right brain feature differences are fused with whole brain features.
5. The symmetric perception Transformer method for brain tumor segmentation according to claim 1, characterized in that, The symmetry processing methods include attention map symmetry processing, and the methods for attention map symmetry processing include: A two-way parallel processing structure is adopted, with both the first and second paths employing a multi-head self-attention mechanism that halves the number of heads. The first approach extracts whole-brain features using a Transformer; The attention graph of the second-path Transformer is symmetricized, including: The input whole-brain feature labels are linearly mapped to a query, key, and value matrix, and an attention map is calculated. Identify symmetrical label pairs, calculate cosine similarity, and sort them; where symmetrical label pairs are determined based on symmetrical labels in the left and right hemispheres of the brain; Identify a specific proportion of target symmetric labels with the lowest similarity, and modify the attention map based on the target symmetric labels so that highly similar labels focus only on themselves.
6. The symmetric perception Transformer method for brain tumor segmentation according to claim 1, characterized in that, The types of brain tumor segmentation models include: TransBTS, BiTr-Unet, mmFormer, UNETR, and SegTransVAE models.
7. The symmetric perception Transformer method for brain tumor segmentation according to claim 1, characterized in that, Also includes: The method was applied to a 3D brain tumor segmentation task.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the symmetric perception Transformer method for brain tumor segmentation as described in any one of claims 1 to 7.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the symmetric perception Transformer method for brain tumor segmentation as described in any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the symmetric perception Transformer method for brain tumor segmentation as described in any one of claims 1 to 7.