A brain tumor detection system based on optimized image segmentation
By combining a dynamic multi-scale hierarchical encoder and a dual-path collaborative cross-attention network, the uncertainties and single-modal accuracy problems in brain tumor segmentation and detection are solved, achieving high-precision tumor detection and personalized feature enhancement.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- LETING COUNTY HOSPITAL
- Filing Date
- 2026-04-13
- Publication Date
- 2026-07-10
AI Technical Summary
Existing brain tumor segmentation and detection methods suffer from uncertainties, poor interpretability, and low accuracy of single-modal detection results.
An image segmentation method based on a dynamic multi-scale hierarchical encoder is adopted. It uses content-aware adaptive grids and gradient-weighted candidate label generation, combined with a dual-path collaborative cross-attention network, to dynamically modulate the visual feature channels using physiological data, establish a deep interaction loop between visual and physiological information, and perform multimodal data fusion.
It improves the accuracy and robustness of tumor detection, reduces computational complexity, enhances the amount of feature information, conforms to clinical knowledge in neuro-oncology, and achieves personalized image feature enhancement.
Smart Images
Figure CN122368092A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent medicine, specifically a brain tumor detection system based on optimized image segmentation. Background Technology
[0002] In neurosurgery and radiation oncology, accurate analysis of brain tumors, especially malignant brain tumors such as glioblastoma, is crucial. Traditional methods relying on manual delineation by physicians have significant limitations. Precise tumor segmentation and detection are fundamental for developing surgical plans, planning radiotherapy targets, and evaluating treatment efficacy. Quantifying tumor volume and location is essential for prognosis. However, existing tumor segmentation and detection methods suffer from uncertainties, poor interpretability, and low accuracy of single-modal detection results. Summary of the Invention
[0003] To address the aforementioned issues and overcome the shortcomings of existing technologies, this invention provides a brain tumor detection system based on optimized image segmentation. Addressing the problems of uncertainty and poor interpretability in existing tumor segmentation and detection methods, this paper proposes an image segmentation method based on a dynamic multi-scale hierarchical encoder. Through content-aware adaptive grids, it can focus on both brain image details and capture global structure. Gradient-weighted candidate label generation ensures that boundary regions containing key discriminative information (for tumor discrimination) are highlighted in feature representation. The introduction of boundary-aware importance scoring reduces computational complexity while retaining the most informative features. Uncertainty estimation and guided propagation improve the robustness of segmentation. To address the low accuracy of single-modal detection results, this invention incorporates semantic labels of brain segmentation maps as explicit priors into attention calculations, allowing the model to automatically assign higher weights to high-risk brain regions, aligning with clinical knowledge in neuro-oncology. Structured physiological data is used to dynamically modulate visual feature channels, achieving individualized image feature enhancement. Furthermore, it breaks through traditional one-way fusion by establishing a deep interactive loop between visual and physiological information, enabling the two modalities to act as contexts during decision-making, further improving the accuracy of tumor detection.
[0004] The technical solution adopted by the present invention is as follows: The present invention provides a brain tumor detection system based on optimized image segmentation, including a multimodal data collaborative acquisition module, a data quality enhancement module, a brain segmentation module and a tumor detection module;
[0005] The multimodal data collaborative acquisition module acquires MRI and CT images of the patient's brain using medical equipment, registers all images to the same coordinate system and fuses them to obtain a CT-MRI image of the brain, and acquires the patient's physiological data, including blood routine, tumor marker tests and cerebrospinal fluid tests.
[0006] The data quality enhancement module denoises all images and uses a local anomaly factor algorithm to identify and remove outliers in physiological data.
[0007] The brain segmentation module employs a brain segmentation method based on dynamic multi-scale hierarchical segmentation to segment the brain from the CT-MRI image, thereby obtaining a brain segmentation map.
[0008] The tumor detection module employs a dual-path collaborative cross-attention network to perform deep feature extraction and fusion of brain segmentation maps and physiological data, obtaining fused features, and classifying and identifying tumors based on these fused features to obtain tumor detection results.
[0009] Furthermore, in the brain segmentation module, a brain segmentation method based on dynamic multi-scale hierarchical segmentation specifically includes the following steps:
[0010] Step S1: Three-dimensional reconstruction, by performing three-dimensional reconstruction on CT-MRI images to obtain a three-dimensional image;
[0011] Step S2: Multi-scale feature extraction, constructing a feature pyramid with L=4 layers to encode features of the 3D image, with each layer outputting a feature map. Where s represents the number of layers, and the feature dimension is C×H×W×D, which represent the number of channels, height, width, and depth, respectively.
[0012] Step S3: Construct a content-aware adaptive mesh, calculate the local gradient magnitude mesh density, and adaptively calculate the mesh size for different layers based on the gradient magnitude. ; ;
[0013] In the formula, This represents the gradient vector at voxel x in the 3D plot. Gradient magnitude is used to estimate the complexity of local content. This represents the preset grid size. This represents the adaptive amplitude coefficient, which controls the range of mesh variation. This represents the gradient sensitivity coefficient, which adjusts the degree of response to gradients. This represents the normalized gradient magnitude at level l. Represents the grid size at level l;
[0014] Step S4: Feature map partitioning. The feature maps of different layers are partitioned according to the grid size as the window size to obtain the grid cells of the feature maps of different layers.
[0015] Step S5: Dynamic candidate label generation. Within each adaptive grid cell, candidate labels are generated through weighted pooling. : ;
[0016] In the formula, This represents the spatial region corresponding to the i-th grid cell. Gradient-weighted functions assign higher weights to regions with high gradients. The feature vector representing position x in layer s. Represents candidate tags;
[0017] Step S6: Candidate label mapping, learning a trainable codebook C={ }, introduce a soft allocation mechanism to assign each candidate tag Mapped to its nearest codebook element: ;
[0018] In the formula, Representative mark The soft weight assigned to the k-th codebook element, Represents preset parameters. Represents the size of the codebook. Represents the mapping result;
[0019] Step S7: Importance score calculation. A lightweight boundary scoring network is introduced to calculate the boundary scores of the mapped candidate labels, and the importance scores of the candidate labels are calculated based on the boundary scores. ; ;
[0020] In the formula, Represents the activation function. Represents multilayer perceptron processing, This represents the boundary score, i.e., the probability that the candidate label lies on the boundary. Represents the information gain coefficient. This represents the number of times the codebook element has been mapped. Represents the total number of candidate tags. The importance score represents the candidate marker;
[0021] Step S8: Candidate label selection. The Top-K selection method is used to filter candidate labels based on their importance scores, resulting in filtered labels. Each filtered label is associated with an uncertainty estimate. Uncertainty is introduced during reprojection. ;
[0022] In the formula, The vector portion of the feature map representing the filter marker. =VarNet( ), representing the uncertainty estimate of the label. Represents preset conditions;
[0023] Step S9: Uncertainty-guided feature propagation, progressive upsampling, and using uncertainty-guided feature propagation to obtain the decoding result, i.e., the segmentation mask. The 3D image is then segmented based on the segmentation mask to obtain the brain segmentation map. The uncertainty-guided feature propagation calculation is as follows: ;
[0024] In the formula, This represents a local convolutional block used for feature refinement. This represents a splicing operation. The representation is the feature map encoded at layer s. This represents an upsampling operation. The graph represents the uncertainty estimates after upsampling. This represents the upsampling result of layer s;
[0025] Furthermore, in the tumor detection module, the data processing of a dual-path collaborative cross-attention network specifically includes the following steps:
[0026] Step M1: Feature extraction. The brain segmentation map is extracted using the Vision Transformer model to obtain a spatial feature map. The physiological data is then mapped to a high-dimensional space using a multilayer perceptron to obtain physiological features.
[0027] Step M2: Spatial feature map weighting. Activation classification is performed on all locations in the spatial feature map to obtain the category label for each location. Through a learnable label embedding layer, discrete categories are mapped to continuous vectors. These continuous vectors are then fused with the spatial feature map to obtain an attention weight map. The spatial feature map is then weighted using the attention weight map to obtain the visual features. ; ;
[0028] In the formula, Represents the category label at position (i,j). Represents the embedding layer. Represents spatial feature map, This represents a convolutional layer, used to fuse information and output single-channel weights. Represents the Sigmoid activation function. Represents the attention weights graph. Represents visual characteristics;
[0029] Step M3: Dynamic gating of physiological data. Physiological features are mapped using a learnable projection matrix to obtain a gating vector. The gating vector is then multiplied by the visual features to obtain the gated visual features.
[0030] Step M4: Collaborative cross-attention fusion. Construct a query vector using gated visual features, construct key and value vectors using physiological features, and perform attention mechanism calculations to obtain physiological-to-visual modulation features. Construct a query vector using physiological features, construct key and value vectors using gated visual features, and perform attention mechanism calculations to obtain visual-to-physiological modulation features. Fuse the two modulation features to obtain the fused feature.
[0031] Step M5: Classification detection. The fused features are activated and classified to output the results of tumor presence, location, and classification type, i.e., the tumor detection results.
[0032] The beneficial results achieved by the present invention using the above solution are as follows:
[0033] (1) To address the problems of uncertainty and poor interpretability in existing tumor segmentation and detection methods, this paper proposes an image segmentation method based on a dynamic multi-scale hierarchical encoder. Through content-aware adaptive grids, it can focus on both brain image details and capture global structure. Through gradient-weighted candidate label generation, it ensures that the boundary regions containing key discriminative information (for tumor discrimination) are highlighted in the feature representation. The introduction of boundary-aware importance scoring reduces computational complexity while retaining the most informative features. Uncertainty estimation and guided propagation improve the robustness of segmentation.
[0034] (2) To address the problem of low accuracy of single-modal detection results, this invention incorporates the semantic labels of brain segmentation maps as explicit priors into attention calculation, enabling the model to automatically assign higher weights to high-risk brain regions, which aligns with clinical knowledge in neuro-oncology. By dynamically modulating visual feature channels using structured physiological data, individualized image feature enhancement is achieved. At the same time, it breaks through the traditional one-way fusion, establishing a deep interactive loop between visual and physiological information, so that the two modalities serve as contexts for each other during decision-making, further improving the accuracy of tumor detection. Attached Figure Description
[0035] Figure 1 This is a block diagram of a brain tumor detection system based on optimized image segmentation provided by the present invention.
[0036] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. Detailed Implementation
[0037] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0038] Example 1, see Figure 1 The present invention provides a brain tumor detection system based on optimized image segmentation, comprising a multimodal data collaborative acquisition module, a data quality enhancement module, a brain segmentation module and a tumor detection module;
[0039] The multimodal data collaborative acquisition module acquires MRI and CT images of the patient's brain using medical equipment, registers all images to the same coordinate system and fuses them to obtain a CT-MRI image of the brain, and acquires the patient's physiological data, including blood routine, tumor marker tests and cerebrospinal fluid tests.
[0040] The data quality enhancement module denoises all images and uses a local anomaly factor algorithm to identify and remove outliers in physiological data.
[0041] The brain segmentation module employs a brain segmentation method based on dynamic multi-scale hierarchical segmentation to segment the brain from the CT-MRI image, thereby obtaining a brain segmentation map.
[0042] The tumor detection module employs a dual-path collaborative cross-attention network to perform deep feature extraction and fusion of brain segmentation maps and physiological data, obtaining fused features, and classifying and identifying tumors based on these fused features to obtain tumor detection results.
[0043] Example 2: Based on the above examples, this example describes a brain segmentation method based on dynamic multi-scale hierarchical segmentation in the brain segmentation module, specifically including the following steps:
[0044] Step S1: Three-dimensional reconstruction, by performing three-dimensional reconstruction on CT-MRI images to obtain a three-dimensional image;
[0045] Step S2: Multi-scale feature extraction, constructing a feature pyramid with L=4 layers to encode features of the 3D image, with each layer outputting a feature map. Where s represents the number of layers, and the feature dimension is C×H×W×D, which represent the number of channels, height, width, and depth, respectively.
[0046] Step S3: Construct a content-aware adaptive mesh, calculate the local gradient magnitude mesh density, and adaptively calculate the mesh size for different layers based on the gradient magnitude. ; ;
[0047] In the formula, This represents the gradient vector at voxel x in the 3D plot. Gradient magnitude is used to estimate the complexity of local content. This represents the preset grid size. This represents the adaptive amplitude coefficient, which controls the range of mesh variation. This represents the gradient sensitivity coefficient, which adjusts the degree of response to gradients. This represents the normalized gradient magnitude at level l. Represents the grid size at level l;
[0048] Step S4: Feature map partitioning. The feature maps of different layers are partitioned according to the grid size as the window size to obtain the grid cells of the feature maps of different layers.
[0049] Step S5: Dynamic candidate label generation. Within each adaptive grid cell, candidate labels are generated through weighted pooling. : ;
[0050] In the formula, This represents the spatial region corresponding to the i-th grid cell. Gradient-weighted functions assign higher weights to regions with high gradients. The feature vector representing position x in layer s. Represents candidate tags;
[0051] Step S6: Candidate label mapping, learning a trainable codebook C={ }, introduce a soft allocation mechanism to assign each candidate tag Mapped to its nearest codebook element: ;
[0052] In the formula, Representative mark The soft weight assigned to the k-th codebook element, Represents preset parameters. Represents the size of the codebook. Represents the mapping result;
[0053] Step S7: Importance score calculation. A lightweight boundary scoring network is introduced to calculate the boundary scores of the mapped candidate labels, and the importance scores of the candidate labels are calculated based on the boundary scores. ; ;
[0054] In the formula, Represents the activation function. Represents multilayer perceptron processing, This represents the boundary score, i.e., the probability that the candidate label lies on the boundary. Represents the information gain coefficient. This represents the number of times the codebook element has been mapped. Represents the total number of candidate tags. The importance score represents the candidate marker;
[0055] Step S8: Candidate label selection. The Top-K selection method is used to filter candidate labels based on their importance scores, resulting in filtered labels. Each filtered label is associated with an uncertainty estimate. Uncertainty is introduced during reprojection. ;
[0056] In the formula, The vector portion of the feature map representing the filter marker. =VarNet( ), representing the uncertainty estimate of the label. Represents preset conditions;
[0057] Step S9: Uncertainty-guided feature propagation, progressive upsampling, and using uncertainty-guided feature propagation to obtain the decoding result, i.e., the segmentation mask. The 3D image is then segmented based on the segmentation mask to obtain the brain segmentation map. The uncertainty-guided feature propagation calculation is as follows: ;
[0058] In the formula, This represents a local convolutional block used for feature refinement. This represents a splicing operation. The representation is the feature map encoded at layer s. This represents an upsampling operation. The graph represents the uncertainty estimates after upsampling. This represents the upsampling result of layer s.
[0059] Example 3: Based on the above examples, this example describes a dual-path collaborative cross-attention network, specifically including the following steps:
[0060] Step M1: Feature extraction. The brain segmentation map is extracted using the Vision Transformer model to obtain a spatial feature map. The physiological data is then mapped to a high-dimensional space using a multilayer perceptron to obtain physiological features.
[0061] Step M2: Spatial feature map weighting. Activation classification is performed on all locations in the spatial feature map to obtain the category label for each location. Through a learnable label embedding layer, discrete categories are mapped to continuous vectors. These continuous vectors are then fused with the spatial feature map to obtain an attention weight map. The spatial feature map is then weighted using the attention weight map to obtain the visual features. ; ;
[0062] In the formula, Represents the category label at position (i,j). Represents the embedding layer. Represents spatial feature map, This represents a convolutional layer, used to fuse information and output single-channel weights. Represents the Sigmoid activation function. Represents the attention weights graph. Represents visual characteristics;
[0063] Step M3: Dynamic gating of physiological data. Physiological features are mapped using a learnable projection matrix to obtain a gating vector. The gating vector is then multiplied by the visual features to obtain the gated visual features.
[0064] Step M4: Collaborative cross-attention fusion. Construct a query vector using gated visual features, construct key and value vectors using physiological features, and perform attention mechanism calculations to obtain physiological-to-visual modulation features. Construct a query vector using physiological features, construct key and value vectors using gated visual features, and perform attention mechanism calculations to obtain visual-to-physiological modulation features. Fuse the two modulation features to obtain the fused feature.
[0065] Step M5: Classification detection. The fused features are activated and classified to output the results of tumor presence, location, and classification type, i.e., the tumor detection results.
[0066] Example 4: This example, based on the above examples, uses the scheme of the present invention for precise detection of glioblastoma at multiple centers:
[0067] Application scenario: A hospital, in collaboration with multiple municipal hospitals, has built a multi-center brain tumor auxiliary diagnostic platform, focusing on early detection and preoperative assessment of glioblastoma (GBM).
[0068] Data collection
[0069] Multimodal data collaborative acquisition module: Acquires preoperative T1 enhanced MRI, T2-FLAIR MRI and CT images of patients, and fuses them into CT-MRI images after registration; at the same time, it collects blood routine (white blood cell and neutrophil counts), tumor markers (GFAP, IDH1 mutation status) and cerebrospinal fluid biochemical indicators.
[0070] Data quality enhancement module: Uses nonlocal mean filtering to denoise images; uses Local Outlier Factor (LOF) to remove outliers caused by sampling errors in physiological data;
[0071] Segmentation and Detection Process
[0072] Brain segmentation module: The fused 3D image is encoded with a four-layer feature pyramid, the content-aware grid size is adaptively adjusted, and candidate labels are generated by gradient weighting. After being filtered by codebook mapping and boundary scoring network, uncertainty-guided feature propagation is used to finally generate a segmentation map containing white matter, gray matter, cerebrospinal fluid and tumor regions.
[0073] Tumor detection module:
[0074] Using the segmentation map as input, ViT extracts spatial feature maps;
[0075] Using anatomical tags (insula, basal ganglia) to guide spatial attention, the weight of high-risk areas (insula) is increased;
[0076] Physiological data is embedded by an MLP to generate gated vectors, which are then used to dynamically modulate the visual feature channels.
[0077] Bidirectional cross-attention integrates physiological and visual information to output the probability of GBM presence, tumor volume, and IDH mutation prediction.
[0078] Results: On the internal test set (n=215), the tumor detection AUC reached 0.963, which is 9.2% higher than that of the traditional single-modality CNN; on the external validation set (n=78), the AUC was 0.941, showing good generalization; uncertainty estimation effectively identified 3 cases with ambiguous boundaries, indicating that manual review is required.
[0079] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0080] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
[0081] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.
Claims
1. A brain tumor detection system based on optimized image segmentation, characterized in that: It includes a multimodal data collaborative acquisition module, a data quality enhancement module, a brain segmentation module, and a tumor detection module; The multimodal data collaborative acquisition module acquires MRI and CT images of the patient's brain using medical equipment, registers all images to the same coordinate system and fuses them to obtain a CT-MRI image of the brain, and acquires the patient's physiological data, including blood routine, tumor marker tests and cerebrospinal fluid tests. The data quality enhancement module denoises all images and uses a local anomaly factor algorithm to identify and remove outliers in physiological data. The brain segmentation module employs a brain segmentation method based on dynamic multi-scale hierarchical segmentation to segment the brain from the CT-MRI image, thereby obtaining a brain segmentation map. The tumor detection module employs a dual-path collaborative cross-attention network to perform deep feature extraction and fusion of brain segmentation maps and physiological data, obtaining fused features, and classifying and identifying tumors based on these fused features to obtain tumor detection results.
2. The brain tumor detection system based on optimized image segmentation according to claim 1, characterized in that: In the brain segmentation module, a brain segmentation method based on dynamic multi-scale hierarchical segmentation specifically includes the following steps: A three-dimensional image is obtained by reconstructing CT-MRI images in three dimensions; A feature pyramid with L=4 layers is constructed to encode the features of the 3D image, with each layer outputting a feature map. Where s represents the number of layers, and the feature dimension is C×H×W×D, which represent the number of channels, height, width, and depth, respectively. Construct a content-aware adaptive mesh, calculate the local gradient magnitude mesh density, and adaptively calculate the mesh size of different layers based on the gradient magnitude; The feature maps of different layers are divided according to the grid size as the window size, resulting in grid cells for the feature maps of different layers; Within each adaptive grid cell, candidate labels are generated through weighted pooling. ; Learn a trainable codebook C={ }, introduce a soft allocation mechanism to assign each candidate tag It is mapped to its nearest codebook element; A lightweight boundary scoring network is introduced to compute the boundary scores of the mapped candidate labels, and the importance scores of the candidate labels are calculated based on the boundary scores. The Top-K selection method is used to filter candidate labels based on their importance scores, resulting in a filtered label. Each filtered label is associated with an uncertainty estimate, and uncertainty is introduced during reprojection. Progressive upsampling and uncertainty-guided feature propagation are used to obtain the decoding result, i.e., the segmentation mask. The 3D image is then segmented based on the segmentation mask to obtain the brain segmentation map.
3. The brain tumor detection system based on optimized image segmentation according to claim 2, characterized in that: In the tumor detection module, data processing using a dual-path collaborative cross-attention network specifically includes the following steps: The brain segmentation map is extracted using the Vision Transformer model to obtain a spatial feature map. The physiological data is then mapped to a high-dimensional space using a multilayer perceptron to obtain physiological features. Activation classification is performed on all locations in the spatial feature map to obtain the category label for each location. Through a learnable label embedding layer, discrete categories are mapped to continuous vectors, and the continuous vectors are fused with the spatial feature map to obtain the attention weight map. The spatial feature map is then weighted by the attention weight map to obtain the visual features. Physiological features are mapped using a learnable projection matrix to obtain a gating vector. The gating vector is then multiplied by the visual features to obtain the gated visual features. A query vector is constructed using gated visual features, and key and value vectors are constructed using physiological features. An attention mechanism is then performed to calculate the modulation features from physiological to visual. The query vector is constructed using physiological features, and key and value vectors are constructed using gated visual features. An attention mechanism is then performed to calculate the modulation features from visual to physiological. The two modulation features are then fused to obtain the fused features. The fusion features are activated and classified to output the results of tumor presence, location, and classification type, i.e., the tumor detection results.