MaskRcnn-based tumor detection method and system

Abstract

The invention provides a MaskRcnn-based tumor detection method and system, and the method comprises the steps: inputting medical image data into a MaskRcnn network integrated with an edge perception module, generating an edge response graph through a Sobel operator, and enabling the edge response graph to serve as an additional channel injection feature graph, and enhancing the boundary representation; fusing multi-scale features through a feature pyramid network FPN, extracting candidate tumor area features through RoI Align, introducing non-local attention modeling global dependence in the area, extracting directional entropy, texture energy, uniformity and contrast in combination with a gray level co-occurrence matrix GLCM, and splicing to generate structure sensing features; performing classification, bounding box regression and mask segmentation based on the structure perception features, and constructing a joint loss function including classification, regression, mask cross entropy, edge alignment and structure consistency; the positioning capability of the model on the fuzzy tumor contour is effectively improved, the boundary positioning error is remarkably optimized, and the defect that over-segmentation or missing detection is likely to occur in a traditional model is overcome.

Description

Technical Field

[0001] This invention relates to the field of tumor image analysis and detection technology, and in particular to a tumor detection method and system based on MaskRcnn. Background Technology

[0002] Early screening and accurate localization of tumors are key issues in modern medical image-assisted diagnosis. Existing methods mostly rely on manual delineation of two-dimensional slices and subjective judgment by doctors, which is not only time-consuming but also suffers from poor repeatability and low recognition accuracy. Especially when tumor boundaries are blurred and tissue heterogeneity is strong, traditional segmentation algorithms (such as U-Net, FCN, etc.) often experience missed detections or false detections. In recent years, instance segmentation networks based on Mask R-CNN have achieved remarkable results in object detection tasks. Combining candidate region recognition and pixel-level segmentation capabilities, they have shown great potential in the field of medical imaging. However, directly applying the standard Mask R-CNN structure to process solid tumor images still presents challenges: On the one hand, the blurred and irregular boundaries of tumors make it difficult to accurately locate the extent of the lesion. Taking common solid tumors such as glioma, pancreatic cancer, and liver cancer as examples, the tumor tissue often lacks clear grayscale or structural transitions with the surrounding tissues, exhibiting characteristics such as infiltrative growth, pseudopodia expansion, or uneven edge enhancement. Especially in T2-weighted MRI and enhanced CT images, the signal at the tumor edge often shows irregular shapes such as blurred transitions, irregular boundaries, and local enhancement.

[0003] On the other hand, the internal structure of tumor tissue is highly heterogeneous and the signal distribution is complex, which affects the segmentation stability. Most malignant tumors may form necrosis, cystic degeneration, calcification, hemorrhage or fibrosis areas during their evolution. These areas often show obvious signal discontinuity, local cavitation or mixed gray distribution in the images, which can easily lead to the omission of key pathological components or misjudgment of artifacts as tumor areas. Therefore, a tumor detection method and system based on MaskRcnn is proposed. Summary of the Invention

[0004] In view of this, the present invention provides a tumor detection method and system based on MaskRcnn to solve or alleviate the technical problems existing in the prior art, and at least provides a beneficial alternative.

[0005] The technical solution of this invention is implemented as follows: a tumor detection method based on MaskRcnn, comprising the following steps: First, MRI or enhanced CT medical image data containing solid tumor regions are acquired. The original images are then subjected to Z-score normalization, contrast-limited adaptive histogram equalization (CLAHE) enhancement, and 3×3 median filtering for noise reduction. This process aims to unify the image grayscale distribution, improve local texture contrast, and reduce image noise, providing clear and stable input data for subsequent deep instance segmentation. Secondly, the preprocessed medical image data is input into the MaskRcnn instance segmentation network with an integrated edge perception module. The edge perception module generates an edge response map based on the Sobel operator and injects this map as an additional channel into the feature map. At the same time, the convolutional block attention module CBAM is combined to enhance the perception of boundary regions in both spatial and channel dimensions. Through the synergistic effect of the ResNet-50 backbone network and the FPN structure, multi-scale contextual information is extracted, supporting high-precision candidate tumor region identification. Within the candidate tumor region, the instance segmentation network integrates the nonlocal attention module with texture features calculated based on the gray-level co-occurrence matrix (GLCM), including directional entropy, energy, uniformity, and contrast, to jointly construct a structure-aware feature representation. This representation is used to capture the global dependencies and micro-texture patterns of the lesion region, thereby improving the ability to identify heterogeneous lesions. During the training phase, based on the aforementioned structure-aware features, three sub-tasks—classification, bounding box regression, and mask segmentation—are performed simultaneously. A joint loss function is designed for end-to-end optimization. This joint loss function, based on the traditional three-item loss, further utilizes edge alignment loss, Sobel edge map Dice difference and structural consistency loss, and Jensen-Shannon divergence based on GLCM texture features to improve the accuracy of segmentation boundaries and the consistency of internal structure representation. Finally, the trained model is applied to the new image data to be detected, and the output includes structured detection results including the boundary mask of the tumor region, the maximum radial length, the texture structure feature index, and the indeterminate region hint map. It supports output in DICOM RT-STRUCT format and is compatible with JSON and HL7 formats. It can realize data docking with platforms such as radiotherapy planning system and image archiving and communication system PACS, thereby realizing automated detection and visual auxiliary diagnosis of solid tumors.

[0006] In addition, the present invention also provides a tumor detection system based on MaskRcnn, comprising: The image input module is used to acquire and preprocess medical image data; An instance segmentation module, connected to the image input module, is used to receive preprocessed images and identify candidate tumor regions based on the MaskRcnn network; The edge perception module is connected to the instance segmentation module. It enhances boundary features based on the Sobel edge response map and the convolutional block attention module CBAM, and feeds the results back to the instance segmentation module to optimize mask prediction. The structural modeling module, connected to the instance segmentation module, is used to apply a nonlocal attention module to the candidate region and combine the texture features extracted by the gray-level co-occurrence matrix (GLCM) to generate a structure-aware feature representation. The joint optimization module is connected to the edge perception module and the structural modeling module respectively, and is used to jointly train the classification, regression and segmentation tasks based on multiple loss terms. The results output module, connected to the joint optimization module, is used to generate detection results containing tumor mask, maximum radial length and structural feature information, and output them in DICOM RT-STRUCT format.

[0007] The embodiments of the present invention have the following advantages due to the adoption of the above technical solutions: I. This invention addresses the common problems of blurred boundaries and irregular spread of solid tumors in medical imaging. It utilizes an edge-aware module in the MaskRcnn instance segmentation network, enhancing boundary features through edge response maps extracted by the Sobel operator. Furthermore, it integrates a CBAM attention module to highlight significant edge regions in both channel and spatial dimensions, effectively improving the model's ability to locate blurred tumor contours. Simultaneously, during the training phase, an edge alignment loss function is used, with the Dice coefficient measuring the consistency between the predicted mask and the true boundary, significantly optimizing boundary localization errors. This addresses the shortcomings of traditional models, which are prone to oversegmentation or missed detections, thereby improving the contour fitting accuracy and clinical applicability of tumor detection.

[0008] II. This invention addresses the challenges of strong tissue heterogeneity and complex signal variations in malignant tumor imaging. It designs a non-local attention module within candidate regions to capture long-range semantic associations within those regions. Furthermore, it combines gray-level co-occurrence matrices to extract texture features such as directional entropy, contrast, and energy, constructing a structure-aware feature representation to characterize various pathological morphologies including necrosis, hemorrhage, cystic degeneration, and calcification. During training, it utilizes a structural consistency loss function with Jensen-Shannon divergence constraints to predict the similarity in texture distribution between the mask and the real region, thereby improving the model's accuracy in representing tissues within lesions.

[0009] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is an overall flowchart of the MaskRcnn-based tumor detection method of the present invention; Figure 2 This is a schematic diagram of the segmentation network structure of an example of the present invention; Figure 3 This is a schematic diagram of nonlocal attention and GLCM texture modeling in this invention; Figure 4 This is a schematic diagram of the system module structure of the present invention. Detailed Implementation

[0012] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0013] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0014] Example 1 like Figure 1-4 As shown, this embodiment of the invention provides a tumor detection method based on MaskRcnn, including: Step S1: Medical Image Data Acquisition and Preprocessing In this embodiment, the raw medical image data to be processed comes from clinically collected or publicly available image datasets, covering common sites of solid tumors, such as the brain, head and neck, chest and abdomen, and pelvic cavity. Specific image types include T1-weighted images and T2-weighted images from magnetic resonance imaging (MRI), as well as enhanced computed tomography (CE-CT) images. Among them, MRI data is suitable for lesions with high resolution in soft tissues, such as gliomas and liver cancer, while CT images are mostly used for the identification and evaluation of nodules and metastases in solid organs. Before inputting the raw medical images into the deep neural network, this invention performs the following three preprocessing steps on each image: First, the images are Z-score standardized. Based on the pixel grayscale distribution of each image, the mean grayscale value of the image is subtracted and then divided by the standard deviation. This standardizes the grayscale intensity distribution of the image to the standard normal distribution range with a mean of 0 and a variance of 1, suppressing grayscale shifts caused by differences in imaging equipment, imaging protocols, or individual patients. The standardized grayscale values ​​are more consistent, which helps the network feature extraction module maintain stable performance across different data sources. Secondly, this invention employs a contrast-limited adaptive histogram equalization method, which divides the entire image into multiple local regions and performs histogram equalization on each region to enhance the local contrast of each region. In particular, it enhances the detail information in low-contrast regions such as tumor edges, low-density lesions, and background tissue transition zones. By limiting the maximum value of local contrast through a contrast-limited strategy, it prevents excessive enhancement of high-contrast regions, avoids distortion of texture information, and improves the visualization clarity of blurred boundaries and low-intensity tumor lesions. Finally, the present invention uses a median filter to smooth the image. The median filter window size is set to 3×3, which can effectively suppress the abrupt artifacts of salt-and-pepper noise while maintaining the continuity of image edges.

[0015] Step S2: Construction of the Edge-Aware Instance Segmentation Network After completing the standardization, enhancement, and denoising preprocessing of medical images, this embodiment constructs an instance segmentation model based on the MaskRcnn framework. On the basis of the traditional structure, it utilizes an edge-aware module and a non-local attention module to focus on enhancing the modeling ability of tumor boundaries and overcome the problems of blurred tumor edges and unclear gray levels in the transition zone. First, the backbone feature extraction network adopts the ResNet-50 structure to achieve deep semantic feature extraction while avoiding the gradient vanishing problem. Then, using the FPN structure, a bottom-up pyramid-style feature fusion method is used to integrate high-level semantics and low-level detailed information layer by layer, so that the network can take into account both the gross tumor outline and fine-grained edge features. This embodiment further integrates an edge perception module to enhance the boundary perception capability of the instance segmentation model. By employing the Sobel operator to perform edge detection on the original image, the gradient maps in the horizontal direction (Gx) and vertical direction (Gy) are calculated to obtain a complete edge response map. This edge response map can reflect the regions in the image with drastic gray-level changes, i.e., the edge contour structure. The aforementioned Sobel edge response map is used as an additional channel input and fused into the feature map output by the FPN during the feature extraction process, thereby improving the network's ability to recognize boundary region features. Furthermore, the Convolutional Block Attention (CBAM) module is used to significantly enhance the saliency of the fused feature map in both the channel and spatial dimensions. The channel attention module extracts the response differences between channels through max pooling and average pooling operations, and generates channel attention weights to suppress irrelevant channels and highlight boundary-related features. The spatial attention module generates a spatial attention map through channel-wise pooling and convolution operations, guiding the network to focus on significant spatial regions near the tumor margin; After feature map enhancement, the instance segmentation model uses the RoI Align module to extract regions of interest in detail. Unlike traditional RoI Pooling, RoI Align preserves pixel-level spatial information within candidate regions through bilinear interpolation, avoiding geometric distortion caused by quantization errors and ensuring accurate localization and structural representation of each candidate tumor region.

[0016] Step S3: Structural Feature Modeling and Fusion After extracting candidate tumor regions and enhancing their edge features, this step further refines the model's perception of the complex structures and heterogeneous tissues within the candidate regions through fine-grained modeling and semantic enhancement, thereby improving the model's ability to perceive morphological changes within solid tumors. This embodiment, based on MaskRcnn, utilizes a nonlocal attention module and the gray-level co-occurrence matrix (GLCM) texture feature extraction method to construct a structure-aware feature representation and perform fusion processing, providing high-quality feature input for the multi-task learning module.

[0017] The non-local attention module uses: For the feature map in each candidate region, three transformation spaces are constructed: query, key, and value. Linear projection is achieved through 1×1 convolution. The feature similarity between all pixels is calculated. The dot product operation is used to measure the response weight between any two positions. The global attention map is applied to the value space to achieve the fusion of long-distance pixel information. The output non-local feature map retains the semantic dependence of the overall structure of the tumor region and effectively makes up for the expression defects caused by the loss of local features or morphological blur. To further refine the identification of tumor internal tissue structures, this embodiment extracts multidimensional texture features based on the gray-level co-occurrence matrix (GLCM) within each candidate region. The operation process is as follows: A 32×32 sliding window is used to traverse the candidate region. For each window, the GLCM matrix is ​​calculated in four directions: 0°, 45°, 90° and 135°. Six statistical texture quantities are extracted from the matrix in each direction: directional entropy, energy, contrast, homogeneity, local variance and correlation. The average of each feature is taken in the four directions to finally construct a 24-dimensional texture feature vector, which can accurately reflect the local structural regularity, density distribution and texture consistency of tumor tissue.

[0018] In this embodiment, the feature map output by the non-local attention module is concatenated with the GLCM texture feature vector along the channel dimension to construct a structure-aware feature tensor. This structure-aware feature tensor is then input into a fusion network consisting of three convolutional layers, including: Each convolutional structure uses a 3×3 convolutional kernel, followed by a batch normalization layer and a ReLU activation function. The final output structure-aware feature tensor is a 128-dimensional embedding representation, which is used to support the feature input for subsequent classification, bounding box regression and mask segmentation tasks.

[0019] Step S4: Construction and Optimization Training of Joint Loss Function After completing the construction of the structure-aware feature representation, this step simultaneously executes multiple tumor recognition tasks based on the structural features, including tumor region classification, bounding box localization and mask segmentation. At the same time, to ensure the optimization stability and segmentation accuracy during the training process, the consistency of boundary structure and texture representation is further strengthened. The MaskRcnn branch network in this embodiment contains three typical subtask modules, including: Classification subtask: used to determine whether each candidate region belongs to tumor tissue; Bounding box regression task: used to accurately locate the position and size of the detection box; Mask segmentation task: used for pixel-level binary segmentation of tumor regions.

[0020] Correspondingly, the initial loss function consists of the following three parts: Classification loss Lcls: The softmax cross-entropy loss function is used to supervise the distinction between tumor and non-tumor. Regression loss Lbox: The Smooth L1 loss function is used to measure the center position and size deviation between the predicted box and the ground truth box; Mask loss Lmask: Pixel-level binary cross-entropy loss is used for the pixel labels within each positive sample candidate region.

[0021] The above three components constitute the basic multi-task loss function in the standard Mask R-CNN framework:

[0022] This embodiment utilizes an edge alignment loss function (Ledge) to enhance the geometric consistency between the mask prediction result and the true boundary, as follows: The Sobel operator is applied to the predicted mask and the corresponding real mask respectively to extract their edge response maps; Calculate the Dice coefficient between two edge maps to measure their degree of overlap; The 1-Dice value is introduced as a loss term during training to penalize boundary differences.

[0023] The loss term is expressed as follows:

[0024] in, The mask predicted by the model; A mask for actual annotation; Extract the edge response map of the input mask for the operator; This loss is used to measure the degree of overlap between two edge response maps (values ​​range from 0 to 1, with larger values ​​indicating greater overlap). This loss forces the model to learn the continuity and closure of tumor boundaries.

[0025] This embodiment utilizes the structural consistency loss function (Lstruct) to measure the similarity in structural features between the predicted mask and the real mask region. The calculation process is as follows: Using the Gray-Level Co-occurrence Matrix (GLCM) algorithm, texture feature vectors of orientation entropy, energy, and contrast are extracted from the predicted mask and the ground mask, respectively. The distribution difference between the two vectors in the multidimensional texture space is calculated, and the Jensen-Shannon divergence is used as a distance metric to obtain the structural consistency loss term.

[0026] in, The eigenvectors of the gray-level co-occurrence matrix (GLCM) are calculated within the prediction mask region; This refers to the corresponding GLCM feature vector calculated within the actual mask region; It is a symmetrical and bounded measure of distributional difference (values ​​range from 0 to 1, with smaller values ​​indicating more similar distributions). This structural loss effectively suppresses mask content misalignment, ensuring that the model not only has accurate boundaries but also matches real clinical pathological manifestations in terms of internal semantic features.

[0027] Finally, the joint loss function (Ltotal) constructed in this invention integrates classification, regression, segmentation, edge and structure multi-dimensional objectives, and its expression is as follows:

[0028] in, and These are the weight adjustment coefficients for edge alignment loss and structural consistency loss, respectively, with an empirical range of 0.2-0.5, which are adjusted based on the model convergence performance and mask quality during training. Step S5: Structured Result Generation and Output Module Building upon the trained instance segmentation network model, this step achieves the final inference and structured output for tumor detection. The medical image data to be analyzed is input into the trained model, which automatically identifies and outputs the detection results for each tumor instance, including: Tumor mask boundary: Outputs pixel-level segmentation results for each tumor region, represented by a binary mask image; Maximum radial length: The maximum diameter of the tumor is calculated by fitting the pair of furthest edge points, and is used for preoperative assessment; Structural feature indicators: Based on texture analysis, the orientation entropy, contrast, energy, and uniformity indicators of the tumor region are extracted to assist in determining the pathological type of the tumor; Uncertainty area hint map: Based on the confidence map output by the model, low confidence areas are marked to provide doctors with a reference for secondary interpretation.

[0029] The output format adopts the DICOM RT-STRUCT format, which conforms to medical image processing standards. It also supports JSON and HL7 structured data interface, and can be integrated into hospital PACS systems or radiotherapy planning systems to realize clinical auxiliary diagnosis and preoperative planning visualization of test results.

[0030] Example 2 In addition, the present invention also provides a tumor detection system based on MaskRcnn, comprising: The image input module is used to acquire and preprocess medical image data. The image acquisition source can be T1-weighted images, T2-weighted images from magnetic resonance imaging (MRI), or enhanced computed tomography (CT) images.

[0031] The instance segmentation module is connected to the image input module to receive preprocessed medical images and identify candidate tumor regions based on the improved MaskRcnn network. This instance segmentation module uses ResNet-50 and FPN structures for multi-scale feature extraction and combines RoIAlign to achieve accurate sampling of candidate regions.

[0032] The edge perception module is connected to the instance segmentation module. It uses the edge response map generated by the Sobel operator and the convolutional block attention module CBAM to enhance the saliency of tumor boundary features, and feeds the optimized boundary features back to the instance segmentation module to improve the mask prediction accuracy.

[0033] The structural modeling module is connected to the instance segmentation module to introduce a non-local attention module within the candidate tumor region, construct long-distance pixel dependencies, and combine the texture features extracted by the gray-level co-occurrence matrix (GLCM) to generate a structure-aware feature representation, thereby enhancing the model's ability to identify heterogeneous structures within the tumor.

[0034] The joint optimization module is connected to the edge perception module and the structural modeling module respectively. Based on multiple loss terms such as classification loss, bounding box regression loss, mask cross-entropy loss, edge alignment loss and structural consistency loss, it performs joint training on classification, regression and segmentation tasks.

[0035] The results output module connects with the joint optimization module to generate structured detection results, including tumor mask, maximum radial length, and structural feature information, and outputs them in DICOMRT-STRUCT format. It is also compatible with JSON and HL7 formats to interface with the radiotherapy planning system TPS or the image archiving and communication system PACS.

[0036] Example 3 Tumor detection performance testing and result analysis: The proposed MaskRcnn-based tumor detection method demonstrates performance advantages in complex lesion identification. The BraTS2021 publicly available glioma image dataset was used as the main validation platform. This dataset contains multiple magnetic resonance imaging (MRI) images, including T1-weighted (T1), T1-contrast-enhanced (T1ce), T2-weighted (T2), and fluid attenuation inversion recovery (FLAIR) sequences. In this experiment, T2 sequence images preprocessed in step S1 were uniformly selected as model input to ensure consistency between the data source and network input in terms of grayscale distribution and noise level. I. Comparison Methods U-Net: A typical encoder-decoder semantic segmentation network that uses a symmetric structure for context feature fusion but lacks instance segmentation capabilities; Standard MaskR-CNN: The original structure does not utilize edge-aware modules, non-local structure modeling, and joint loss functions; it only retains the basic instance segmentation framework. The method of this invention integrates an edge awareness module (Sobel operator generates edge responses + convolutional block attention module CBAM), a structural modeling module (non-local attention + texture features calculated based on gray-level co-occurrence matrix GLCM), and a joint loss function (classification loss, bounding box regression loss, mask cross-entropy loss, edge alignment loss, and structural consistency loss) on the basis of MaskRcnn.

[0037] II. Evaluation Indicators Dice coefficient: measures the degree of overlap between the predicted mask and the real mask; IoU (Intersection over Union): The intersection over union ratio is a metric used to evaluate the overall accuracy of mask prediction; Edge overlap: The Sobel operator is applied to both the predicted mask and the real mask to extract edge maps, and the Dice coefficient of the edge region is calculated to reflect the degree of boundary alignment; Time taken (ms / image): The average detection and segmentation time per image.

[0038] Comparison of experimental results (Table 1)

[0039] Results analysis shows that the method of this invention outperforms the comparative methods in all three metrics: Dice coefficient, IoU, and edge overlap. Compared with U-Net, the Dice coefficient increased from 0.812 to 0.872 (an improvement of approximately 7.39%), the IoU increased from 0.693 to 0.763 (an improvement of approximately 10.10%), and the edge overlap increased from 0.648 to 0.793 (an improvement of approximately 22.38%). Compared to the standard MaskRcnn, the Dice coefficient increased from 0.844 to 0.872 (an improvement of approximately 3.32%), the IoU increased from 0.726 to 0.763 (an improvement of approximately 5.09%), and the edge overlap increased from 0.687 to 0.793 (an improvement of approximately 15.44%). This indicates that, while maintaining the detection efficiency essentially unchanged, the present invention enhances the ability to locate tumor boundaries and the segmentation accuracy of tumors with complex structures, especially in cases with blurred boundaries and strong tissue heterogeneity. Although the time consumption is slightly increased (about 7ms / image), it is still within the clinically acceptable range.

[0040] Example 4 Module ablation analysis: To further verify the synergistic performance of the modules of this invention, variant models with different module combinations were constructed based on the standard Mask R-CNN, and ablation experiments were conducted under the same BraTS2021 dataset and experimental conditions as in Example 2. I. Model Setup Baseline (B1): Standard MaskRcnn, without any enhancement modules; B2: B1+ Edge Awareness Module (Sobel operator generates edge responses + Convolutional Block Attention Module CBAM); B3: B1+ Structure Modeling Module (Non-local attention + Texture features calculated based on Gray-Level Co-occurrence Matrix (GLCM)); B4: B1+ Joint Loss Function (adding edge loss + structural consistency loss). B5: The complete module combination of this invention (edge ​​perception module + structural modeling module + joint loss function).

[0041] II. Evaluation Indicators Dice coefficient: Overall segmentation accuracy; Edge overlap: the consistency between the predicted boundary and the actual boundary; Feature description: The ability to express details of tumor boundaries and texture of internal structures.

[0042] Ablation test results (Table 2)

[0043] Ablation test results show that the edge sensing module, structural modeling module, and joint loss function proposed in this invention all effectively improve performance. Regarding the Dice coefficient, the addition of the edge-aware module (B2) improved it by approximately 1.42% compared to the baseline model (B1), the structural modeling module (B3) improved it by approximately 0.95%, and the joint loss function (B4) improved it by approximately 1.78%. In terms of edge overlap, B2 improved it by approximately 11.50% compared to B1, B3 by approximately 9.46%, and B4 by approximately 13.84%. When the three components are used in combination (B5), the Dice coefficient increases from 0.844 to 0.872 (an increase of approximately 3.32%), and the edge overlap increases from 0.687 to 0.793 (an increase of approximately 15.44%), verifying that the synergistic effect between the modules of this invention can improve the overall performance of the model in complex tumor identification.

[0044] In this embodiment, the terminology is defined as follows: MaskRcnn is a deep convolutional neural network framework that combines object detection and pixel-level segmentation. It adds a mask prediction branch to Faster RCNN and can simultaneously output the object's category, location bounding box, and pixel mask. In this invention, MaskRcnn is used for candidate box detection and fine segmentation of tumor regions, and its improved structure enhances the ability to recognize complex boundaries and heterogeneity within tumors.

[0045] The Sobel operator is a commonly used edge detection operator. It generates a response map that reflects the edge intensity and direction by weighting the gradients of the image in the horizontal and vertical directions. This invention uses the Sobel operator to extract the edge response map of medical images and inputs it as an additional channel into a deep network to improve the perception accuracy of blurred tumor boundary areas.

[0046] The Convolutional Block Attention Module (CBAM) is a lightweight attention module that reweights the feature map for saliency by calculating weights in both channel attention and spatial attention dimensions. This invention utilizes CBAM in the edge perception module to highlight high-value features related to the tumor boundary and suppress background information irrelevant to diagnosis.

[0047] The nonlocal attention module achieves the interaction of distant pixel information by calculating the similarity between any two positions on the feature map, thereby capturing global dependencies. This invention uses a nonlocal attention module within the candidate tumor region, enabling the model to perceive the global spatial connections of pathological features such as central necrosis, ring enhancement, cystic degeneration, and calcification, thus improving the integrity of the tumor's internal structural representation.

[0048] Gray-level co-occurrence matrix (GLCM) is a commonly used method for statistical image texture. It extracts texture features such as energy, contrast, uniformity, and orientation entropy by calculating the joint occurrence probability of gray-level value pairs under specific directions and distances. This invention concatenates GLCM texture features with non-local attention features to form a "structure-aware feature representation" to enhance the model's ability to identify heterogeneous tumor tissues.

[0049] The joint loss function in this invention consists of classification loss, bounding box regression loss, mask cross-entropy loss, edge alignment loss, and structural consistency loss. Classification loss: measures the difference between the predicted class and the true label; Bounding box regression loss: optimizes the positional matching between candidate boxes and ground truth boxes; Mask cross-entropy loss: used for pixel-level mask accuracy optimization; Edge alignment loss: Improve edge overlap by extracting Sobel edge maps from the predicted and real masks respectively and calculating the difference in Dice coefficients; Structural consistency loss: Using texture vectors generated by GLCM, the difference between the predicted mask and the real mask under Jensen-Shannon divergence is calculated to ensure that the model maintains consistency in texture distribution within the tumor.

[0050] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A tumor detection method based on MaskRCNN, characterized in that, Includes the following steps: S1. Acquire medical imaging data containing solid tumor regions, including T1-weighted or T2-weighted magnetic resonance imaging images, or enhanced computed tomography images. S2. Input medical image data into the MaskRcnn instance segmentation network with an integrated edge perception module. The edge perception module generates an edge response map based on the Sobel operator and injects the edge response map as an additional channel into the feature map. S3. The Feature Pyramid Network (FPN) is used to fuse multi-scale features, and the Region of Interest Align (RoI) module is used to extract candidate tumor region features. In each candidate tumor region, the global dependency relationship of pixels in the region is modeled by the non-local attention module. At the same time, orientation entropy, texture energy, uniformity and contrast are extracted based on the Gray-Level Co-occurrence Matrix (GLCM) and stitched together to form structure-aware features. S4. Based on the structure-aware features, perform tumor region classification, bounding box regression and mask segmentation subtasks respectively, and construct a joint loss function, which includes classification loss, bounding box regression loss, mask cross-entropy loss, edge alignment loss and structural consistency loss. S5. Apply the trained segmentation network to new image data and output structured detection results including tumor mask, maximum radial length, structural feature indices, and indeterminate region hints.

2. The tumor detection method based on MaskRCNN according to claim 1, characterized in that: Before being input into the instance segmentation network, the medical image data described in step S1 undergoes the following preprocessing: Perform Z-score normalization on the image to unify the grayscale distribution; The contrast of local images is enhanced by using contrast-limited adaptive histogram equalization (CLAHE). Denoising was performed using a median filter with a 3×3 window, keeping the grayscale standard deviation of the image between 0.1 and 0.

3.

3. The tumor detection method based on MaskRcnn according to claim 1, characterized in that: The edge perception module in step S2 uses the Convolutional Block Attention Module (CBAM), which assigns attention weights in the channel dimension and spatial dimension respectively, and fuses the Sobel edge response map. The instance segmentation network uses a ResNet-50 backbone network combined with a Feature Pyramid Network (FPN) to extract multi-scale spatial features, and optimizes tumor contour modeling through boundary continuity loss and edge breakage compensation terms.

4. The tumor detection method based on MaskRCNN according to claim 1, characterized in that: In step S3, the nonlocal attention module generates an attention map by calculating the feature similarity between pixels within the candidate region and uses dot multiplication to achieve remote feature fusion, which is used to enhance the expression ability of complex pathological tissue structures such as central necrosis, ring enhancement, cystic degeneration, calcification or hemorrhage.

5. The tumor detection method based on MaskRcnn according to claim 4, characterized in that: Texture features extracted based on the gray-level co-occurrence matrix (GLCM) include directional entropy, energy, contrast, homogeneity, local variance, and correlation. Within the candidate tumor region, a 32×32 sliding window is used to extract statistics in four directions: 0°, 45°, 90°, and 135°. The average of the data in each direction is then used to construct a 24-dimensional texture feature vector.

6. The tumor detection method based on MaskRcnn according to claim 5, characterized in that: The structure-aware feature representation is achieved by concatenating the output feature map of the non-local attention module with the texture vector in the channel dimension, and then inputting it into a fusion network containing three convolutional modules. The convolutional modules consist of 3×3 convolutional kernels, batch normalization layers, and ReLU activation functions, and the final output is a 128-dimensional structured embedding feature.

7. The tumor detection method based on MaskRCNN according to claim 1, characterized in that: In step S4, the edge alignment loss function is based on applying the Sobel operator to extract edge images of the predicted mask and the real mask respectively, and calculating the difference in Dice coefficients between their edge images. The edge overlap is used as an indicator to participate in the training of the joint loss function.

8. The tumor detection method based on MaskRcnn according to claim 7, characterized in that: The structural consistency loss function measures the difference in texture statistical distribution between the predicted mask and the real mask on the texture feature vector generated based on the gray-level co-occurrence matrix GLCM by calculating the Jensen-Shannon divergence between the two, and is used to constrain the model's expression consistency of the internal structure of tumor tissue.

9. The tumor detection method based on MaskRcnn according to claim 1, characterized in that: In step S5, the structured inspection results are output in DICOM RT-STRUCT format, compatible with JSON and HL7 formats. The output fields include: (1) Boundary mask coordinates for each tumor instance, (2) Maximum radial length, (3) Classification confidence, (4) Texture label information, and (5) Model version number. The structured results can be interfaced with radiotherapy planning systems or PACS systems for preoperative auxiliary diagnosis and radiotherapy planning.

10. A tumor detection system based on MaskRcnn according to any one of claims 1-9, characterized in that, include: The image input module is used to acquire and preprocess medical image data; An instance segmentation module, connected to the image input module, is used to receive preprocessed images and identify candidate tumor regions based on the MaskRcnn network; The edge perception module is connected to the instance segmentation module. It enhances boundary features based on the Sobel edge response map and the convolutional block attention module CBAM, and feeds the results back to the instance segmentation module to optimize mask prediction. The structural modeling module, connected to the instance segmentation module, is used to apply a nonlocal attention module to the candidate region and combine the texture features extracted by the gray-level co-occurrence matrix (GLCM) to generate a structure-aware feature representation. The joint optimization module is connected to the edge perception module and the structural modeling module respectively, and is used to jointly train the classification, regression and segmentation tasks based on multiple loss terms. The results output module, connected to the joint optimization module, is used to generate detection results containing tumor mask, maximum radial length and structural feature information, and output them in DICOM RT-STRUCT format.

Images