Automatic tumor region detection method and system based on image segmentation

Abstract

The invention discloses a tumor area automatic detection method and system based on image segmentation. According to the method, continuous two-dimensional slices are extracted and preprocessed by obtaining a three-dimensional pathological tissue slice image sequence; an improved encoder-decoder architecture is adopted, an encoder adopts a three-branch asymmetric structure, a main branch extracts multi-scale features of a current slice, two auxiliary branches extract features of adjacent slices and are embedded into a CBAM attention mechanism, and three-dimensional context perception is enhanced. And the decoder fuses the deep and shallow layer features through up-sampling and jump connection to generate a tumor probability map. Subsequently, a segmentation result is optimized through adaptive threshold segmentation, morphological processing and an active contour model, an accurate tumor segmentation mask is output, and quantitative characteristic parameters are extracted. According to the method, the problems of fuzzy tumor boundary, insufficient feature extraction and difficult diagnosis in a traditional pathological image are effectively solved, and the accuracy and robustness of pathological diagnosis of early infiltration and micro infiltration of the tumor are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of pathological tissue slide image processing technology, and specifically relates to an automatic tumor region detection method and system based on image segmentation in pathological tissue slides. Background Technology

[0002] With the rapid development of pathological imaging technology, traditional 2D pathological image analysis is gradually moving towards more advanced non-destructive 3D pathological technology. This emerging technology not only improves the accuracy of tumor diagnosis, but also provides a new perspective for precision medicine and clinical decision-making.

[0003] Among existing automatic segmentation methods, deep learning-based image segmentation techniques have shown significant advantages. Encoder-decoder architectures (such as U-Net) have achieved good results in pathological image segmentation by fusing deep and shallow features through skip connections. However, these methods have obvious limitations when processing three-dimensional pathological images: First, most models are based on two-dimensional slice processing, making it difficult to effectively utilize the spatial context information between pathological tissue slices; second, the boundaries between tumors and surrounding tissues in pathological tissue slices are often blurred, it is difficult to identify whether the tumor has broken through the capsule or basement membrane, the contrast is low, and traditional network structures are insufficient in extracting subtle features; in addition, the detection effect of early malignant tumors or microinvasive malignant tumors is poor, and they are easily affected by noise.

[0004] In recent years, although improved methods such as 3D convolutional neural networks and attention mechanisms have emerged, the high computational complexity of 3D networks makes them difficult to apply clinically. Furthermore, existing attention mechanisms mostly focus on channel or spatial dimensions, and their modeling of multi-scale features and long-range dependencies remains insufficient. In particular, the diagnostic capabilities of existing methods in thyroid microcarcinoma, early-stage lung cancer, and early gastrointestinal cancers still need improvement.

[0005] Therefore, there is an urgent need to develop an automatic tumor region detection method that can effectively utilize three-dimensional contextual information, enhance feature expression capabilities, and has high computational efficiency, in order to assist doctors in making accurate diagnoses and treatment plans. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention aims to provide an automatic tumor region detection method and system based on image segmentation, which can effectively utilize three-dimensional contextual information, enhance feature expression capabilities, and has high automatic detection efficiency for tumor regions in pathological tissue sections.

[0007] In a first aspect, embodiments of this application provide an automatic tumor region detection method based on image segmentation, the method comprising: S1. Obtain the sequence of three-dimensional pathological tissue slice images to be detected, extract continuous two-dimensional pathological tissue slice images from them, and preprocess the pathological tissue slice images to obtain a standardized input image. S2. The current pathological tissue slice to be segmented and its preceding and following slices are input into the trained tumor region segmentation model, and the preliminary tumor region segmentation results in the pathological tissue slice are output. The tumor region segmentation model adopts an encoder-decoder architecture. S3. The decoder upsamples the fused features and fuses them with the shallow features extracted by the backbone network of the encoder through skip connections. Then, after convolution processing, a tumor region probability map with the same size as the input image is generated. S4. Post-process the probability map to generate the final tumor region segmentation mask in the pathological tissue slice, and complete the automatic detection of the tumor region.

[0008] Optionally, in one implementation of the first aspect of the present invention, step S1 involves acquiring a sequence of three-dimensional pathological tissue slice images to be detected, extracting continuous two-dimensional pathological tissue slice images from it, and preprocessing the pathological tissue slice images to obtain a standardized input image, including: Acquire image data containing three-dimensional pathological tissue slices of the target, wherein the pathological tissue slice image data includes computed tomography (CT) images or positron emission tomography (PET) images. Extract continuous two-dimensional pathological tissue slice images along the vertical axis, with the extraction interval set according to image resolution and clinical needs; The extracted two-dimensional pathological tissue slice images are preprocessed, including window width and window level adjustment, pixel value normalization, and image enhancement. The current pathological tissue slice to be processed is combined with its preceding and subsequent slices to form a multi-channel input image, which is used to preserve the spatial context information between slices.

[0009] Optionally, in one implementation of the first aspect of the present invention, the encoder adopts a three-branch asymmetric structure: The first branch is the main feature extraction branch, which uses a lightweight MobileNetV3 network structure as the backbone network to extract multi-scale features of the current pathological tissue slice. The second and third branches are auxiliary feature extraction branches, which are used to extract features of preceding and subsequent pathological tissue sections, respectively. Each convolutional layer is followed by a compound attention CBAM, which is used to adaptively adjust the weights of adjacent section features. The encoder also includes a multi-scale feature extraction module, which adopts a parallel multi-branch structure, including four parallel branches, which respectively use 1×1 convolution and three different dilation rates of dual-branch depthwise separable convolution TbsConv. The output features of each level of the first branch are concatenated and integrated with the output features of the same level of the second and third branches after being weighted by CBAM to form an enhanced feature map that integrates multi-scale spatial context information.

[0010] Optionally, in one implementation of the first aspect of the present invention, the decoder processing in step S3 includes: S3.1. Upsample the fused features output by the encoder and use bilinear interpolation to gradually restore the spatial resolution of the feature map to the original input size. S3.2. The upsampled deep semantic features are fused with the shallow detail features of the corresponding layer in the encoder backbone network through skip connections. The shallow detail features are first adjusted by 1×1 convolution to make the number of channels consistent with the number of channels of the deep semantic features. The fusion of deep and shallow features is achieved by feature concatenation or element-wise addition. S3.3. The fused features are processed by convolution, passing through: a 3×3 convolutional layer for feature refinement; a batch normalization layer to accelerate the training process and improve model stability; and a ReLU activation function to introduce non-linear transformation. S3.4 Repeat steps S3.1 to S3.3 to gradually restore the feature map resolution until the original input image size is reached; S3.5 Finally, a probability map of the tumor region is generated using a 1×1 convolutional layer and a Sigmoid activation function, where the value of each pixel represents the probability that the pixel's location belongs to the tumor region.

[0011] Optionally, in one implementation of the first aspect of the present invention, the calculation process of the dual-branch depthwise separable convolution used in the encoder includes: The input feature map is subjected to depthwise convolution operations through two parallel branches. The first branch uses a convolution kernel of the first size for depthwise convolution, and the second branch uses a convolution kernel of the second size for depthwise convolution. The first size and the second size are different. The output feature maps of the two branches are added and fused element by element; Features are further fused through pointwise convolution; The depthwise convolution operation involves performing convolution on each channel of the input feature map separately, without cross-channel information fusion; the pointwise convolution operation is used to adjust the number of channels in the feature map.

[0012] Optionally, in one implementation of the first aspect of the present invention, the composite attention CBAM includes a channel attention submodule and a position attention submodule: The channel attention submodule is calculated as follows: spatial context descriptors are extracted from the input feature map, the extracted descriptors are input into a multilayer perceptron with shared weights, the channel attention weights are output, and the channel attention weights are multiplied by the input feature map; The location attention submodule is calculated in the following way: the channel attention output is passed through a convolutional layer to generate multiple feature maps, a spatial attention map is calculated, the spatial attention map is multiplied with the feature map, and then fused with the original feature map through a scale parameter; The spatial attention map is obtained through matrix multiplication and normalization operations between feature maps, and the scale parameter is a learnable parameter.

[0013] Optionally, in one implementation of the first aspect of the present invention, the tumor region segmentation model is trained using a multi-task loss function, the multi-task loss function comprising: Boundary-aware loss function This is used to enhance the accuracy of tumor boundary localization, and its calculation formula is as follows: , in, and These represent the gradients of the true label and the predicted result, respectively. Represents the gradient of the input image. Total number of pixels For pixel index; Shape constraint loss function Based on the shape difference between the Hausdorff distance metric prediction results and the true labels: , Where A represents the set of boundary points of the model prediction results, B represents the set of boundary points of the true labels, and a and b represent a point in sets A and B, respectively. This represents the Euclidean distance between a and b. and Let f(x) represent the supremum and inf(x) of a set, respectively. Regional consistency loss function Used to maintain the connectivity and integrity of the tumor region: , in, Indicates a connected component index. This represents the total number of connected regions. For smoothing coefficients, , They represent the connected regions respectively. In the middle, pixels The true label value and the model prediction value; The final expression for the multi-task loss function is: , in, The weight coefficients for each loss term are used to determine the optimal weight combination through grid search.

[0014] Optionally, in one implementation of the first aspect of the present invention, the post-processing of the probability map in step S4 to generate the final tumor region segmentation mask in the pathological tissue section includes: Threshold segmentation is performed on the probability map of tumor regions in the pathological tissue sections, and an adaptive threshold algorithm is used to determine the segmentation threshold. Morphological operations are performed on the binarized segmentation results, including filling the internal holes of the region with morphological closing operations and eliminating false positive regions with morphological opening operations. Based on connected component analysis, noise regions are filtered out, and connected regions with an area greater than a preset threshold are retained. An active contour model based on level sets is used to refine the segmentation boundaries; Output the final tumor region segmentation mask after post-processing optimization, and calculate the morphological feature parameters of the tumor.

[0015] Optionally, in one implementation of the first aspect of the present invention, the method further includes a step of quantitative analysis of tumor characteristics in pathological tissue sections: Based on the final segmentation mask, the morphological characteristic parameters of the tumor are calculated, including tumor volume, maximum diameter, and sphericity. Extracting texture feature parameters of tumors, including statistical features based on gray-level co-occurrence matrix and frequency domain features based on wavelet transform; Generate a structured test report, including the spatial location information of the tumor, the probability of benign or malignant assessment, treatment response assessment indicators, and comparative analysis data with historical test results.

[0016] Secondly, embodiments of this application provide an automatic tumor region detection system based on pathological tissue slice image segmentation, applied to the automatic tumor region detection method based on pathological tissue slice image segmentation as described in the second aspect, the system comprising: The image acquisition and preprocessing module is used to acquire the sequence of three-dimensional pathological tissue slice images to be detected, extract continuous two-dimensional pathological tissue slice images from them, and preprocess the pathological tissue slice images to obtain a standardized input image. The feature extraction and fusion module is used to take the current slice to be segmented and its preceding and following slices as input, extract multi-scale features through the encoder and fuse them. The encoder adopts a three-branch asymmetric structure, including a main feature extraction branch, an auxiliary feature extraction branch and a multi-scale feature extraction module. The probability map generation module is used to upsample the fused features through the decoder and fuse them with the shallow features extracted by the encoder through skip connections. After convolution processing, a probability map of the tumor region with the same size as the input image is generated. The post-processing module is used to post-process the probability map to generate a tumor region segmentation mask in the final pathological tissue slice, thereby completing the automatic detection of the tumor region.

[0017] Thirdly, embodiments of this application provide an electronic device, including: processor; Memory used to store processor-executable instructions; The processor is configured to implement the automatic tumor region detection method based on segmentation in pathological tissue slice images as described in the first aspect when executing the instructions.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that instructs a device to perform the image segmentation-based automatic tumor region detection method as described in the first aspect.

[0019] This invention discloses an automatic tumor region detection method and system based on image segmentation. The method acquires a sequence of three-dimensional pathological tissue slice images and extracts continuous pathological tissue slice images. An improved encoder-decoder architecture is used for feature extraction and fusion. The encoder employs a three-branch asymmetric structure: the main branch extracts features of the current slice, and the auxiliary branch extracts features of adjacent slices and embeds an attention mechanism. A multi-scale feature extraction module enhances contextual awareness. The decoder fuses deep and shallow features through upsampling and skip connections to generate a tumor probability map. Subsequently, adaptive threshold segmentation, morphological processing, and an active contour model are used to optimize the segmentation results, ultimately outputting an accurate tumor segmentation mask and extracting quantitative feature parameters. This invention effectively solves the problems of blurred tumor boundaries and insufficient feature extraction in complex pathological tissue slice images using traditional methods, significantly improving the accuracy and robustness of tumor detection.

[0020] Beneficial effects:

[0021] 1. A three-branch asymmetric encoder structure is proposed. The main branch extracts the features of the current slice, and the auxiliary branch extracts the features of adjacent slices. This effectively utilizes the three-dimensional spatial context information and significantly improves the recognition accuracy of tumor boundaries.

[0022] 2. A multi-scale feature extraction module is introduced, which combines bi-branch depthwise separable convolution and dual attention mechanism to enhance the model's ability to perceive multi-scale tumor features and effectively improve the detection effect of small tumors.

[0023] 3. An improved combination of loss functions is adopted, which integrates boundary awareness, shape constraint and region consistency loss, effectively improving the accuracy of segmentation boundaries and the connectivity of regions.

[0024] 4. The accuracy and reliability of the segmentation results are further improved through adaptive threshold segmentation, morphological operations and active contour model optimization in the post-processing workflow.

[0025] 5. While maintaining high accuracy, the overall solution significantly reduces computational complexity through lightweight network design and efficient module optimization, and has good clinical application value. Attached Figure Description

[0026] Figure 1 This is a schematic flowchart of an automatic tumor region detection method based on image segmentation provided in an embodiment of this application.

[0027] Figure 2 This is a structural diagram of an encoder provided in an embodiment of this application.

[0028] Figure 3 This is a decoder structure diagram provided in one embodiment of this application.

[0029] Figure 4 This is a diagram illustrating the architecture of an automatic tumor region detection system based on image segmentation, provided in one embodiment of this application.

[0030] Figure 5 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0032] It should be noted that in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0033] It should be noted that in the embodiments of this application, the terms "first," "second," etc., are used only for descriptive purposes and should not be construed as indicating or implying relative importance, nor as indicating or implying order. Features defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, terms such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiments or designs described as exemplary or for example in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present related concepts in a specific manner.

[0034] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] Example 1

[0036] Figure 1 This is a schematic flowchart of an automatic tumor region detection method based on image segmentation provided in an embodiment of this application. Figure 1 As shown, an automatic tumor region detection method based on image segmentation includes: S1. Obtain the sequence of three-dimensional pathological tissue slice images to be detected, extract continuous two-dimensional pathological tissue slice images from it, and preprocess the pathological tissue slice images to obtain standardized input images. This step is a key data engineering step, aiming to transform and enhance the raw, heterogeneous three-dimensional medical data into a high-quality, standardized two-dimensional data package rich in spatial context information, laying a solid foundation for the high-precision and robust segmentation of subsequent deep learning models.

[0037] Specifically, in this embodiment, step S1 involves acquiring a sequence of three-dimensional pathological tissue slice images to be detected, extracting continuous two-dimensional pathological tissue slice images from it, and preprocessing the pathological tissue slice images to obtain a standardized input image, including: Acquire three-dimensional pathological tissue image data containing the target tissue, wherein the pathological tissue slice image data includes computed tomography (CT) images or positron emission tomography (PET) images.

[0038] Extract continuous two-dimensional slice images along the vertical axis, with the extraction interval set according to image resolution and clinical needs. Three-dimensional volume data (such as DICOM sequences) are sliced ​​along their vertical axis (usually the Z-axis, i.e., the patient's head-to-toe direction) to generate a series of continuous two-dimensional slice images.

[0039] The model transforms the data it needs to process from three-dimensional space to a two-dimensional plane. This cleverly avoids the problems of huge computational cost and high memory consumption of three-dimensional convolution, enabling the use of mature and efficient two-dimensional deep learning model architectures (such as U-Net, DeepLab, etc.) to process three-dimensional medical data, greatly improving the feasibility and computational efficiency of the algorithm.

[0040] The extraction interval (i.e., step size) is not fixed but dynamically set based on two factors: Image resolution: For high-resolution images with thin slices and high isotropy, a smaller interval can be used to retain more details; for images with thicker slices, the interval can be appropriately increased to avoid redundancy. Clinical needs: For screening or preoperative planning scenarios requiring detailed analysis, a smaller interval is used; for quick browsing or preliminary screening, a larger interval can be used to improve processing speed. This achieves an optimal balance between data integrity, computational efficiency, and clinical applicability. This adaptability ensures that the method can meet clinical tasks with varying accuracy requirements while making rational use of computational resources.

[0041] The extracted 2D slice images are preprocessed, including window width and level adjustment, pixel value normalization, and image enhancement. Based on the grayscale characteristics of the target tissue and tumor, a suitable display range (window width) and center value (window level) are selected. The dynamic range of the original pathological digital slice images is large (pixel value range is typically 0-65535 or 0-255), but the human eye and computer models are only sensitive to a portion of this range. This operation significantly enhances the contrast between the target tissue (such as the tumor area) and the surrounding background by compressing the display range, suppressing irrelevant tissue information (such as blank background areas, staining artifacts, and non-target tissue structures), making key features more prominent and easier for the model to recognize. For example, window width and level adjustment is set as follows: the window width is set to 300-500 HU, and the window level is set to 30-50 HU based on the grayscale characteristics of the target organ and tumor; pixel value normalization is performed by linearly transforming the image pixel values ​​to the [0,1] interval; and image enhancement is achieved by using histogram equalization or contrast-limited adaptive histogram equalization to enhance image contrast.

[0042] Through a series of professional image processing techniques, the most critical information for segmentation tasks is extracted and enhanced from the raw and redundant pathological image data, while the data is standardized into a form most suitable for deep learning model processing.

[0043] The current slice to be processed is combined with its preceding and subsequent slices to form a multi-channel input image, which preserves the spatial context information between slices. Three adjacent 2D slices along the vertical axis (current slice, preceding slice, and subsequent slice) are stacked along the channel dimension to form a multi-channel image (e.g., three-channel). Within the framework of a 2D convolutional network, a third dimension (spatial context) is cleverly introduced. This allows the model to simultaneously "see" the information of its adjacent slices while processing the current slice, thereby understanding the continuous morphology, structural changes, and spatial relationships of the target organization in 3D space. 3D information is simulated through multi-channel input, rather than directly using 3D convolution. While introducing valuable 3D context information, this avoids the exponentially increasing computational complexity and memory consumption of 3D convolution operations. This is an extremely efficient computational strategy that makes it possible to achieve high-precision quasi-3D segmentation with limited hardware resources, balancing performance and efficiency, and enhancing the practicality and deployability of the solution.

[0044] S2. The current slice to be segmented and its preceding and following slices are input into the trained tumor region segmentation model, and the preliminary tumor region segmentation result is output. The tumor region segmentation model adopts an encoder-decoder architecture.

[0045] Specifically, in this embodiment, a multi-channel image composed of adjacent slices is input into a trained segmentation model through information fusion and feature extraction. The model's encoder automatically learns and fuses deep features from the current layer and its neighboring layers. The model can identify and extract spatial contextual features and semantic features related to the tumor region across multiple slices, which is the foundation for achieving high-precision segmentation. The model's decoder upsamples and refines the deep, abstract features extracted by the encoder. The spatial resolution of the feature map is gradually restored to the original input size, and finally, a probability value is assigned to each pixel in the input image, generating a probability map of the same size as the input. This probability map represents the "preliminary tumor region segmentation result," where the value of each pixel represents the confidence that the location belongs to the tumor region.

[0046] Employing an encoder-decoder architecture (such as U-Net and its variants) fully leverages the inherent advantages of this architecture. The encoder effectively captures the global context and high-level semantic information of the image through layer-by-layer downsampling and convolution. The decoder fuses the low-level spatial details from the encoder with the high-level semantic information from the decoder through upsampling and skip connections, thereby achieving precise pixel-level localization and ensuring the accuracy of segmentation boundaries.

[0047] Specifically, Figure 2 An encoder structure diagram provided in one embodiment of this application is shown below. Figure 2 As shown, the encoder adopts a three-branch asymmetric structure: the first branch is the main feature extraction branch, which uses a lightweight MobileNetV3 network structure as the backbone network to extract multi-scale features of the current slice; the second and third branches are auxiliary feature extraction branches, which are used to extract features of the preceding and following slices, respectively. Each convolutional layer is followed by a composite attention CBAM to adaptively adjust the weights of features of adjacent slices. The encoder also includes a multi-scale feature extraction module, which adopts a parallel multi-branch structure, including four parallel branches, which respectively use 1×1 convolution and three different dilation rates of bi-branch depthwise separable convolutions TbsConv; the output features of each level of the first branch are concatenated and integrated with the output features of the same level of the second and third branches weighted by CBAM to form an enhanced feature map that integrates multi-scale spatial context information.

[0048] Specifically, the structure of the encoder can be as follows: S2.1 The encoder's three branches adopt an asymmetric structure design: the first branch contains 5 convolutional modules, using a lightweight RegNet or MobileNetV3 network structure as the backbone network; the second and third branches each contain 4 convolutional modules, using the same convolutional layer settings as the UNet encoder. MobileNetV3 is a lightweight convolutional neural network (CNN) architecture designed specifically for mobile and embedded devices (such as mobile phones, drones, autonomous driving sensors, etc.). As the backbone network for feature extraction, it provides powerful feature extraction capabilities for the core segmentation task, while ensuring the efficiency and practicality of the entire system.

[0049] S2.2 Each convolutional module in the second and third branches is followed by a composite attention module (CBAM). The computation process of this module includes: performing global max pooling and global average pooling on the input feature map to obtain two 1×1×C feature vectors; concatenating the two feature vectors along the channel dimension to form a 1×2×C tensor; processing the concatenated tensor using a one-dimensional convolutional layer with a kernel size of 2×k, where k is adaptively adjusted according to the number of channels C; generating channel attention weights using the Sigmoid activation function; and multiplying the attention weights by the input feature map channel by channel to output a weighted feature map.

[0050] S2.3 The multi-scale feature extraction module includes four parallel branches: the first branch uses 1×1 convolution; the second to fourth branches use two-branch depthwise separable convolutions TbsConv with dilation rates of 6, 12 and 18 respectively; each TbsConv branch contains two parallel branches, which use 3×3 and 5×5 convolution kernels for depthwise convolution respectively, and fuse features by element-wise addition; the output feature maps of each branch are processed by a dual attention module, which sequentially includes a channel attention submodule and a position attention submodule.

[0051] S2.4 Feature fusion adopts a hierarchical concatenation method: the output layer of each convolution module in the first branch is concatenated with the output features of the same layer in the second and third branches after CBAM weighting; the channel number of the concatenated features is integrated using 1×1 convolution to keep the number of output channels consistent with the output of the first branch at that level; the integrated features are input to the multi-scale feature extraction module for further processing.

[0052] The calculation process of the dual-branch depthwise separable convolution used in the encoder includes: the input feature map is subjected to depthwise convolution operations through two parallel branches, the first branch uses a convolution kernel of a first size for depthwise convolution, and the second branch uses a convolution kernel of a second size for depthwise convolution, wherein the first size and the second size are different; the output feature maps of the two branches are added and fused element-wise; and the features are further fused through pointwise convolution; wherein the depthwise convolution operation is performed on each channel of the input feature map separately, without cross-channel information fusion; and the pointwise convolution operation is used to adjust the number of channels of the feature map.

[0053] Specifically, the computation process of the dual-branch depthwise separable convolution TbsConv used in the encoder includes: First, the input feature map is subjected to depthwise convolution operations through two parallel branches: The first branch uses a 3×3 convolution kernel for depthwise convolution: The second branch uses a 5×5 convolution kernel for depthwise convolution: Then, the output feature maps of the two branches are added and fused element by element: Finally, features are further fused through pointwise convolution: ,in, For the input feature map, This indicates a batch normalization operation. For activation function, This indicates an element-wise addition operation. This represents pointwise convolution, i.e. Convolution operations are used to adjust the number of channels in a feature map. , This indicates a depthwise convolution operation, which means performing convolution operations on each channel of the input feature map separately, without fusing information across channels.

[0054] TbsConv functions like a dual-lens camera system (such as a wide-angle lens plus a telephoto lens). Two branches simultaneously capture the same scene, each capturing information from different ranges of the field of view. Element-wise addition fuses the information from the two images into a richer one. Pointwise convolution then performs final color and detail adjustments on this fused image, making it more suitable for subsequent processing. By using convolutional kernels of different sizes in parallel, it captures both detailed and global contextual information, overcoming the limitations of single-path, single-scale processing and significantly improving the model's feature extraction capabilities for multi-scale targets (especially tumors of varying sizes).

[0055] This structure combines the advantages of depthwise convolution and pointwise convolution, enhancing the model's expressive power through multi-scale feature extraction and feature fusion. This structure aligns with the principle of depthwise separable convolution, which reduces computation and parameter count while maintaining feature extraction capabilities through a combination of depthwise and pointwise convolution.

[0056] The composite attention CBAM comprises a channel attention submodule and a position attention submodule. The channel attention submodule is calculated as follows: spatial context descriptors are extracted from the input feature map, the extracted descriptors are input into a multilayer perceptron with shared weights, the channel attention weights are output, and the channel attention weights are multiplied by the input feature map. The position attention submodule is calculated as follows: the channel attention output is passed through a convolutional layer to generate multiple feature maps, a spatial attention map is calculated, the spatial attention map is multiplied by the feature maps, and fused with the original feature map using a scale parameter. The spatial attention map is obtained through matrix multiplication and normalization operations between feature maps, and the scale parameter is a learnable parameter.

[0057] Specifically, the Convolutional Block Attention Module (CBAM) includes a channel attention submodule and a position attention submodule. The calculation process of the channel attention submodule is as follows: given an input feature map... First, two spatial context descriptors are obtained through global average pooling and global max pooling; then, the two descriptors are input into a two-layer perceptron with shared weights, and the output is the channel attention weights. Multiply the channel attention weights by the input feature map: The calculation process of the position attention submodule is as follows: output the channel attention. Three feature maps are generated through a convolutional layer. Computational spatial attention map: Spatial attention map and feature map Multiplication: ; Fusing with the original feature map using scale parameters: ,in These are learnable parameters.

[0058] The Channel Attention Sub-module takes as input a feature map F (size: H×W×C) and outputs channel attention weights Mc (size: 1×1×C). The steps include: 1. Global Pooling: Compressing the spatial dimension H×W to obtain two different spatial context descriptors: Global Average Pooling: (Size: 1×1×C), Global max pooling: (Size: 1×1×C). Average pooling extracts overall features, while max pooling extracts salient features; the two are complementary. 2. Shared Perceptron: Two descriptors are input into a shared two-layer perceptron (MLP). First layer: Dimensionality reduction (e.g., reducing to C / r neurons, where r is the reduction ratio). Second layer: Dimensionality increase (restoring to C neurons). Output: Two processed vectors. and 3. Generate weights: Add the two vectors element-wise, then pass them through the Sigmoid activation function to obtain the final channel attention weights. . 4. Reweighting: Reweighting With input feature map Multiplying by channel one by one yields refined features. . .

[0059] The input to the Spatial Attention Sub-module is the features after channel attention. (Size: H×W×C), its output is: spatial attention weights. (Dimensions: H×W×1). The steps include: 1. Channel pooling: Compressing along the channel dimension to obtain two different channel descriptors: Average pooling: (Size: H×W×1), Max pooling: (Size: H×W×1). The average response and maximum response together identify important spatial locations. 2. Concatenation and Convolution: The two descriptors are concatenated (size: H×W×2) and then fused through a standard convolutional layer (usually a 7x7 convolution) to generate a spatial attention map. 3. Weight Generation: The convolution result is activated using the Sigmoid activation function to obtain the spatial attention weights. . 4. Reweighting: Reweighting the weights With input feature map Multiplying by spatial position yields the final refined feature. . .

[0060] CBAM uses channel attention to determine which features to prioritize and then spatial attention to determine which location to prioritize, allowing the model to focus more precisely on key information. In the patent, it is used to refine multi-scale features and improve segmentation accuracy.

[0061] S3. The decoder upsamples the fused features and fuses them with the shallow features extracted by the encoder's backbone network via skip connections. This is then processed by convolution to generate a tumor region probability map of the same size as the input image. The decoder progressively enlarges the size of the compressed low-resolution high-level feature map output by the encoder through upsampling operations (such as transposed convolution and bilinear interpolation). This reconstructs and maps abstract features rich in semantic information but lacking spatial detail back to the original input image size, preparing the space for generating pixel-level prediction maps.

[0062] Specifically, Figure 3 This is a diagram illustrating the decoder structure according to an embodiment of this application. Figure 3 As shown, in this embodiment, the decoder processing in step S3 includes: 3.1 Upsampling the fused features output by the encoder, and using bilinear interpolation to gradually restore the spatial resolution of the feature map to the original input size; S3.2 Fusing the upsampled deep semantic features with the corresponding shallow detail features in the encoder backbone network through skip connections, wherein: the shallow detail features are first adjusted for the number of channels by 1×1 convolution to make them consistent with the number of channels of the deep semantic features; the fusion of deep and shallow features is achieved by feature concatenation or element-wise addition. S3.3. Perform convolution processing on the fused features, sequentially passing them through: a 3×3 convolutional layer for feature refinement; a batch normalization layer to accelerate the training process and improve model stability; and a ReLU activation function to introduce a non-linear transformation. S3.4. Repeat steps S3.1 to S3.3 to gradually restore the feature map resolution until it reaches the original input image size. S3.5. Finally, generate a probability map of the tumor region using a 1×1 convolutional layer and a Sigmoid activation function, where the value of each pixel represents the probability that the pixel's location belongs to the tumor region.

[0063] The tumor region segmentation model is trained using a multi-task loss function, which includes: Boundary-aware loss function This is used to enhance the accuracy of tumor boundary localization, and its calculation formula is as follows: , in, and These represent the gradients of the true label and the predicted result, respectively. Represents the gradient of the input image. Total number of pixels This function uses pixel indices. It considers the ground truth boundary gradient, predicted boundary gradient, and original image gradient simultaneously. It encourages the predicted boundary to align with the ground truth boundary at image structural edges (high gradient regions). The original image gradient is used as a weighting term to ensure that the model penalizes the prediction error more heavily in areas where the image itself has clear boundaries. This results in segmentation results with clear boundaries and accurate contours, reducing boundary blurring and jagged edges.

[0064] Shape constraint loss function Based on the shape difference between the Hausdorff distance metric prediction results and the true labels: , Where A represents the set of boundary points of the model prediction results, B represents the set of boundary points of the true labels, and a and b represent a point in sets A and B, respectively. This represents the Euclidean distance between a and b. and These represent the supremum and infimum of a set, respectively. Based on the Hausdorff distance, a rigorous metric for measuring the maximum and minimum distance between two point sets, it focuses on the points where the predicted result least matches the true label (i.e., the maximum distance). By minimizing this worst-case error, it forces the model's predicted shape to more closely approximate the true shape overall. It is particularly suitable for improving the overall shape of segmentation results and preventing strange, anatomically incompatible segmentation errors (such as overly distorted or fragmented predicted tumor shapes).

[0065] Regional consistency loss function Used to maintain the connectivity and integrity of the tumor region: , in, Indicates a connected component index. This represents the total number of connected regions. For smoothing coefficients, , They represent the connected regions respectively. In the middle, pixels The function calculates the Dice coefficient between the true and predicted tumor regions. Instead of being pixel-based, it's based on connected regions. It computes the Dice coefficient between each predicted tumor connected region and the true tumor connected region. By penalizing predicted regions that don't correctly overlap with the true region, it encourages the model to output a complete, continuous segmentation mask, rather than multiple scattered patches. This effectively reduces false positives, fragmentation, and internal holes, resulting in a segmented tumor region that appears as a single, cohesive unit.

[0066] The final expression for the multi-task loss function is: , in, The optimal weight combination is determined by grid search, assigning weight coefficients to each loss term. Grid search or other hyperparameter optimization methods are used to find an optimal weight combination for each loss term. This ensures that during training, the model does not become overly biased towards any single optimization objective (e.g., focusing only on boundaries while ignoring shape), but rather achieves optimal performance simultaneously and in a balanced manner across these three key metrics.

[0067] The composite loss function does not simply measure the difference between the predicted and the true values, but rather guides the model learning precisely from three different and complementary dimensions, ensuring that the model not only makes the correct predictions, but also makes good predictions.

[0068] S4. Post-process the probability map to generate the final tumor region segmentation mask in the pathological tissue slice, completing the automatic detection of the tumor region in the pathological tissue slice, including: thresholding the tumor region probability map and determining the segmentation threshold using an adaptive thresholding algorithm; performing morphological operations on the binarized segmentation results, including filling internal holes in the region using morphological closing operations and eliminating false positive regions using morphological opening operations; filtering noise regions based on connected component analysis and retaining connected regions with an area greater than a preset threshold; finely adjusting the segmentation boundary using an active contour model based on level sets; outputting the final tumor region segmentation mask after post-processing optimization, and calculating the morphological feature parameters of the tumor.

[0069] Specifically, in this embodiment, step S4 includes the following post-processing procedures: S4.1. Threshold segmentation is performed on the probability map of the tumor region in the pathological tissue section. An adaptive threshold algorithm is used to determine the optimal segmentation threshold, and the calculation formula is as follows: , in, The mean of the probability plot is... The standard deviation of the probability plot. The adjustment factor has a range of 0.5-2.0.

[0070] S4.2 Perform morphological operations on the binarized segmentation results, including: using morphological closing operations to fill the holes inside the tumor region, with the structuring element size adaptively selected according to the tumor size; using morphological opening operations to eliminate small false positive regions, with the structuring element size set to 3×3 to 7×7 pixels.

[0071] S4.3. Based on connected component analysis, filter noise regions and retain connected regions with an area greater than a preset threshold. The area threshold is determined based on prior knowledge of image resolution and tumor size, and the calculation formula is as follows: , in, This is a priori estimate of the minimum radius of the tumor.

[0072] S4.4. An active contour model based on level sets is used to refine the segmentation boundary. Its energy function is defined as: , Where φ is the level set function, g is the edge indicator function, μ, λ, and ν are weight parameters, δ is the Dirac function, and H is the Heaviside function.

[0073] S4.5 Output the final tumor region segmentation mask after post-processing optimization, and calculate the morphological feature parameters of the tumor, including tumor area, perimeter, roundness and irregularity index.

[0074] The method also includes a tumor feature quantification analysis step: based on the final segmentation mask, calculate the morphological feature parameters of the tumor, including tumor volume, maximum diameter, and sphericity; extract the texture feature parameters of the tumor, including statistical features based on the gray-level co-occurrence matrix and frequency domain features based on wavelet transform; generate a structured detection report, including the spatial location information of the tumor, the probability of benign or malignant assessment, treatment response assessment indicators, and comparative analysis data with historical examination results.

[0075] Specifically, this includes: S5.1, calculating the morphological feature parameters of the tumor based on the final segmentation mask, including: tumor volume: ,in Voxel volume The probability that a voxel belongs to a tumor; maximum tumor diameter: the maximum diameter of the tumor measured on three orthogonal planes; tumor sphericity: Where A is the tumor surface area; S5.2, extract the texture feature parameters of the tumor, including: gray-level co-occurrence matrix features: contrast, correlation, energy, homogeneity; wavelet transform features: extract the energy distribution features of the tumor region at different decomposition levels; S5.3, generate a structured detection report, including: spatial coordinates of the tumor; assessment of the probability of benign or malignant tumor; treatment response assessment indicators; comparative analysis data with historical examination results.

[0076] Example 2 like Figure 4 As shown, this application provides an architecture diagram of an automatic tumor region detection system based on image segmentation, which is applied to the automatic tumor region detection system based on image segmentation as described in Embodiment 1. The system includes an image acquisition and preprocessing module 11, a feature extraction and fusion module 12, a probability map generation module 13, and a post-processing module 14.

[0077] The image acquisition and preprocessing module 11 is used to acquire the sequence of three-dimensional pathological tissue slice images to be detected, extract continuous two-dimensional pathological tissue slice images from them, and preprocess the pathological tissue slice images to obtain a standardized input image.

[0078] The feature extraction and fusion module 12 is used to take the current slice to be segmented and its preceding and following slices as input, extract multi-scale features through the encoder and fuse them. The encoder adopts a three-branch asymmetric structure, including a main feature extraction branch, an auxiliary feature extraction branch and a multi-scale feature extraction module.

[0079] The probability map generation module 13 is used to upsample the fused features through the decoder and fuse them with the shallow features extracted by the encoder through skip connections. After convolution processing, a probability map of the tumor region with the same size as the input image is generated.

[0080] The post-processing module 14 is used to post-process the probability map to generate the final tumor region segmentation mask and complete the automatic detection of the tumor region.

[0081] Figure 5 This is an electronic device provided in one embodiment of this application. For example... Figure 5 As shown, the electronic device includes at least the following components: processor 101 and memory 100, communication interface 103, and bus 102.

[0082] In this embodiment of the application, memory 100 is used to store executable instructions of processor 101, which, when configured to execute instructions, implements the method as described in the first aspect.

[0083] In embodiments of this application, a computer-readable storage medium includes instructions that instruct a device to perform the method as described in the first aspect. For example, the instructions instruct the device to perform... Figure 1 The method is shown in the process steps.

[0084] In one embodiment of this application, the program operating in the electronic device may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). Information processed by these systems is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (FlashROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0085] It should be noted that a portion of the electronic device described above can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0086] It should be noted that the computer mentioned here refers to a computer built into an electronic device, employing hardware including an operating system and peripheral devices. Furthermore, computer-readable recording media refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage systems such as hard drives built into the computer.

[0087] Furthermore, computer-readable recording media can include: media that dynamically stores programs for short periods of time, such as communication lines used when transmitting programs via networks like the Internet or communication lines like telephone lines; and media that store programs for fixed periods of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining them with programs already recorded in the computer.

[0088] Furthermore, the electronic device in the above embodiments can also be implemented as an assembly (system group) composed of multiple systems. Each system constituting the system group can possess some or all of the functions or functional blocks of the electronic device in the above embodiments. As a system group, it is sufficient to have all the functions or functional blocks of the electronic device.

[0089] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. An automatic tumor region detection method based on image segmentation, the method comprising: S1. Obtain the sequence of three-dimensional pathological tissue slice images to be detected, extract continuous two-dimensional slice images from them, and preprocess the slice images to obtain a standardized input image. S2. The current slice to be segmented and its preceding and following slices are input into the trained tumor region segmentation model, and the preliminary tumor region segmentation result is output. The tumor region segmentation model adopts an encoder-decoder architecture. S3. The decoder upsamples the fused features and fuses them with the shallow features extracted by the backbone network of the encoder through skip connections. Then, after convolution processing, a tumor region probability map with the same size as the input image is generated. S4. Post-process the probability map to generate the final tumor region segmentation mask, and complete the automatic detection of the tumor region.

2. The automatic tumor region detection method based on image segmentation according to claim 1, characterized in that, In step S1, a sequence of three-dimensional pathological tissue slice images to be detected is obtained, continuous two-dimensional slice images are extracted from it, and the slice images are preprocessed to obtain a standardized input image, including: Acquire three-dimensional pathological tissue slice image data containing the target pathological tissue, wherein the pathological tissue slice image data includes computed tomography images or positron emission tomography images. Extract continuous two-dimensional pathological tissue slice images along the vertical axis, with the extraction interval set according to image resolution and clinical needs; The extracted two-dimensional pathological tissue slice images are preprocessed, including window width and window level adjustment, pixel value normalization, and image enhancement. The current pathological tissue slice to be processed is combined with its preceding and subsequent pathological tissue slices to form a multi-channel input image, which is used to preserve the spatial context information between pathological tissue slices.

3. The automatic tumor region detection method based on image segmentation according to claim 1, characterized in that, The encoder adopts a three-branch asymmetric structure: The first branch is the main feature extraction branch, which uses a lightweight MobileNetV3 network structure as the backbone network to extract multi-scale features of the current pathological tissue slice. The second and third branches are auxiliary feature extraction branches, which are used to extract features of preceding and subsequent pathological tissue sections, respectively. Each convolutional layer is followed by a compound attention CBAM, which is used to adaptively adjust the weights of adjacent pathological tissue section features. The encoder also includes a multi-scale feature extraction module, which adopts a parallel multi-branch structure, including four parallel branches, which respectively use 1×1 convolution and three different dilation rates of dual-branch depthwise separable convolution TbsConv. The output features of each level of the first branch are concatenated and integrated with the output features of the same level of the second and third branches after being weighted by CBAM to form an enhanced feature map that integrates multi-scale spatial context information.

4. The automatic tumor region detection method based on image segmentation according to claim 3, characterized in that, The decoder processing in step S3 includes: S3.

1. Upsample the fused features output by the encoder and use bilinear interpolation to gradually restore the spatial resolution of the feature map to the original input size. S3.

2. The upsampled deep semantic features are fused with the shallow detail features of the corresponding layer in the encoder backbone network through skip connections. The shallow detail features are first adjusted by 1×1 convolution to make the number of channels consistent with the number of channels of the deep semantic features. The fusion of deep and shallow features is achieved by feature concatenation or element-wise addition. S3.

3. The fused features are processed by convolution, passing through: a 3×3 convolutional layer for feature refinement; a batch normalization layer to accelerate the training process and improve model stability; and a ReLU activation function to introduce non-linear transformation. S3.4 Repeat steps S3.1 to S3.3 to gradually restore the feature map resolution until the original input image size is reached; S3.5 Finally, a probability map of the tumor region in the pathological tissue section is generated by using a 1×1 convolutional layer and a Sigmoid activation function, where the value of each pixel represents the probability that the position of that pixel belongs to the tumor region.

5. The automatic tumor region detection method based on image segmentation according to claim 3, characterized in that, The calculation process for the dual-branch depthwise separable convolution used in the encoder includes: The input feature map is subjected to depthwise convolution operations through two parallel branches. The first branch uses a convolution kernel of the first size for depthwise convolution, and the second branch uses a convolution kernel of the second size for depthwise convolution. The first size and the second size are different. The output feature maps of the two branches are added and fused element by element; Features are further fused through pointwise convolution; The depthwise convolution operation involves performing convolution on each channel of the input feature map separately, without cross-channel information fusion; the pointwise convolution operation is used to adjust the number of channels in the feature map.

6. The automatic tumor region detection method based on image segmentation according to claim 3, characterized in that, The composite attention CBAM includes a channel attention submodule and a position attention submodule: The channel attention submodule is calculated as follows: spatial context descriptors are extracted from the input feature map, the extracted descriptors are input into a multilayer perceptron with shared weights, the channel attention weights are output, and the channel attention weights are multiplied by the input feature map; The location attention submodule is calculated in the following way: the channel attention output is passed through a convolutional layer to generate multiple feature maps, a spatial attention map is calculated, the spatial attention map is multiplied with the feature map, and then fused with the original feature map through a scale parameter; The spatial attention map is obtained through matrix multiplication and normalization operations between feature maps, and the scale parameter is a learnable parameter.

7. The automatic tumor region detection method based on image segmentation according to claim 1, characterized in that, The tumor region segmentation model in the pathological tissue sections is trained using a multi-task loss function, which includes: Boundary-aware loss function This is used to enhance the accuracy of tumor boundary localization, and its calculation formula is as follows: ， in, and These represent the gradients of the true label and the predicted result, respectively. Represents the gradient of the input image. Total number of pixels For pixel index; Shape constraint loss function Based on the shape difference between the Hausdorff distance metric prediction results and the true labels: ， Where A represents the set of boundary points of the model prediction results, B represents the set of boundary points of the true labels, and a and b represent a point in sets A and B, respectively. This represents the Euclidean distance between a and b. and Let f(x) represent the supremum and inf(x) of a set, respectively. Regional consistency loss function Used to maintain the connectivity and integrity of the tumor region: ， in, Indicates a connected component index. This represents the total number of connected regions. For smoothing coefficients, , They represent the connected regions respectively. In the middle, pixels The true label value and the model prediction value; The final expression for the multi-task loss function is: ， in, The weight coefficients for each loss term are used to determine the optimal weight combination through grid search.

8. The automatic tumor region detection method based on image segmentation according to claim 1, characterized in that, The post-processing of the probability map in step S4 to generate the final tumor region segmentation mask in the pathological tissue section includes: The probability map of the tumor region is segmented using a threshold method, and an adaptive threshold algorithm is used to determine the segmentation threshold. Morphological operations are performed on the binarized segmentation results, including filling the internal holes of the region with morphological closing operations and eliminating false positive regions with morphological opening operations. Based on connected component analysis, noise regions are filtered out, and connected regions with an area greater than a preset threshold are retained. An active contour model based on level sets is used to refine the segmentation boundaries; Output the final tumor region segmentation mask after post-processing optimization, and calculate the morphological feature parameters of the tumor.

9. The automatic tumor region detection method based on image segmentation according to claim 8, characterized in that, The method also includes a step of quantitative analysis of tumor characteristics in pathological tissue sections: Based on the final segmentation mask, the morphological characteristic parameters of the tumor are calculated, including tumor volume, maximum diameter, and sphericity. Extracting texture feature parameters of tumors, including statistical features based on gray-level co-occurrence matrix and frequency domain features based on wavelet transform; Generate a structured test report, including the spatial location information of the tumor, the probability of benign or malignant assessment, treatment response assessment indicators, and comparative analysis data with historical test results.

10. An automatic tumor region detection system based on image segmentation, applied to the automatic tumor region detection method based on image segmentation as described in any one of claims 1 to 9, characterized in that, The system includes: The image acquisition and preprocessing module is used to acquire the sequence of three-dimensional pathological tissue slice images to be detected, extract continuous two-dimensional pathological tissue slice images from them, and preprocess the slice images to obtain a standardized input image. The feature extraction and fusion module is used to take the current pathological tissue slice to be segmented and its preceding and subsequent pathological tissue slices as input, extract multi-scale features through the encoder and fuse them. The encoder adopts a three-branch asymmetric structure, including a main feature extraction branch, an auxiliary feature extraction branch and a multi-scale feature extraction module. The probability map generation module is used to upsample the fused features through the decoder and fuse them with the shallow features extracted by the encoder through skip connections. After convolution processing, a probability map of the tumor region with the same size as the input image is generated. The post-processing module is used to post-process the probability map to generate a tumor region segmentation mask in the final pathological tissue slice, thereby completing the automatic detection of the tumor region.

Images