A PET image segmentation method, system, computer device and storage medium

Abstract

The application provides a PET image segmentation method and system, computer equipment and a storage medium, and belongs to the medical image field.The method comprises the following steps: extracting multi-scale features and omics features of a PET image; processing the multi-scale features to obtain a three-dimensional weight vector, performing strategy fusion on the multi-scale and omics features to obtain three groups of strategy fusion images, weighting the three groups of strategy fusion images by using the three-dimensional weight vector to obtain adaptive fusion features, extracting boundary feature maps of different scales of the adaptive fusion features by using different convolution kernels, weighting and fusing the three feature maps to obtain multi-scale boundary enhancement features, calculating the multi-head attention of the multi-scale boundary enhancement features, and finally outputting clear and reliable multi-scale segmentation mask images; and the omics features are used to break through the generalization bottleneck of a fixed fusion strategy, and the PET image boundary blur problem is solved in a targeted manner, and the generated multi-scale segmentation mask images directly show segmentation decision basis and enhance clinical reliability.

Description

Technical Field

[0001] This invention belongs to the field of medical imaging, specifically relating to a PET image segmentation method, system, computer equipment, and storage medium. Background Technology

[0002] With an aging population and changing lifestyles, the incidence of complex diseases such as cancer, neurodegenerative diseases, and cardiovascular diseases is rising year by year, becoming a major global health threat. These diseases are characterized by pathological complexity, difficulty in early diagnosis, inaccurate prognostic assessment, and the need for personalized treatment. Disease progression involves the interaction of multiple molecular pathways and cell types, leading to high heterogeneity in lesion morphology, function, and metabolic characteristics. Traditional diagnostic methods often only detect the disease when it has progressed to a significant stage, missing the optimal treatment window. Existing prognostic assessment models are mainly based on clinical indicators and single imaging features, making it difficult to comprehensively reflect the biological characteristics and progression risk of the disease.

[0003] In the field of medical imaging diagnostics, positron emission tomography (PET), as a functional molecular imaging technique, reflects the functional state of tissues, such as metabolism, proliferation, and receptor expression, by detecting the distribution of radiotracers in the body. It boasts advantages such as functional sensitivity, molecular specificity, quantitative assessment capabilities, and whole-body scanning. PET can detect abnormalities in tissue metabolism and function at an early stage, identifying diseases before anatomical changes occur. Different tracers can target specific molecular pathways; for example, 18F-FDG reflects glucose metabolism, 18F-FLT reflects cell proliferation, and 11C-MET reflects amino acid transport. However, PET imaging also has limitations, such as low resolution and unclear visualization of anatomical structures. It needs to be used in conjunction with anatomical imaging techniques such as CT and MRI to achieve precise localization and comprehensive assessment.

[0004] However, existing PET image segmentation technologies face numerous challenges: First, insufficient segmentation accuracy. Traditional segmentation methods rely primarily on low-level features such as image grayscale and texture, making it difficult to adapt to the complex morphology and heterogeneity of lesions. Deep learning segmentation networks lack an understanding of the biological characteristics of lesions, resulting in significant performance degradation in cases of blurred boundaries and low contrast. Second, a disconnect between feature extraction and segmentation. Radiomics feature extraction is typically based on pre-segmentation results, and segmentation errors are directly transmitted to the feature extraction stage. The omics information obtained during feature extraction cannot be fed back to guide the segmentation process. Third, limited model generalization ability. Existing segmentation models are often designed for specific disease types and tracers, making it difficult to adapt to PET image data from different diseases and with different tracers. Finally, insufficient interpretability. Deep learning models are often considered "black boxes," making it difficult to explain their decision-making basis and reasoning process. The lack of visual explanations of segmentation results and feature importance makes it difficult to gain the trust of clinicians.

[0005] In deep learning-based PET image segmentation methods, representative works such as 3D U-Net and V-Net network architectures have been widely applied. These methods automatically extract multi-level features from PET images through convolutional neural networks and achieve end-to-end segmentation through an encoder-decoder structure. However, these methods mainly rely on the visual features of the images themselves and lack a deep understanding of the biological characteristics of lesions. Due to the low spatial resolution, blurred boundaries, and low signal-to-noise ratio of PET images, it is difficult to accurately identify the boundaries of complex lesions based solely on image features, especially when tumor heterogeneity is high, resulting in a significant decrease in segmentation accuracy. Furthermore, these methods typically employ fixed network structures and lack adaptability to different lesion characteristics, leading to poor generalization performance on PET images of different disease types and with different tracers.

[0006] In the area of ​​deep learning-based multimodal fusion methods, some studies have attempted to fuse PET images with anatomical images such as CT or MRI to improve segmentation accuracy. These methods typically employ strategies such as early fusion, late fusion, or feature-level fusion to integrate information from different modalities into the segmentation network. However, these methods primarily focus on the complementarity between different image modalities, neglecting the biological characteristics of lesions. Although some research has begun to explore the introduction of omics features into the segmentation process, most existing methods employ basic fusion strategies such as simple feature splicing or weighted averaging, lacking in-depth cross-modal interaction mechanisms and failing to fully leverage the guiding role of omics features in segmentation. Furthermore, existing fusion methods often use fixed fusion strategies, unable to dynamically adjust the fusion method according to the characteristics of the input data, resulting in unstable fusion effects in complex cases.

[0007] In summary, the main shortcomings of existing technical solutions are: 1) lack of effective cross-modal fusion mechanisms, which cannot fully utilize the guiding role of omics features in PET image segmentation; 2) insufficient boundary processing capabilities, making it difficult to accurately segment blurred lesion boundaries in PET images; 3) lack of adaptability in fusion strategies, which cannot be dynamically adjusted according to the characteristics of different input data; 4) lack of specific designs for PET image characteristics, such as boundary enhancement and attention guidance mechanisms; and 5) lack of interpretability in segmentation results, making it difficult to gain the trust and acceptance of clinicians. Summary of the Invention

[0008] To address the shortcomings of existing technologies, the root cause of these shortcomings lies in the fact that current methods fail to deeply understand the intrinsic relationship between PET images and omics features, lack the ability to design fusion and enhancement mechanisms specifically tailored to the characteristics of PET images, and fail to fully consider the need for interpretability of segmentation results in clinical applications. Therefore, this invention provides a PET image segmentation method, system, computer equipment, and storage medium.

[0009] To achieve the above objectives, the present invention provides a PET image segmentation method, comprising:

[0010] Acquire PET images and radiomics data.

[0011] Multi-scale texture feature tensors are extracted from PET images, and biological feature vectors are extracted from the image omics data. Attention fusion, gated fusion, and residual fusion mechanisms are applied to fuse the multi-scale texture feature tensors and biological feature vectors respectively, yielding attention fusion features, gated fusion features, and residual fusion features. Global features of the multi-scale texture feature tensors are extracted using a pooling function, and a three-dimensional weight vector is calculated based on these global features. The attention fusion features, gated fusion features, and residual fusion features are then weighted and fused using the three-dimensional weight vector, outputting a dynamic strategy weighted fusion feature of PET images and omics features.

[0012] Extract the boundaries of different scales of the dynamically weighted fusion features; perform convolutional fusion on the boundaries of different scales to generate a comprehensive boundary map; fuse the dynamically weighted fusion features and the comprehensive boundary map features to output multi-scale boundary enhancement features; use convolution to transform the multi-scale boundary enhancement features into a self-attention map; fuse the multi-scale boundary enhancement features and the self-attention map features to output attention boundary refinement features.

[0013] PET images are segmented using attention boundary refinement features, and multi-scale image segmentation masks are output.

[0014] Preferably, the PET images and radiomics data are processed by the dual-modal intelligent fusion boundary network model MultiFuseEdge_Net to obtain a multi-scale image segmentation mask map; the dual-modal intelligent fusion boundary network model MultiFuseEdge_Net is composed of a deep feature extraction network, an adaptive cross-modal fusion module AdaptCro, a multi-scale boundary enhancement module Multi-Sca, and a self-attention boundary refinement module AttBouRef connected in sequence.

[0015] Preferably, the multi-scale texture feature tensor and biological feature vector are processed by the AdaptCro module to obtain a dynamic strategy-weighted fusion feature map of PET images and omics features; wherein, the AdaptCro consists of a feature transformation layer, a quality assessment network, a three-strategy fusion unit, a weighted fusion module, and a refined convolutional network connected in sequence; the quality assessment network consists of an adaptive average pooling layer, a convolutional layer, and a ReLU activation function connected in sequence; the three-strategy fusion unit includes attention fusion, gating fusion, and residual fusion mechanisms.

[0016] Preferably, the dynamic policy weighted fusion features are processed by the Multi-Sca module to obtain multi-scale boundary enhancement features; the Multi-Sca module consists of a multi-scale boundary detection unit, a cross-scale boundary fusion module, a self-attention module, and a feature enhancement module connected in sequence; the multi-scale boundary detection unit includes multiple convolutional kernels of different sizes; the cross-scale boundary fusion module consists of a convolutional layer, a batch normalization layer (BN), a ReLU activation function, and a Sigmoid gating function connected in sequence.

[0017] Preferably, the multi-scale boundary enhancement features are processed by the AttBouRef module to obtain attention boundary refined features. The AttBouRef module consists of a self-attention mechanism and a residual connection module connected in sequence. Specifically, it includes: mapping the multi-scale boundary enhancement features to a three-dimensional feature tensor of query Q, key K, and value V using the self-attention mechanism; obtaining a feature similarity matrix using Q and K; generating a self-attention map using V and the feature similarity matrix; and performing feature fusion on the multi-scale boundary enhancement features and the self-attention map using a learnable parameter γ to output the attention boundary refined features.

[0018] Preferably, before extracting the multi-scale texture feature tensor of the PET image, the method further includes preprocessing the PET image, specifically: cleaning the acquired PET image data and denoising the cleaned data; performing grayscale adjustment, threshold setting, region extraction, and morphological feature processing on the denoised data using threshold segmentation technology to obtain an accurate region mask image; having multiple experienced physicians independently annotate the region mask image to obtain an annotated image; then using image data rotation and translation to expand the annotated image, adding Gaussian noise to the expanded image to obtain the preprocessed PET image.

[0019] Preferably, before extracting the biological feature vectors from the radiomics data, the method further includes preprocessing the radiomics data, specifically: reading omics features from the radiomics data, performing standard normal distribution processing on the omics features using StandardScaler technology, storing the processed normal distribution data in the self.omics_features database, and obtaining the preprocessed radiomics data file.

[0020] The present invention also provides a PET image segmentation system, comprising:

[0021] The data acquisition module is used to acquire PET images and radiomics data.

[0022] The model processing module is used to extract multi-scale texture feature tensors from PET images and biological feature vectors from radiomics data. Attention fusion, gated fusion, and residual fusion mechanisms are used to fuse the multi-scale texture feature tensors and biological feature vectors, respectively, to obtain attention fusion features, gated fusion features, and residual fusion features. Global features of the multi-scale texture feature tensors are extracted using pooling functions, and a three-dimensional weight vector is calculated based on these global features. The attention fusion features, gated fusion features, and residual fusion features are weighted and fused using the three-dimensional weight vector to output a dynamic strategy weighted fusion feature of PET images and radiomics features. Boundaries at different scales of the dynamic strategy weighted fusion features are extracted. Convolutional fusion is performed on the boundaries at different scales to generate a comprehensive boundary map. The dynamic strategy weighted fusion features and the comprehensive boundary map features are fused to output multi-scale boundary enhancement features. Convolution is used to transform the multi-scale boundary enhancement features into a self-attention map, and feature fusion is performed on the multi-scale boundary enhancement features and the self-attention map to output refined attention boundary features.

[0023] The model application module is used to refine features using attention boundaries to segment PET images and output multi-scale image segmentation masks.

[0024] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps in the PET image segmentation method.

[0025] The present invention also provides a computer-readable storage medium storing a computer program that, when loaded by a processor, can execute any of the steps in the PET image segmentation method.

[0026] The PET image segmentation method provided by this invention has the following beneficial effects:

[0027] This study extracts multi-scale texture feature tensors from PET images and biological feature vectors from radiomics data. It then processes the multi-scale texture feature tensors to obtain a three-dimensional weight vector based on the quality evaluation of the features. A strategy fusion is performed on the multi-scale texture feature tensors and biological feature vectors to obtain three sets of fused images. The three-dimensional weights are then used to weight these three sets of fused images. Dynamically evaluating feature quality allows for real-time adjustment of the fusion strategy weights, resulting in dynamically weighted fusion features that overcome the generalization bottleneck of fixed fusion strategies. Three boundary feature maps at different scales are extracted from the dynamically weighted fusion features using convolutional kernels of different sizes. These three feature maps are then weighted and fused to obtain multi-scale boundary enhancement features. Multi-head attention is then calculated to further refine the boundary features, ultimately outputting a clear and reliable multi-scale segmentation mask. Multi-scale convolution and self-attention refinement improve boundary segmentation accuracy, specifically addressing the issue of blurred boundaries in PET images. The generated multi-scale image segmentation mask visually demonstrates the basis for segmentation decisions, enhancing clinical credibility. Attached Figure Description

[0028] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart of a PET image segmentation method according to an embodiment of the present invention;

[0030] Figure 2 This is a flowchart of the MultiFuseEdge_Net model according to an embodiment of the present invention;

[0031] Figure 3 This is a flowchart illustrating the chest region segmentation process according to an embodiment of the present invention.

[0032] Figure 4 This is a diagram of the deep feature extraction structure according to an embodiment of the present invention;

[0033] Figure 5 This is a structural diagram of the AdaptCro adaptive cross-modal fusion module according to an embodiment of the present invention;

[0034] Figure 6 This is a structural diagram of the Multi-Sca multi-scale boundary enhancement module according to an embodiment of the present invention;

[0035] Figure 7 This is a structural diagram of the AdaptCro adaptive cross-modal fusion module according to an embodiment of the present invention. Detailed Implementation

[0036] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0037] This invention provides a PET image segmentation method, specifically as follows: Figure 1 As shown, it includes:

[0038] S1. Acquire PET images and radiomics data.

[0039] The acquired PET image data is cleaned and then denoised. The PET images are then subjected to grayscale adjustment, threshold setting, region extraction, and morphological processing through threshold segmentation to obtain accurate region mask images. The region mask images are independently annotated by several experienced physicians to obtain an annotated image. The annotated image is then expanded by rotating and translating the image data, and Gaussian noise is added to the expanded image to obtain the preprocessed PET image.

[0040] like Figure 3 As shown, the thoracic region was extracted through threshold segmentation (the chest is a common site for lung cancer lesions, and chest region segmentation is crucial for early diagnosis of lung cancer). First, images were independently annotated by at least three experienced nuclear medicine physicians, and samples with motion artifacts and metal artifacts were removed. Then, data augmentation was performed on the annotated images, including: randomly rotating the images left and right around their geometric center by a certain angle, randomly translating them horizontally / vertically by ±t pixels, and adding Gaussian noise (σ=0.01) to increase data diversity.

[0041] Omics feature extraction primarily relies on externally preprocessed CSV files (radiomics data) rather than being extracted directly from medical images. The CSV files contain pre-extracted omics features from the images, implemented using the PyRadiomics library. Pre-computed omics features (such as shape, texture, and functional features) are then read from the CSV files, assuming the last column of the CSV is the label. StandardScaler is then used to standardize all features, converting each feature to a standard normal distribution with a mean of 0 and a standard deviation of 1, eliminating the influence of different units on subsequent processing. Finally, the standardized feature matrix is ​​stored in self.omics_features for use in subsequent feature selection and fusion.

[0042] The purpose of omics feature selection is to filter out the most relevant and informative subset from a large number of possible features to reduce dimensionality, improve computational efficiency, avoid overfitting, and enhance model interpretability. This framework employs the ANOVA F-test, which tests whether there are significant differences in the means of multiple samples. The logic is to calculate the F-value by comparing between-group and within-group variances; if the F-value is greater than a critical value, the null hypothesis of equal means is rejected. This is accomplished through the following steps: First, pre-extracted omics feature data is loaded from a CSV file. Then, StandardScaler is used to standardize the features to eliminate the influence of dimensions. Next, the SelectKBest algorithm is applied to calculate the F-statistic of each feature relative to the target variable (e.g., tumor / non-tumor) to evaluate its discriminative power. Finally, the top k features with the highest F-values ​​are selected as key features, and the processed normally distributed data is stored in the self.omics_features database. This data is then used for subsequent fusion with deep learning features, thereby improving the accuracy and reliability of medical image segmentation.

[0043] S2. Extract the multi-scale texture feature tensor from PET images and extract the biological feature vector from the image omics data; fuse the multi-scale texture feature tensor and the biological feature vector using attention fusion, gated fusion, and residual fusion mechanisms respectively to obtain attention fusion features, gated fusion features, and residual fusion features; extract the global features of the multi-scale texture feature tensor using a pooling function, and calculate the three-dimensional weight vector based on the global features; use the three-dimensional weight vector to perform weighted fusion of the attention fusion features, gated fusion features, and residual fusion features, and output the dynamic strategy weighted fusion features of PET images and omics features; extract the boundaries of the dynamic strategy weighted fusion features at different scales; perform convolutional fusion on the boundaries at different scales to generate a comprehensive boundary map; fuse the dynamic strategy weighted fusion features and the comprehensive boundary map features to output multi-scale boundary enhancement features; use convolution to convert the multi-scale boundary enhancement features into a self-attention map, and perform feature fusion on the multi-scale boundary enhancement features and the self-attention map to output attention boundary refined features.

[0044] The MultiFuseEdge_Net bimodal intelligent fusion boundary network model employs a multi-stage processing flow. For example... Figure 2As shown, the MultiFuseEdge_Net dual-modal intelligent boundary network model consists of a deep feature extraction network, an adaptive cross-modal fusion module (AdaptCro), a multi-scale boundary enhancement module (Multi-Sca), and a self-attention boundary refinement module (AttBouRef), connected sequentially. First, the system processes PET images and omics data in parallel, acquiring multi-level deep features and high-dimensional omics feature representations through the deep feature extraction network and the omics feature encoder, respectively. Then, these features enter the feature fusion module, which supports multiple fusion strategies (feature-level fusion, decision-level fusion, hybrid fusion, cross-modal fusion, and adaptive cross-modal fusion). The adaptive cross-modal fusion can dynamically select the optimal fusion method based on the characteristics of the input data. The fused features are then processed by the multi-scale boundary enhancement module, which accurately captures lesion boundaries through multi-scale convolution and self-attention mechanisms. Finally, the attention generation module generates various attention maps (spatial attention, channel attention, fusion attention, and boundary attention) based on the enhanced features, guiding the segmentation head module to generate the final accurate segmentation result, thereby achieving effective guidance of omics features for PET image segmentation.

[0045] A deep feature extraction network is used to extract the multi-scale feature tensor F_deep from PET images, and an omics feature encoder is used to extract the biological feature vector F_omics from the omics feature CSV file. The multi-scale feature tensor F_deep and the biological feature vector F_omics are input into the adaptive cross-modal fusion module AdaptCro, where the number of channels in F_deep is compressed to a dimension compatible with the biological feature vector through 1×1 convolutions, resulting in the transformed deep feature F_deep_transformed. F_omics is then expanded into a spatial tensor through a fully connected layer to match the spatial dimension of the deep feature, resulting in the omics feature F_omics_transformed. Finally, a feature evaluation network is used to evaluate the deep feature F_deep. The deep_transformed method performs global feature extraction, extracting feature quality scores through two convolutional layers plus ReLU activation. The quality scores are then converted into a three-dimensional fusion weight vector W=(w1,w2,w3) using Softmax. Three strategies are applied to fuse F_deep_transformed and F_omics_transformed to obtain three fusion results: F_attention, F_gated, and F_residual. The three fusion results F_attention, F_gated, and F_residual are then weighted and fused using the fusion weight vector W=(w1,w2,w3) to output an adaptive fusion feature F_adaptive that combines PET images and omics features.

[0046] The F_adaptive function is input into the Multi-Sca multi-scale boundary enhancement module, which performs convolution using three kernels of different scales to output a set of three-scale boundary maps {EdgeMap_1, EdgeMap_2, EdgeMap_3}. A comprehensive boundary map, EdgeMap_total, is generated by weighted summation and fusion of boundary information from different scales. The boundary enhancement feature F_enhanced is obtained by element-wise multiplication of F_adaptive and EdgeMap_total. Finally, the low-level features from the deep feature extraction module are fused with F_enhanced through feature concatenation and convolution to output the multi-scale boundary enhancement feature F_boundary.

[0047] The F_boundary is input into the self-attention boundary refinement module AdaptCro. The self-attention mechanism maps the F_boundary into three-dimensional feature tensors Q, K, and V. The feature similarity matrix is ​​calculated using the Q and K feature tensors. The self-attention map AttentionMap is generated using the V feature tensor and the feature similarity matrix. The F_boundary and AttentionMap are fused using a learnable parameter γ to output the attention boundary refined feature F_refined. The attention boundary refined feature F_refined is then used to segment the PET image, outputting a multi-scale segmentation mask map.

[0048] like Figure 4 As shown, the data is input into the input block of the deep feature extraction section, where it undergoes initial convolution, batch normalization, and activation processing. Next, the data sequentially passes through multiple encoder stages, including Enc1, Enc2, Enc3, and Enc4. These stages progressively extract feature representations from low to high levels using residual blocks, with the resolution of the feature maps gradually decreasing. The encoder stage consists of multiple residual blocks, each composed of 3D convolution, normalization, and ReLU activation functions.

[0049] The high-level features extracted from the encoder stage (Enc4) will be fed into the ASPP (Spatial Pyramid Pooling) module. This module expands the receptive field without changing the input size by using dilated convolutions with different sampling intervals (i.e., "holes"), thereby capturing target information and context at different scales and enhancing the semantic understanding of the features.

[0050] This embodiment utilizes a Feature Pyramid Network (FPN) to fuse features from different encoder stages (Enc1, Enc2, Enc3) and the ASPP module output. The FPN primarily consists of two paths: a top-down path and a lateral connection path. The top-down path upsamples high-level semantically strong but spatially low-resolution feature maps and fuses them with lower-level high-resolution feature maps, gradually recovering detailed information. The lateral connection directly connects feature maps from different layers of the backbone network to their corresponding top-down layers, achieving semantic alignment and information complementarity at different scales. Finally, the FPN generates a fused feature map that contains both rich detail and semantic information.

[0051] Finally, the fused feature map is refined sequentially through Channel Attention and Spatial Attention modules. The Channel Attention module focuses on enhancing important channel features, while the Spatial Attention module emphasizes highlighting key spatial region features. The features processed by these two attention mechanisms, along with the low-level and mid-level features extracted from Enc1 and Enc2, and the high-level features output by the ASPP module, form the final output feature set, providing multi-dimensional and multi-scale deep feature representations for subsequent tasks (such as classification and segmentation).

[0052] The Adaptive Cross-Modal Fusion (AdaptCro) module dynamically adjusts the fusion strategy through a feature quality assessment network, enabling adaptive processing of different input data. For example... Figure 5 As shown, AdaptCro consists of a feature transformation layer, a quality evaluation network, a three-strategy fusion unit, a weighted fusion module, and a refined convolutional network connected sequentially. The quality evaluation network, composed of an adaptive average pooling layer, a convolutional layer, and a ReLU activation function connected sequentially, is used to evaluate the feature quality score of F_deep_transformed, generating three-dimensional weight vectors w1, w2, and w3 from the feature quality score. The three-strategy fusion unit consists of attention fusion, gating fusion, and residual fusion. Attention fusion extracts channel attention from F_deep and F_omics to obtain F_attention. Gating fusion uses a sigmoid gating function to dynamically assign weights to F_deep and F_omics to obtain F_gated. Residual fusion injects incremental information from F_omics into F_deep through residual connections to obtain F_residual.

[0053] The AdaptCro module first processes features from two modalities through deep feature transformation and omics feature transformation, respectively. Then, it uses a feature quality estimator to evaluate the quality of the deep features, generating weights for three fusion strategies. The feature quality estimator employs a combination of adaptive pooling and convolution, calculating the weights using the formula W = Softmax(Conv(ReLU(Conv(AdaptiveAvgPool(F_{deep}))))), where W is a three-dimensional vector corresponding to the weights of attention fusion, gated fusion, and residual fusion strategies, respectively. The module executes these three fusion strategies simultaneously and performs weighted fusion based on the quality estimator weights, with the fusion formula F_{adaptive}=\sum_{i=1}^{3}w_i\cdot F_i. Finally, a refined convolutional network further optimizes the fused features. This design enables the system to automatically select the optimal fusion strategy based on the characteristics of the input data, significantly improving the model's adaptability and robustness.

[0054] The Multi-Scale Boundary Enhancement (Multi-Sca) module is specifically designed to address the common blurred boundary problem in PET images, enhancing boundary detection capabilities through multi-scale feature analysis. For example... Figure 6 As shown, Multi-Sca consists of a multi-scale boundary detection unit, a cross-scale boundary fusion module, a self-attention module, and a feature enhancement module connected sequentially. The multi-scale boundary detection unit uses multiple convolutional kernels of different sizes to detect boundaries in parallel, extracting EdgeMap_1, EdgeMap_2, and EdgeMap_3. The cross-scale boundary fusion module consists of a convolutional layer, a batch normalization layer (BN), a ReLU activation function, and a sigmoid gating function connected sequentially. This module unifies the image sizes of EdgeMap_1, EdgeMap_2, and EdgeMap_3 and fuses them to generate a comprehensive boundary map, EdgeMap_total. The feature enhancement module uses EdgeMap_total to perform element-wise weighted enhancement of F_adaptive, outputting the multi-scale boundary enhancement feature F_boundary.

[0055] The Multi-Sca module comprises three boundary detection branches at different scales, using 3×3, 5×5, and 7×7 convolutional kernels to capture boundary information at different scales. These boundary maps are then integrated into a single comprehensive boundary map via a boundary fusion network. Boundary enhancement employs the formula F_{enhanced} = F_{input} \cdot (1 + EdgeMap), enhancing the original features using the boundary map. Simultaneously, the module integrates detailed information from low-level features, combining the enhanced boundary features with the low-level features through a feature fusion network. Finally, self-attention boundary refinement further optimizes the boundary features. This multi-scale design effectively captures lesion boundaries of varying sizes and shapes, significantly improving segmentation accuracy.

[0056] The SelfAttention Refinement (AttBouRef) module utilizes the self-attention mechanism to further optimize boundary features, enhancing the coherence and accuracy of the boundaries. For example... Figure 7 As shown, AttBouRef is connected sequentially by a self-attention mechanism and a residual connection module, including: using the self-attention mechanism to map F_boundary into a three-dimensional feature tensor of Q, K, and V; using Q and K to calculate the feature similarity matrix; using V and the feature similarity matrix to generate a self-attention map AttentionMap; using the learnable parameter γ to fuse features of F_boundary and AttentionMap; and outputting the attention boundary refined feature F_refined.

[0057] The AttBouRef module is based on a query-key-value (QKV) architecture. It first generates query, key, and value features through 1×1 convolutions, then calculates a self-attention map, expressed as Attention(Q,K,V) = softmax(QK^T)V, where Q, K, and V are the query, key, and value features, respectively. The module controls the degree of self-attention through a learnable parameter γ, and the final feature calculation formula is F_{refined} = γ \cdot Attention(F) + F, where residual connections ensure the preservation of original information. This self-attention mechanism can capture long-range dependencies within the feature map, making the boundaries more coherent and accurate, and is particularly suitable for processing lesions with complex shapes.

[0058] S3. Use attention boundary refinement features to segment PET images and output multi-scale image segmentation mask.

[0059] Through innovative adaptive cross-modal fusion and multi-scale boundary enhancement modules, this invention effectively addresses the problems of blurred lesion boundaries and insufficient segmentation accuracy in existing technologies, significantly improving segmentation accuracy, especially in complex lesion scenarios. The adaptive cross-modal fusion and gated fusion modules dynamically adjust the fusion strategy based on the characteristics of the input data, overcoming the limitations of fixed fusion strategies in existing technologies and enhancing the model's adaptability and generalization ability to different disease types and tracer data. By generating attention maps and boundary enhancement maps, this invention provides clinicians with intuitive segmentation criteria, solving the "black box" problem of existing technologies and enhancing clinical trust and acceptance. The deep fusion of deep features and omics features breaks the traditional separation between segmentation and omics analysis, forming a two-way feedback and improving the overall efficiency and accuracy of the diagnostic process.

[0060] Dynamically adjusting the multimodal feature fusion method fully utilizes omics information to significantly improve segmentation performance; multi-scale design effectively captures lesion boundaries, solving the problem of blurred boundaries in PET images; visualization tools enhance clinicians' understanding and trust in the segmentation results.

[0061] Based on the same inventive concept, the present invention also provides a PET image segmentation system, comprising:

[0062] The data acquisition module is used to acquire PET images and radiomics data.

[0063] The model processing module is used to extract multi-scale texture feature tensors from PET images and biological feature vectors from radiomics data. Attention fusion, gated fusion, and residual fusion mechanisms are used to fuse the multi-scale texture feature tensors and biological feature vectors, respectively, to obtain attention fusion features, gated fusion features, and residual fusion features. Global features of the multi-scale texture feature tensors are extracted using pooling functions, and a three-dimensional weight vector is calculated based on these global features. The attention fusion features, gated fusion features, and residual fusion features are weighted and fused using the three-dimensional weight vector to output a dynamic strategy weighted fusion feature of PET images and radiomics features. Boundaries at different scales of the dynamic strategy weighted fusion features are extracted. Convolutional fusion is performed on the boundaries at different scales to generate a comprehensive boundary map. The dynamic strategy weighted fusion features and the comprehensive boundary map features are fused to output multi-scale boundary enhancement features. Convolution is used to transform the multi-scale boundary enhancement features into a self-attention map, and feature fusion is performed on the multi-scale boundary enhancement features and the self-attention map to output refined attention boundary features.

[0064] The model application module is used to refine features using attention boundaries to segment PET images and output multi-scale image segmentation masks.

[0065] This invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the PET image segmentation method provided above.

[0066] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the PET image segmentation method provided above.

[0067] For specific limitations regarding the computational system for PET image segmentation, please refer to the limitations of the PET image segmentation method described above, which will not be repeated here. Each module in the aforementioned PET image segmentation system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0068] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. Furthermore, the above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A PET image segmentation method, characterized in that, The method includes: Acquire PET images and radiomics data; Multi-scale texture feature tensors are extracted from PET images, and biological feature vectors are extracted from radiomics data. Attention fusion, gated fusion, and residual fusion mechanisms are applied to fuse the multi-scale texture feature tensors and biological feature vectors respectively, yielding attention fusion features, gated fusion features, and residual fusion features. Global features of the multi-scale texture feature tensors are extracted using pooling functions, and a three-dimensional weight vector is calculated based on these global features. The attention fusion features, gated fusion features, and residual fusion features are then weighted and fused using the three-dimensional weight vector, outputting a dynamic strategy weighted fusion feature of PET images and radiomics features. Extract the boundaries of different scales of the dynamically weighted fusion features; perform convolutional fusion on the boundaries of different scales to generate a comprehensive boundary map; fuse the dynamically weighted fusion features and the comprehensive boundary map features to output multi-scale boundary enhancement features; use convolution to transform the multi-scale boundary enhancement features into a self-attention map; fuse the multi-scale boundary enhancement features and the self-attention map features to output attention boundary refinement features; PET images are segmented using attention boundary refinement features, and multi-scale image segmentation masks are output.

2. The PET image segmentation method according to claim 1, characterized in that, The PET images and radiomics data are processed by the MultiFuseEdge_Net dual-modal intelligent fusion boundary network model to obtain a multi-scale image segmentation mask. The MultiFuseEdge_Net dual-modal intelligent fusion boundary network model is composed of a deep feature extraction network, an adaptive cross-modal fusion module AdaptCro, a multi-scale boundary enhancement module Multi-Sca, and a self-attention boundary refinement module AttBouRef connected in sequence.

3. The PET image segmentation method according to claim 2, characterized in that, The AdaptCro module processes the multi-scale texture feature tensor and biological feature vector to obtain a dynamic strategy-weighted fusion feature map of PET images and omics features. The AdaptCro module consists of a feature transformation layer, a quality assessment network, a three-strategy fusion unit, a weighted fusion module, and a refined convolutional network connected sequentially. The quality assessment network consists of an adaptive average pooling layer, a convolutional layer, and a ReLU activation function connected sequentially. The three-strategy fusion unit includes attention fusion, gated fusion, and residual fusion mechanisms.

4. The PET image segmentation method according to claim 2, characterized in that, The Multi-Sca module processes the weighted fusion features of the dynamic strategy to obtain multi-scale boundary enhancement features. The Multi-Sca module consists of a multi-scale boundary detection unit, a cross-scale boundary fusion module, a self-attention module, and a feature enhancement module connected in sequence. The multi-scale boundary detection unit includes multiple convolutional kernels of different sizes. The cross-scale boundary fusion module consists of a convolutional layer, a batch normalization layer (BN), a ReLU activation function, and a Sigmoid gating function connected in sequence.

5. A PET image segmentation method according to claim 2, characterized in that, The AttBouRef module processes the multi-scale boundary enhancement features to obtain attention boundary refined features. The AttBouRef module consists of a self-attention mechanism and a residual connection module connected sequentially. Specifically, it includes: mapping the multi-scale boundary enhancement features to a three-dimensional feature tensor of query Q, key K, and value V using the self-attention mechanism; obtaining a feature similarity matrix using Q and K; generating a self-attention map using V and the feature similarity matrix; and fusing the multi-scale boundary enhancement features and the self-attention map using a learnable parameter γ to output the attention boundary refined features.

6. The PET image segmentation method according to claim 1, characterized in that, Before extracting the multi-scale texture feature tensor of the PET image, the process also includes preprocessing the PET image, specifically: cleaning the acquired PET image data and denoising the cleaned data; using threshold segmentation technology to perform grayscale adjustment, threshold setting, region extraction, and morphological feature processing on the denoised data to obtain an accurate region mask image; having multiple experienced physicians independently annotate the region mask image to obtain an annotated image; then using image data rotation and translation to expand the annotated image, adding Gaussian noise to the expanded image to obtain the preprocessed PET image.

7. The PET image segmentation method according to claim 1, characterized in that, Before extracting the biological feature vectors from the radiomics data, the process also includes preprocessing the radiomics data, specifically: reading the omics features from the radiomics data, using StandardScaler technology to process the omics features into a standard normal distribution, storing the processed normal distribution data in the self.omics_features database, and obtaining the preprocessed radiomics data file.

8. A PET image segmentation system, characterized in that, include: The data acquisition module is used to acquire PET images and radiomics data; The model processing module is used to extract multi-scale texture feature tensors from PET images and biological feature vectors from radiomics data. Attention fusion, gated fusion, and residual fusion mechanisms are applied to fuse the multi-scale texture feature tensors and biological feature vectors respectively, resulting in attention fusion features, gated fusion features, and residual fusion features. Global features of the multi-scale texture feature tensors are extracted using pooling functions, and a three-dimensional weight vector is calculated based on these global features. The attention fusion features, gated fusion features, and residual fusion features are weighted and fused using the three-dimensional weight vector to output a dynamic strategy weighted fusion feature of PET images and radiomics features. Boundaries at different scales of the dynamic strategy weighted fusion features are extracted. Convolutional fusion is performed on the boundaries at different scales to generate a comprehensive boundary map. The dynamic strategy weighted fusion features and the comprehensive boundary map features are fused to output multi-scale boundary enhancement features. Convolution is used to transform the multi-scale boundary enhancement features into a self-attention map, and feature fusion is performed on the multi-scale boundary enhancement features and the self-attention map to output refined attention boundary features. The model application module is used to refine features using attention boundaries to segment PET images and output multi-scale image segmentation masks.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the processor executes the computer program, it implements the method steps of any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method steps as described in any one of claims 1 to 7.

Images