A head and neck tumor image segmentation method based on CT and PET combined analysis

Abstract

The application discloses a head and neck tumor image segmentation method based on CT and PET combined analysis, comprising the following steps: acquiring a head and neck tumor data set, and preprocessing the head and neck tumor data set; constructing a parallel shared attention mechanism module; constructing an improved UNet3D network model; optimizing the improved UNet3D network model by using a combined loss function; inputting the head and neck tumor data set after preprocessing into the optimized improved UNet3D network model, and outputting a segmentation task result of a dual-mode image. By introducing a parallel shared attention mechanism and a multi-scale feature fusion technology, the application efficiently combines multi-modal image information, captures the characteristics of tumors at different scales, and accurately segments the tumors, thereby significantly improving the accuracy and efficiency of head and neck tumor segmentation, overcoming the limitation that a single-mode image is difficult to comprehensively reflect tumor characteristics, and solving the scale diversity problem of tumors in images.

Description

Technical Field

[0001] This invention belongs to the technical field of image segmentation, specifically relating to a method for segmenting head and neck tumor images based on joint analysis of CT and PET. Background Technology

[0002] Head and neck tumors are among the most common types of malignant tumors worldwide, including nasopharyngeal carcinoma, oral cancer, and laryngeal cancer. Due to the complex anatomy of the head and neck and the indistinct tumor boundaries, clinical diagnosis and treatment are challenging. Precise tumor segmentation is crucial for treatment planning, surgical planning, and radiotherapy. Currently, most tumor segmentation methods used in clinical practice rely on manual annotation by physicians or rule-based semi-automatic algorithms. However, these traditional methods have significant limitations in complex anatomical regions and struggle to handle the irregularity of tumor morphology and the uncertainty of its boundaries. Therefore, developing automated and intelligent tumor segmentation methods to improve segmentation efficiency and accuracy has become a key direction in current medical imaging research.

[0003] Multimodal imaging technology has shown great potential in the diagnosis and treatment of head and neck tumors. Computed tomography (CT) provides high-resolution anatomical information, while positron emission tomography (PET) reflects the metabolic activity of tissues. Joint analysis of these two different modalities can more comprehensively reflect the biological characteristics of tumors, helping to improve tumor segmentation accuracy. However, due to the differences in spatial resolution and image features between CT and PET, achieving efficient fusion of the two modalities remains a challenge for segmentation techniques. Traditional single-modal segmentation methods often perform poorly when processing such multimodal data. Therefore, employing deep learning models that can effectively combine multimodal information, especially segmentation methods based on convolutional neural networks (CNNs), is key to improving segmentation performance.

[0004] In recent years, deep learning technology, especially convolutional neural networks (CNNs), has made groundbreaking progress in medical image segmentation tasks. Compared with traditional methods, deep learning automatically learns image features in a data-driven manner, greatly reducing the burden of manually designing features, and demonstrating superior performance, especially when processing complex images. Three-dimensional convolutional neural networks (3D-CNNs) have shown powerful capabilities in learning the three-dimensional spatial features of medical images, fully utilizing the information in three-dimensional data to significantly improve the accuracy of tumor segmentation. In particular, the attention mechanism introduced in recent years helps the model automatically focus on key regions in the image, improving the accuracy of tumor edge segmentation while reducing background interference. Furthermore, multi-scale fusion techniques, by combining features at different scales, further enhance the model's sensitivity to tumor morphology, making it more robust in complex tumor segmentation tasks.

[0005] The development of automated medical image segmentation technology has provided new ideas and methods for tumor segmentation. Deep learning-based automated segmentation systems not only reduce the time cost of manual operations but also maintain high segmentation efficiency when dealing with large amounts of image data. Automated segmentation methods have shown great potential in the joint analysis of multimodal data, especially in segmenting complex anatomical regions such as head and neck tumors. By fusing multimodal information from CT and PET, automated segmentation systems can significantly improve the accuracy and robustness of segmentation. However, despite the excellent performance of deep learning models in segmentation tasks, their clinical application still faces some challenges, including image data heterogeneity, imaging differences between devices, artifacts, and noise interference. Therefore, enhancing the generalization ability of models and ensuring stable performance under different patient and device conditions is a significant challenge in the clinical application of automated segmentation technology.

[0006] In the clinical management of head and neck tumors, traditional manual segmentation methods rely heavily on the physician's experience. This method is time-consuming and labor-intensive, increasing the physician's workload and making it susceptible to subjective judgment, leading to significant differences in results between physicians. Furthermore, the complex anatomy of the head and neck region and the often blurred boundaries between tumors and surrounding normal tissues make it difficult for traditional manual segmentation methods to accurately distinguish these subtle differences, resulting in insufficient precision and stability in tumor segmentation. Especially when meticulous marking of complex tumor boundaries is required, manual operation is not only inefficient but also lacks accuracy, directly impacting subsequent treatment planning.

[0007] The application of single-modal imaging (such as CT or PET) in medical segmentation also has significant limitations. CT images provide anatomical structural information, while PET reflects metabolic functional information. Relying solely on either modality for tumor segmentation cannot comprehensively reflect the characteristics of the tumor. This incompleteness of information often leads to difficulties in accurately identifying tumor boundaries, especially in cases of complex tumor morphology or blurred boundaries. Single-modal imaging cannot provide sufficient contrast, resulting in low segmentation accuracy. Furthermore, traditional deep learning segmentation models primarily process single-modal data, and current technologies do not fully utilize the combined use of multimodal imaging. This prevents the model from effectively capturing complementary information from CT and PET images, thus limiting the improvement of segmentation performance.

[0008] Introducing attention mechanisms into medical image segmentation has become a research hotspot in recent years. However, most existing attention mechanisms only extract features from a single modality or scale, making it difficult to fully capture the multi-scale information of tumors. When dealing with structurally complex and variable targets such as head and neck tumors, existing attention mechanisms cannot accurately locate important feature regions, affecting the accuracy of segmentation. Although multi-scale fusion can effectively enhance the network's ability to capture tumor features at different resolutions, existing multi-scale fusion strategies often face the problems of feature information loss or poor fusion results, leading to poor performance of segmentation models when dealing with complex lesions. Furthermore, while deep learning models have achieved automated segmentation to some extent, their generalization ability and sensitivity to clinical noise remain major bottlenecks hindering their widespread adoption. Summary of the Invention

[0009] The purpose of this invention is to address the aforementioned shortcomings in the prior art by providing a head and neck tumor image segmentation method based on joint analysis of CT and PET, thereby solving the problem that the existing technology does not fully utilize the joint use of multimodal images, resulting in the model's inability to effectively capture complementary information from CT and PET images, which restricts the improvement of segmentation performance.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0011] A method for segmenting head and neck tumor images based on combined CT and PET analysis, comprising the following steps:

[0012] S1. Obtain a head and neck tumor dataset containing CT images, PET images, and segmented labeled images, and preprocess the head and neck tumor dataset.

[0013] S2. Construct a parallel common attention mechanism module;

[0014] S3. Insert several parallel co-attention mechanism modules into the backbone network of the UNet3D network model to construct an improved UNet3D network model.

[0015] S4. An improved UNet3D network model optimized using a combined loss function;

[0016] S5. Input the preprocessed head and neck tumor dataset into the optimized and improved UNet3D network model to output the segmentation results of the bimodal images.

[0017] Furthermore, S1 preprocesses the neck tumor dataset, including the following steps:

[0018] S11. The CT and PET images are resampled using triple linear interpolation, and the label image is resampled using nearest neighbor interpolation. During sampling, the size of the resampled images is adjusted based on the spatial orientation, origin, and coordinate system of the CT image until the voxel spacing and spatial coordinate system of the CT, PET, and label images are consistent.

[0019] S12. Detect the spatial location of the tumor in the label image, determine the bounding box where the tumor is located, calculate the center point of the tumor based on the position of the bounding box, and crop the CT image, PET image and label image to 144px×144px×144px based on the center point.

[0020] S13. Perform z-score normalization on the PET images, and perform grayscale truncation and maximum-minimum normalization on the CT images in sequence.

[0021] Furthermore, S2 includes the following sub-steps:

[0022] S21. Input the CT image features and PET image features into the SE module respectively, and output the feature weighted vectors of the CT image and PET image respectively;

[0023] S22. Calculate the affinity matrix based on the feature weighted vectors of CT and PET images;

[0024] S23. Based on the affinity matrix, calculate the common attention weights of CT image features and PET image features respectively;

[0025] S24. Calculate the weighted sum of CT image features and PET image features based on the co-attention weights of CT image features and PET image features respectively, and then concatenate the weighted sums of CT image features and PET image features to obtain the fused co-attention feature map.

[0026] Furthermore, in S21, the output feature weighted vectors of the CT and PET images include:

[0027]

[0028]

[0029] Among them, F ct and F pet , respectively, represent CT image features and PET image features; X represents global features; GAP represents global average pooling; T represents the feature vector output by the fully connected layer; FC represents the fully connected layer; σ1 is the ReLU activation function, σ2 is the sigmoid activation function; se represents weights; ⊙ represents element-wise multiplication; Q and V are the feature weight vectors of CT image features and PET image features, respectively.

[0030] Furthermore, in S22, the affinity matrix is ​​calculated, including:

[0031] Affinity = tanh(W) b (Q)·W b (V))

[0032] Where Affinity is the affinity matrix; tanh is the activation function; W b This is the weight matrix.

[0033] Furthermore, in S23, the common attention weights of CT image features and PET image features are calculated respectively, including:

[0034] H v =tanh(W v (V)+W q (Q)·Affinity)

[0035] H q =tanh(W q (Q)+W v (V)·Affinity)

[0036] a v =softmax(W h ·H v )

[0037] a q =softmax(W h ·H q )

[0038] Among them, H v H q Modality enhancement feature vectors for PET and CT images, respectively; W v W q W h These are the weight parameters; a v and aq These are the co-attention weights for PET image features and CT image features, respectively.

[0039] Furthermore, in S24, the fused common attention feature map is obtained, including:

[0040] F′ ct =∑a q ·Q

[0041] F′ pet =∑a v ·V

[0042] F fusion =Concat(F′ ct ,F′ pet )

[0043] Among them, F′ ct and F′ pet The output features are weighted from CT and PET image features; Concat is the stitching operation; F fusion This is the fused co-attention feature map.

[0044] Furthermore, S3 includes the following sub-steps:

[0045] S31. Input the CT image and PET image into two independent UNet encoder stages respectively, and output the preliminary features q1, q2, q3, and q4 of the CT image, and the preliminary features v1, v2, v3, and v4 of the PET image respectively.

[0046] S32. Insert parallel co-attention mechanism modules at the back end of each encoder stage of the UNet3D backbone network;

[0047] The original CT and PET image data are input into the first parallel co-attention mechanism module, which outputs the fused bimodal feature F0.

[0048] The preliminary features q1 of the CT image and the preliminary features v1 of the PET image obtained in the first stage are input into the second parallel co-attention mechanism module, and the fused bimodal features F1 are output.

[0049] The preliminary CT image features q2 and preliminary PET image features v2 obtained in the second stage are input into the third parallel co-attention mechanism module, which outputs the fused bimodal features F2.

[0050] The preliminary CT image features q3 and preliminary PET image features v3 obtained in the third stage are input into the fourth parallel co-attention mechanism module, which outputs the fused dual-modal features F3.

[0051] The preliminary CT image features q4 and preliminary PET image features v4 obtained in the fourth stage are input into the fifth parallel co-attention mechanism module, which outputs the fused dual-modal features F4.

[0052] S33. The dual-modal feature F0 and the original CT and PET image data are stitched together to form a feature vector with 4 channels, and then input into the encoder module of the first stage. After feature extraction by the encoder module, the number of channels becomes 16.

[0053] The spliced ​​feature map output from the first stage encoder module is spliced ​​with F1 and then input to the second stage encoder module. After feature extraction by the encoder module, the number of channels becomes 32.

[0054] The spliced ​​feature map output from the second-stage encoder module is spliced ​​with F2 and then input into the third-stage encoder module. After feature extraction by the encoder module, the number of channels becomes 64.

[0055] The spliced ​​feature map output from the third-stage encoder module is spliced ​​with F3 and then input into the fourth-stage encoder module. After feature extraction by the encoder module, the number of channels becomes 128.

[0056] The spliced ​​feature map output from the fourth stage encoder module is spliced ​​with F4 and then input into the fifth stage encoder module. After feature extraction by the encoder module, the number of channels becomes 256.

[0057] S34. In the decoder section, the output of the encoder module in the fifth stage is deconvolutioned to restore the number of channels from 256 to 128, and this output is concatenated with the feature map of the encoder module in the fourth stage, and features are extracted through convolution operation.

[0058] The output of the fourth-stage decoder module is deconvolutioned to restore the number of channels from 128 to 64, and this output is concatenated with the feature map of the third-stage encoder module. Features are then extracted through convolution.

[0059] The output of the third-stage decoder module is deconvolutioned to restore the number of channels from 64 to 32, and this output is concatenated with the feature map of the second-stage encoder module. Features are then extracted through convolution.

[0060] The output of the second-stage decoder module is deconvolutioned to restore the number of channels from 32 to 16, and this output is concatenated with the feature map of the first-stage encoder module. Features are then extracted through convolution.

[0061] The output of the first-stage decoder module is passed through a 1x1x1 convolutional layer to reduce the number of channels from 16 to 2, and the segmentation task result is output.

[0062] Furthermore, in S4, the combined loss function includes the Soft Dice Loss function and the weighted FocalLoss loss function, specifically as follows:

[0063] Loss=Soft Dice Loss+0.5×Focal Loss

[0064] Focal Loss(p) = -α(1-p) γ log(p)

[0065] Where Loss is the combined loss function; p is the confidence level of the model prediction; and α and γ are parameters.

[0066] The head and neck tumor image segmentation method based on combined CT and PET analysis provided by this invention has the following beneficial effects:

[0067] 1. By introducing a parallel co-attention mechanism and multi-scale feature fusion technology, this invention can efficiently combine multimodal image information, capture the features of tumors at different scales, and accurately segment them. The method of this invention can not only improve the accuracy and efficiency of head and neck tumor segmentation, but also provide more reliable technical support for clinical diagnosis and treatment.

[0068] 2. This invention combines deep learning with multimodal imaging technology, making full use of the anatomical structure information in CT images and the metabolic function information in PET images, overcoming the limitation that single-modal images cannot fully reflect tumor characteristics, and significantly improving the accuracy of segmentation results.

[0069] 3. This invention introduces a parallel co-attention (PCA) mechanism and a multi-scale feature fusion technique. The attention mechanism can automatically focus on important features of the tumor region and ignore irrelevant information, making the model more accurate when processing complex tumor morphologies; multi-scale feature fusion ensures that details of tumors of different sizes can be captured and processed, thereby solving the problem of scale diversity of tumors in images.

[0070] 4. The present invention also provides a method for preprocessing CT and PET images to improve the quality and consistency of the input images to the model. The image preprocessing process includes resampling, cropping and normalizing CT and PET data to ensure the alignment and clarity of multimodal images during joint analysis. Through this preprocessing method, the present invention can better ensure that the segmentation model maintains high performance when faced with image data from different sources and of different qualities.

[0071] 5. This invention preserves the detailed information of multimodal image features: by resampling and unifying the coordinate system of CT images, PET images and Label images, the spatial consistency of multimodal images is ensured, and the details and structural information of the images are preserved.

[0072] 6. This invention enhances the model's focus on tumor regions: It employs a parallel common attention mechanism (PCA) to deeply fuse CT and PET features, thereby enhancing the model's comprehensive perception of image information from different modalities and improving the segmentation effect of tumor and pathological regions.

[0073] 7. This invention enhances feature representation capabilities: By introducing a PCA module at each encoding stage, cross-modal fusion of CT and PET image features is achieved, enhancing the model's ability to represent complex images and improving segmentation accuracy.

[0074] 8. This invention improves the stability of model training and segmentation performance: By combining the combined loss function of Soft Dice Loss and weighted Focal Loss, the class imbalance problem is effectively addressed, the focus on hard-to-classify samples is enhanced, and the model's segmentation performance is further improved. Attached Figure Description

[0075] Figure 1 This is the data preprocessing operation flow of the present invention.

[0076] Figure 2 This is a comparison image before and after the preprocessing of this invention.

[0077] Figure 3 This is a schematic diagram of the PCA module structure of the present invention.

[0078] Figure 4 This is a schematic diagram of the SE module structure of the present invention.

[0079] Figure 5 This is the PCA-UNet network model architecture of the present invention.

[0080] Figure 6 To obtain v for this invention i and q i The structural diagram.

[0081] Figure 7 This is a comparison chart of the segmentation results of various models in this invention.

[0082] Figure 8 This is a flowchart of a head and neck tumor image segmentation method based on combined CT and PET analysis according to the present invention. Detailed Implementation

[0083] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0084] Example 1

[0085] This embodiment presents a head and neck tumor image segmentation method based on joint CT and PET analysis. By combining deep learning with multimodal imaging technology, it fully utilizes the anatomical structure information in CT images and the metabolic function information in PET images, overcoming the limitation of single-modal images in comprehensively reflecting tumor characteristics, and significantly improving the accuracy of segmentation results. (Reference) Figure 8 Specifically, it includes the following:

[0086] Step S1: Obtain a head and neck tumor dataset containing CT images, PET images, and segmented label images (Labels), and preprocess the head and neck tumor dataset, which specifically includes the following sub-steps:

[0087] Step S11: Resampling;

[0088] The spatial voxel spacing of the input CT image, PET image, and Label image is resampled to 1mm×1mm×1mm, and the spatial coordinates of the CT image are used as a unified reference for the three images to ensure consistent voxel spacing and aligned spatial coordinate systems, resulting in precise spatial matching. The specific steps are as follows:

[0089] First, for CT and PET images, triple linear interpolation is used for resampling to maintain the smoothness and continuity of image details, ensuring that grayscale information and fine structures are not lost during the resampling process.

[0090] For Label images, since they contain discrete category values, the Nearest Neighbor Interpolation method is used for resampling to avoid confusion of category values. This ensures that the category values ​​remain unchanged after resampling and avoids the introduction of non-integer intermediate values ​​during the interpolation process.

[0091] During the resampling process, the spatial orientation, origin, and coordinate system of the CT image are uniformly adopted, and the size of the resampled image is adjusted accordingly based on its original voxel spacing to ensure that the three have consistent spatial resolution after resampling and can be strictly aligned in the same spatial coordinate system.

[0092] Step S12: Image cropping;

[0093] The resampled Label image is cropped. First, pixels with a label value of 2 (lymph nodes) are relabeled as background with a value of 0, retaining only the background (value 0) and the tumor (value 1). Then, the spatial dimensions of the CT image, PET image, and Label image are cropped to reduce the image size, alleviate the computational burden of subsequent model training, and retain the effective information region containing the tumor.

[0094] The specific steps are as follows: First, the bounding box of the tumor is determined by detecting its spatial location in the Label image. The center point of the tumor is calculated based on the bounding box's location. Then, using this center point as a reference, the CT image, PET image, and Label image are cropped to a fixed size of 144px × 144px × 144px to ensure the tumor region is completely preserved. Simultaneously, image boundaries are considered during cropping to ensure the cropping operation does not exceed the actual image size. If the cropped area is smaller than expected, the starting point is adjusted appropriately to ensure the final cropped image and label meet the predetermined size requirements.

[0095] Step S13: Image normalization;

[0096] To perform z-score normalization on PET images, first calculate the mean and standard deviation of the PET image, and then normalize each pixel according to the following formula, where z is the normalized value, x is the original data point, μ is the image mean, and σ is the standard deviation.

[0097]

[0098] Next, the CT images are truncated in grayscale units (HU), limiting their values ​​to the range of [-1024, 1024]. This avoids the influence of extreme values ​​in the CT images on subsequent processing and preserves the common range of medical image data. Finally, the CT images are normalized using Min-Max Normalization, mapping their grayscale values ​​to [-1, 1] to ensure a consistent range of image data, which is helpful for the subsequent learning and processing of the neural network. Min-Max Normalization can be represented by the following formula (2), where x is the original pixel value, x′ is the normalized pixel value, and x... min and x max These are the minimum and maximum values ​​of the data, respectively. After the above steps, the CT image, PET image, and Label image are all preprocessed to a size of 144px × 144px × 144px.

[0099]

[0100] Step S2: Construct a Parallel Co-Attention (PCA) module;

[0101] The PCA module constructed in this embodiment is used to perform deep fusion of input CT and PET images. It aims to enhance the joint feature representation of multimodal images through an attention mechanism, thereby improving the segmentation effect of tumors and other pathological regions in medical images. The PCA module captures the correlation features between CT and PET images through joint learning, enhancing the model's comprehensive perception of information from different modalities. Specifically, it includes the following steps:

[0102] Step S21: Input the CT image features and PET image features into the SE module respectively, and output the feature weighted vectors of the CT image and PET image respectively;

[0103] Specifically, in this embodiment, the input CT image features F ct and PET image features F pet Their shapes are all Where B is the batch size, C is the number of channels, and D, H, and W are the depth, height, and width, respectively. Channel-level feature enhancement is performed using the Squeeze-and-Excitation Module (SE) to obtain the weighted feature vectors Q and V.

[0104] The SE module obtains global features through global average pooling. Then, it passes through a fully connected layer with a ReLU activation function to obtain... Then, the sigmoid activation function is used to generate weights se, and finally, se is multiplied element-wise with the input features to obtain the output of the entire module.

[0105]

[0106]

[0107] Among them, F ct and F pet , respectively, represent CT image features and PET image features; X represents global features; GAP represents global average pooling; T represents the feature vector output by the fully connected layer; FC represents the fully connected layer; σ1 is the ReLU activation function, σ2 is the sigmoid activation function; se represents weights; ⊙ represents element-wise multiplication; Q and V are the feature weight vectors of CT image features and PET image features, respectively.

[0108] Step S22: Calculate the affinity matrix based on the feature weighted vectors of the CT and PET images;

[0109] Affinity = tanh(W) b (Q)·W b (V)) (4)

[0110] Where Affinity is the affinity matrix; tanh is the activation function; W b This is the weight matrix. In this embodiment, the affinity matrix is ​​used to quantify the similarity between CT images and PET images, helping the model capture the correlation between them.

[0111] Step S23: Calculate the co-attention weights of CT image features and PET image features based on the affinity matrix;

[0112] H v =tanh(W v (V)+W q (Q)·Affinity) (5)

[0113] H q =tanh(W q (Q)+W v (V)·Affinity) (6)

[0114] a v =softmax(W h ·H v (7)

[0115] a q =softmax(W h ·H q (8)

[0116] Among them, H v H q These are the modality enhancement feature vectors of PET and CT images, respectively. and W v W q W h These are the weight parameters; a v and a q These are the common attention weights for PET image features and CT image features, respectively. and

[0117] Step S24: Based on the common attention weight a v and a qCalculate the weighted sums of PET and CT features separately. After weighting the features of the two modalities, obtain the following results: and As output features. By using F′ ct and F′ pet By concatenating along the channel number dimension, a fused common attention feature map is obtained.

[0118] F′ ct =∑a q ·Q (9)

[0119] F′ pet =∑a v ·V (10)

[0120] F fusion =Concat(F′ ct ,F′ pet (11)

[0121] The entire PCA module will have two shapes. The modal features are fused together, and the final output is an attention-weighted feature map that combines information from both modalities, with a shape of [shape missing].

[0122] Step S3: Insert several parallel co-attention mechanism modules into the backbone network of the UNet3D network model to construct the improved UNet3D network model;

[0123] This embodiment, after constructing the PCA module, further constructs an improved UNet3D network model. The core idea of ​​this model is to extract features by inputting CT and PET image data into two independent UNet3D encoders, and then fusing the CT and PET features at appropriate locations using a parallel common attention module (PCA). The fused features are then input into the backbone network for further processing, ultimately outputting the segmentation results for the dual-modal image task. This network model enhances the interaction between different modal features by appropriately inserting a PCA module into the encoder part of the backbone UNet3D, thereby improving the feature expressiveness and model performance. Specifically, it includes the following steps:

[0124] Step S31: Input the CT image and PET image into two independent UNet encoder stages respectively, and output the preliminary features q1, q2, q3, and q4 of the CT image and the preliminary features v1, v2, v3, and v4 of the PET image respectively.

[0125] In this embodiment, each encoder stage includes multiple 3D convolutional layers of size 3×3×3, BatchNormalization, PReLU activation function, and Dropout operation. The number of output channels in each encoding stage is 16, 32, 64, and 128, respectively.

[0126] Step S32: Insert a parallel co-attention mechanism module at the back end of each encoder stage of the UNet3D backbone network;

[0127] This embodiment applies a PCA module to the back end of each encoder stage of the backbone network, targeting the preliminary features of CT and PET images. The PCA module enhances the network's ability to represent different modal features by establishing the relationship between two modal features. Specifically, it includes the following:

[0128] The original CT and PET image data are input into the first parallel co-attention mechanism module, which outputs the fused bimodal feature F0.

[0129] The preliminary features q1 of the CT image and the preliminary features v1 of the PET image obtained in the first stage are input into the second parallel co-attention mechanism module, and the fused bimodal features F1 are output.

[0130] The preliminary CT image features q2 and preliminary PET image features v2 obtained in the second stage are input into the third parallel co-attention mechanism module, which outputs the fused bimodal features F2.

[0131] The preliminary CT image features q3 and preliminary PET image features v3 obtained in the third stage are input into the fourth parallel co-attention mechanism module, which outputs the fused dual-modal features F3.

[0132] The preliminary CT image features q4 and preliminary PET image features v4 obtained in the fourth stage are input into the fifth parallel co-attention mechanism module, which outputs the fused bimodal features F4.

[0133] Step S33: The dual-modal feature F0 and the original CT and PET image data are stitched together to form a feature vector with 4 channels, and then input into the encoder module of the first stage. After feature extraction by the encoder module, the number of channels becomes 16.

[0134] The spliced ​​feature map output from the first stage encoder module is spliced ​​with F1 and then input to the second stage encoder module. After feature extraction by the encoder module, the number of channels becomes 32.

[0135] The spliced ​​feature map output from the second-stage encoder module is spliced ​​with F2 and then input into the third-stage encoder module. After feature extraction by the encoder module, the number of channels becomes 64.

[0136] The spliced ​​feature map output from the third-stage encoder module is spliced ​​with F3 and then input into the fourth-stage encoder module. After feature extraction by the encoder module, the number of channels becomes 128.

[0137] The spliced ​​feature map output from the fourth stage encoder module is spliced ​​with F4 and then input into the fifth stage encoder module. After feature extraction by the encoder module, the number of channels becomes 256.

[0138] Step S34: In the decoder section, the output of the encoder module in the fifth stage is deconvolved to restore the number of channels from 256 to 128, and the output is concatenated with the feature map of the encoder module in the fourth stage, and features are extracted through convolution operation.

[0139] The output of the fourth-stage decoder module is deconvolutioned to restore the number of channels from 128 to 64, and this output is concatenated with the feature map of the third-stage encoder module. Features are then extracted through convolution.

[0140] The output of the third-stage decoder module is deconvolutioned to restore the number of channels from 64 to 32, and this output is concatenated with the feature map of the second-stage encoder module. Features are then extracted through convolution.

[0141] The output of the second-stage decoder module is deconvolutioned to restore the number of channels from 32 to 16, and this output is concatenated with the feature map of the first-stage encoder module. Features are then extracted through convolution.

[0142] The output of the first-stage decoder module is passed through a 1x1x1 convolutional layer to reduce the number of channels from 16 to 2, and the segmentation task result is output.

[0143] At this point, the network model architecture has been constructed. Through this network structure design, CT and PET image features are fully fused and interacted at each encoder stage. The introduction of the PCA module enhances the feature complementarity between CT and PET, helping to improve the network's representation ability and segmentation performance for dual-modal images. In the backbone network, through layer-by-layer feature extraction, fusion, decoding, and reconstruction, a final, highly efficient segmentation result is achieved.

[0144] Step S4: Optimize and improve the UNet3D network model using a combined loss function, which specifically includes the following:

[0145] To optimize the network training process, this embodiment employs a combined loss function, including the Soft Dice Loss function and the weighted Focal Loss function. Soft Dice Loss effectively addresses class imbalance in the data by measuring the overlap between predicted results and true labels to optimize model performance. Focal Loss, by introducing α and γ parameters, further enhances attention to hard-to-classify samples and reduces the impact of easily classified samples on gradient updates. The specific definition of the combined loss function is as follows:

[0146] Loss=Soft Dice Loss+0.5×Focal Loss (12)

[0147] Focal Loss is defined as:

[0148] FocalLoss(p)=-α(1-p) γ log(p) (13)

[0149] The parameters α = 0.5 and γ = 3 can be chosen to adjust the weights of difficult-to-classify and easy-to-classify samples, making the network more stable when dealing with class imbalance.

[0150] This invention employs the Adam optimizer and introduces a cosine annealing learning rate scheduling strategy during training. This strategy dynamically adjusts the learning rate to avoid getting trapped in local optima. Specifically, the learning rate decreases periodically with increasing training epochs, with an initial learning rate of 1×10⁻⁶. -4 The minimum learning rate is set to 1×10. -9 The model is trained in multiple rounds. In each round, the predicted value is calculated through forward propagation, and automatic mixed precision (AMP) technology is used to reduce memory usage and improve training efficiency.

[0151] Step S5: Input the preprocessed head and neck tumor dataset into the optimized and improved UNet3D network model, update the network weight parameters, and output the segmentation results of the dual-modal image.

[0152] Example 2

[0153] This embodiment is based on the method steps of Embodiment 1, and uses the head and neck tumor (HECKTOR) dataset released by MICCAI in 2022 as an example to clearly and completely describe the technical solution in this embodiment of the invention, which specifically includes the following:

[0154] Dataset Introduction: The HECKTOR 2022 dataset is a medical image segmentation dataset focusing on head and neck PET / CT data. It contains 524 case training data from 7 different data centers. Each case includes CT images, PET images, and ground truth labels (including tumors and lymph nodes) annotated by professional physicians, and is saved in NIFTI format. The original image information of the dataset is shown in Table 1, where the grayscale values ​​of the PET images are the average values ​​from all data centers.

[0155] Table 1 Original information of the dataset

[0156]

[0157] Step S1: Refer to Figure 1 The obtained HECKTOR 2022 dataset will undergo preprocessing operations, specifically including three parts: resampling, image cropping, and image normalization.

[0158] Step S11: Resampling;

[0159] The spatial voxel spacing of the input CT, PET, and Label images is resampled to 1mm×1mm×1mm, and the spatial coordinates of the CT image are used as a unified reference for the three images to ensure consistent voxel spacing and aligned spatial coordinate systems, resulting in precise spatial matching. The specific steps are as follows: First, for the CT and PET images, triple linear interpolation is used for resampling to maintain the smoothness and continuity of image details, ensuring no loss of grayscale information and fine structure during resampling. For the Label image, since it contains discrete category values, nearest neighbor interpolation is used to avoid category value confusion, ensuring that the category values ​​remain unchanged after resampling and avoiding the introduction of non-integer intermediate values ​​during interpolation. During the resampling process, the spatial orientation, origin, and coordinate system of the CT image are uniformly adopted, and the size of the resampled image is adjusted accordingly based on its original voxel spacing to ensure consistent spatial resolution and strict alignment within the same spatial coordinate system for all three images after resampling.

[0160] Step S12: Image cropping;

[0161] The resampled Label image is cropped. First, pixels with a label value of 2 (lymph nodes) are relabeled as background with a value of 0, retaining only the background (value 0) and the tumor (value 1). Then, the spatial dimensions of the CT image, PET image, and Label image are cropped to reduce image size, alleviate computational burden on subsequent model training, and retain the effective information region containing the tumor. Specific steps include: first, determining the bounding box of the tumor by detecting its spatial location in the Label image; calculating the tumor's center point based on the bounding box position; and then cropping the CT image, PET image, and Label image to a fixed size of 144px × 144px × 144px using this center point as a reference, ensuring the tumor region is completely preserved. Simultaneously, image boundaries are considered during cropping to ensure the cropping operation does not exceed the actual image size. If the cropped area is smaller than expected, the starting point is adjusted appropriately to ensure the final cropped image and label meet the predetermined size requirements.

[0162] Step S13: Image normalization;

[0163] The PET image is z-score normalized, the mean and standard deviation of the PET image are calculated, and each pixel is normalized according to formula (1).

[0164] Next, the CT images are truncated in grayscale units (HU), limiting their values ​​to the range of [-1024, 1024]. This avoids the influence of extreme values ​​in the CT images on subsequent processing, preserving the common range of medical image data. Finally, the CT images are normalized using Min-Max Normalization, mapping their grayscale values ​​to [-1, 1] to ensure a consistent range of image data. The Min-Max Normalization formula can be found in formula (2). After the above steps, the CT images, PET images, and Label images are all preprocessed to a size of 144px × 144px × 144px.

[0165] After the above operations, the case information provided by each data center was processed to a uniform voxel spacing, uniform shape and size, and uniform grayscale threshold. Figure 2 Table 2 shows the comparison before and after preprocessing of a case in the dataset. The grayscale value of PET is the overall mean.

[0166] Table 2. Dataset information after preprocessing.

[0167]

[0168]

[0169] Step S2: Construct the Parallel Co-Attention (PCA) module, refer to... Figure 3 The algorithm steps of the parallel common attention mechanism module are as follows:

[0170] Step S21: Input image feature processing;

[0171] For the input CT image features F ct and PET image features F pet Their shapes are all Where B is the batch size, C is the number of channels, and D, H, and W are the depth, height, and width, respectively. Channel-level feature enhancement is performed through the SE module (Squeeze-and-Excitation Module) to obtain the weighted feature vectors Q and V. The structure diagram of the SE module is shown below. Figure 4 As shown, the calculation formula can be found in formula (3), and the global features are obtained through global average pooling. Then, it passes through a fully connected layer with a ReLU activation function to obtain... Then, the weights se are generated by the sigmoid activation function, and finally, se is multiplied element-wise with the input features to obtain the output of the entire module.

[0172] Step S22: Affinity matrix calculation;

[0173] The affinity matrix is ​​calculated using formula (4) for the Q and V features obtained after processing the SE module. This affinity matrix is ​​used to quantify the similarity between CT and PET, helping the model capture the correlation between the two.

[0174] Step S23, Calculation of co-attention features: Further interactive operations are performed on CT and PET features to calculate their co-attention features. These features are then calculated using formulas (5)-(8) in conjunction with the affinity matrix. and Next, the attention weights for the two modalities are calculated. and Normalization is performed within each modality using the softmax function.

[0175] Step S24, Attention-weighted feature generation: Finally, based on the co-attention weight a v and a q Calculate the weighted sums of PET and CT features separately. After weighting the features of the two modalities, refer to formulas (9)-(11) to obtain the results. and As output features. By using F′ ct and F′ pet By concatenating along the channel number dimension, a fused common attention feature map is obtained.

[0176] The entire PCA module will have two shapes. The modal features are fused together, and the final output is an attention-weighted feature map that combines information from both modalities, with a shape of [shape missing].

[0177] Step S3: Construct an improved network model, improve the framework structure, and refer to... Figure 5 Specifically, it includes the following:

[0178] Step S31, as follows Figure 6 As shown, CT and PET image data are input into two independent UNet encoder stages. CT data undergoes feature extraction in each stage to obtain preliminary CT image features q1 to q4, and PET data is similarly processed through independent feature extraction modules to obtain preliminary features v1 to v4. These features will be fused in the subsequent PCA module. Each encoder stage contains multiple 3×3×3 3D convolutional layers, Batch Normalization, PReLU activation function, and Dropout operation. The number of output channels for each encoding stage is 16, 32, 64, and 128, respectively.

[0179] Step S32: Based on the preliminary features of CT and PET images, PCA modules are applied at the back end of each encoder stage of the backbone network. The first PCA module directly processes the raw CT and PET data, outputting the fused bimodal feature F0. Subsequently, after feature extraction in the first stage, the features (q1 and v1) obtained from the CT and PET images are input into the second PCA module to obtain the bimodal feature F1. Similarly, after feature extraction in the second, third, and fourth stages, the CT and PET features are input into the third, fourth, and fifth PCA modules, respectively, to obtain F2, F3, and F4, achieving cross-modal fusion of CT and PET features at each stage. By establishing the interrelationship between the two modal features, the PCA module enhances the network's ability to represent different modal features.

[0180] Step S33: In the backbone UNet3D network, the fused CT and PET features F0 and their original data are first concatenated to form a feature vector with 4 channels, which is then input into the first-stage encoder module. After feature extraction, the number of channels becomes 16, and it is then concatenated with F1 and input into the next encoder stage. Next, this concatenated feature is input into the second-stage encoder, where the number of feature channels further increases to 32. Similarly, the feature map processed by the second-stage encoder is concatenated with F2 and input into the third-stage encoder, where the number of channels increases to 64. This process is performed sequentially in each stage to ensure that the cross-modal fusion features of the PCA module can be combined in each stage, and the number of feature channels increases to 256 at the end of the encoder stage.

[0181] Step S34: After the encoder stage of the backbone UNet3D network is completed, the decoder stage begins. In the decoder, the spatial resolution of the feature maps is gradually restored through deconvolution. Specifically, the output of the last layer of the encoder is first deconvolved to restore the number of channels from 256 to 128, then concatenated with the features from the fourth stage encoder, and further features are extracted through convolution. Subsequently, similar operations are repeated in each stage of the decoder to gradually restore the resolution of the feature maps, and the output of each stage is concatenated with the feature maps of the corresponding stage of the encoder. Finally, after processing by the decoder, the number of channels is reduced to 2 through a 1x1x1 convolutional layer, and then passed through a sigmoid activation function, with the output being the final predicted segmentation map.

[0182] Step S4: Input the preprocessed data into the network model constructed above for processing, obtain the predicted segmentation results, verify the training effect and evaluate the performance of the model, and save the optimal trained network model.

[0183] In this invention, comparative experiments were designed and conducted to evaluate the effectiveness of the proposed PCA-UNet model. Using UNet3D as the baseline network, the experiments compared and analyzed inputs from CT monomodal, PET monomodal, and PET-CT bimodal networks. Multiple evaluation metrics were employed, including DSC (Dice Similarity Coefficient), HD (Hausdorff Distance), Precision, and Recall, to comprehensively measure the model's segmentation performance. Simultaneously, an experiment was designed to introduce a PCA module before the baseline network input layer to explore the improvement in model performance brought about by the PCA module.

[0184] The following metrics were used to evaluate the network performance throughout the experiment:

[0185] The Dice similarity coefficient (DSC) measures the degree of overlap between the predicted segmented region and the ground truth labeled region. The formula for calculation is:

[0186]

[0187] Where P represents the predicted segmentation region and T represents the true labeled region. The higher the DSC value, the closer the segmentation result is to the true tumor region.

[0188] The Hausdorff distance (HD), used to evaluate the maximum distance difference between the predicted segmentation boundary and the true boundary, is defined as:

[0189] HD(P,T)=max{sup p∈P inf t∈T ∥pt∥,sup t∈T inf p∈P ∥tp∥} (15)

[0190] Where P represents the set of points in the predicted segmentation region, i.e. the set of boundary points of the tumor region generated by the model, T represents the set of points in the actual segmentation region, i.e. the set of boundary points of the actual tumor region, ∥pt∥ represents the Euclidean distance between two points p and t, sup represents the maximum distance in the set of points (supremum, i.e. supremum), and inf represents the minimum distance in the set of points (infimum, i.e. infimum).

[0191] Precision, used to measure the accuracy of a model in predicting the positive class, is defined as:

[0192]

[0193] In this model, TP represents true positives and FP represents false positives. Higher precision indicates better segmentation accuracy.

[0194] Recall is used to evaluate the coverage of a model on positive class samples, and is calculated using the following formula:

[0195]

[0196] In this model, TP represents true positives and FN represents false negatives. A higher recall rate indicates that the model can capture more tumor regions.

[0197] Implementation details of this experiment: The experiment is based on Python 3.8, PyTorch 2.3.0, and CUDA 11.8, using a 16GB NVIDIA 4060TI GPU, with a batch size of 3, a training epoch count of 300, and the Adam optimizer. A cosine annealing learning rate scheduling strategy was introduced during training. This strategy dynamically adjusts the learning rate to avoid getting trapped in local optima. Specifically, the learning rate decreases periodically with increasing training epochs, with an initial learning rate of 1×10⁻⁶. -4 The minimum learning rate is set to 1×10. -9 The model is trained in multiple rounds. In each round, the predicted value is calculated through forward propagation, and automatic mixed precision (AMP) technology is used to reduce memory usage and improve training efficiency.

[0198] To optimize the network training process, this example employs a combined loss function of SoftDice Loss + 0.5×Focal Loss, which includes Soft Dice Loss and weighted Focal Loss. Soft Dice Loss effectively addresses class imbalance in the data by optimizing model performance by measuring the overlap between predicted results and true labels. Focal Loss, by introducing α and γ parameters, further enhances attention to hard-to-classify samples and reduces the impact of easily classified samples on gradient updates.

[0199] All models were trained and evaluated under the same settings to ensure the fairness and reproducibility of the comparative experiments. The results of the comparative experiments are shown in the table below.

[0200] Table 3 Experimental Results

[0201]

[0202] In the table, PCA-UNet represents the model constructed in this invention, while PCA+UNet3D represents the model that incorporates a PCA module before the input layer of the baseline network. The PCA-UNet model proposed in this invention outperforms both the baseline model UNet3D and the improved model PCA+UNet3D in all metrics. Specifically, PCA-UNet improves the DSC metric by approximately 3% compared to the best result of the baseline UNet3D, demonstrating a significant improvement in segmentation accuracy. In terms of HD, PCA-UNet exhibits a smaller error distance, with an optimization margin of approximately 7%, showing its improvement in boundary accuracy. For Precision and Recall, PCA-UNet improves by approximately 1% and 4%, respectively, further enhancing the model's ability to identify and recall tumor regions. The total number of parameters increased only by approximately 1 million from 5,648,386 in the baseline network to 6,650,262 in PCA-UNet. Overall results show that PCA-UNet can better integrate multimodal information, effectively improve the model's segmentation performance and boundary clarity, and fully verify its advantages in head and neck tumor segmentation.

[0203] Step S5: Input the preprocessed head and neck tumor CT and PET images of the patient into the trained optimal network model, update the network weight parameters, and obtain the optimal network segmentation result.

[0204] like Figure 7 The image shows a comparison of tumor segmentation using the optimal models trained from various models. The segmentation comparison results demonstrate that the PCA-UNet model exhibits superior detail restoration and contour matching in tumor segmentation. Specifically, PCA-UNet generates more refined and smoother edges during segmentation, and the segmented regions more closely resemble the actual tumor contours, effectively capturing subtle edge features. In contrast, the baseline network, under single-modal input, especially in PET and CT single-modal results, shows significant detail loss and fails to accurately characterize tumor edges. Thus, PCA-UNet is not only better adapted to multimodal information but also significantly improves the fineness and accuracy of segmentation, better serving the precise segmentation needs in practical applications.

[0205] Although specific embodiments of the invention have been described in detail with reference to the accompanying drawings, this should not be construed as limiting the scope of protection of this patent. Various modifications and variations that can be made by a person skilled in the art without inventive effort within the scope described in the claims still fall within the scope of protection of this patent.

Claims

1. A method for segmenting head and neck tumor images based on combined CT and PET analysis, characterized in that, Includes the following steps: S1. Obtain a head and neck tumor dataset containing CT images, PET images, and segmented labeled images, and preprocess the head and neck tumor dataset. S2. Construct a parallel co-attention mechanism module; S2 includes the following sub-steps: S21. Input the CT image features and PET image features into the SE module respectively, and output the feature weighted vectors of the CT image and PET image respectively; S22. Calculate the affinity matrix based on the feature weighted vectors of CT and PET images; In step S22, the affinity matrix is ​​calculated, including: Where Affinity is the affinity matrix; tanh is the activation function; Here, Q is the weight matrix; Q and V are the feature weight vectors of CT image features and PET image features, respectively. S23. Based on the affinity matrix, calculate the common attention weights of CT image features and PET image features respectively; S24. Calculate the weighted sum of CT image features and PET image features based on the co-attention weights of CT image features and PET image features respectively, and then concatenate the weighted sums of CT image features and PET image features to obtain the fused co-attention feature map. S3. Insert several parallel co-attention mechanism modules into the backbone network of the UNet3D network model to construct an improved UNet3D network model; S3 includes the following sub-steps: S31. Input the CT image and PET image into two separate UNet encoder stages, and output the preliminary features of the CT image respectively. , , ,and and output preliminary features of PET images , , and ; S32. Insert parallel co-attention mechanism modules at the back end of each encoder stage of the UNet3D backbone network; The original CT and PET image data are input into the first parallel co-attention mechanism module, which outputs the fused bimodal features. ; Preliminary features of CT images obtained in the first stage and preliminary features of PET images The input is fed into the second parallel co-attention mechanism module, which outputs the fused bimodal features. ; Preliminary features of CT images obtained in the second stage and preliminary features of PET images The input is fed into the third parallel co-attention mechanism module, which outputs the fused bimodal features. ; Preliminary features of CT images obtained in the third stage and preliminary features of PET images The input is fed into the fourth parallel co-attention mechanism module, which outputs the fused bimodal features. ; Preliminary features of CT images obtained in the fourth stage and preliminary features of PET images The input is fed into the fifth parallel co-attention mechanism module, which outputs the fused bimodal features. ; S4. An improved UNet3D network model optimized using a combined loss function; S5. Input the preprocessed head and neck tumor dataset into the optimized and improved UNet3D network model to output the segmentation results of the bimodal images.

2. The head and neck tumor image segmentation method based on CT and PET combined analysis according to claim 1, characterized in that, The preprocessing of the neck tumor dataset in S1 includes the following steps: S11. The CT and PET images are resampled using triple linear interpolation, and the label image is resampled using nearest neighbor interpolation. During sampling, the size of the resampled images is adjusted based on the spatial orientation, origin, and coordinate system of the CT image until the voxel spacing and spatial coordinate system of the CT, PET, and label images are consistent. S12. Detect the spatial location of the tumor in the label image, determine the bounding box where the tumor is located, calculate the center point of the tumor based on the position of the bounding box, and crop the CT image, PET image and label image to 144px×144px×144px based on the center point. S13. Perform z-score normalization on the PET images, and perform grayscale truncation and maximum-minimum normalization on the CT images in sequence.

3. The head and neck tumor image segmentation method based on CT and PET combined analysis according to claim 1, characterized in that, In step S21, the output feature weighted vectors of the CT and PET images include: in, and These represent CT image features and PET image features, respectively; X represents global features; GAP represents global average pooling; T represents the feature vector output by the fully connected layer; FC represents the fully connected layer. It is the ReLU activation function. It is the sigmoid activation function; se is the weight; This is an element-wise multiplication operation.

4. The head and neck tumor image segmentation method based on CT and PET combined analysis according to claim 3, characterized in that, In step S23, the common attention weights of CT image features and PET image features are calculated respectively, including: in, , These are the modal enhancement feature vectors for PET and CT images, respectively; , , These are the weight parameters; and These are the co-attention weights for PET image features and CT image features, respectively.

5. The head and neck tumor image segmentation method based on combined CT and PET analysis according to claim 4, characterized in that, In step S24, the fused common attention feature map is obtained, including: in, and The output features are weighted sums of CT image features and PET image features; For splicing operations; This is the fused co-attention feature map.

6. The head and neck tumor image segmentation method based on CT and PET combined analysis according to claim 1, characterized in that, S3 further includes the following steps: S33, Dual-modal features The original CT and PET image data are stitched together to form a feature vector with 4 channels, which is then input into the encoder module in the first stage. After feature extraction by the encoder module, the number of channels becomes 16. The concatenated feature map output by the first-stage encoder module and After being spliced, the data is input into the encoder module in the second stage. After feature extraction by the encoder module, the number of channels becomes 32. The concatenated feature map output by the second-stage encoder module and After being spliced, the signal is input into the encoder module of the third stage. After feature extraction by the encoder module, the number of channels becomes 64. The concatenated feature map output by the third-stage encoder module and After being spliced, the signal is input into the encoder module of the fourth stage. After feature extraction by the encoder module, the number of channels becomes 128. The concatenated feature map output by the fourth-stage encoder module and After being spliced, the signal is input into the encoder module of the fifth stage. After feature extraction by the encoder module, the number of channels becomes 256. S34. In the decoder section, the output of the encoder module in the fifth stage is deconvolutioned to restore the number of channels from 256 to 128, and this output is concatenated with the feature map of the encoder module in the fourth stage, and features are extracted through convolution operation. The output of the fourth-stage decoder module is deconvolutioned to restore the number of channels from 128 to 64, and this output is concatenated with the feature map of the third-stage encoder module. Features are then extracted through convolution. The output of the third-stage decoder module is deconvolutioned to restore the number of channels from 64 to 32, and this output is concatenated with the feature map of the second-stage encoder module. Features are then extracted through convolution. The output of the second-stage decoder module is deconvolutioned to restore the number of channels from 32 to 16, and this output is concatenated with the feature map of the first-stage encoder module. Features are then extracted through convolution. The output of the first-stage decoder module is passed through a 1x1x1 convolutional layer to reduce the number of channels from 16 to 2, and the segmentation task result is output.

7. The head and neck tumor image segmentation method based on CT and PET combined analysis according to claim 1, characterized in that, In S4, the combined loss function includes the Soft Dice Loss function and the weighted Focal Loss function, specifically as follows: in, This is the combined loss function; The confidence level of the model's prediction; and For parameters.

Images