A method for image segmentation of head and neck tumor lesion regions and a computer-readable medium

Abstract

This invention proposes a method for image segmentation of head and neck tumor lesion regions and a computer-readable medium. The invention acquires multiple sets of original head and neck tumor PET-CT images, sequentially preprocesses them to obtain preprocessed images for each set, and labels them with corresponding ground truth classification tags. A lesion image segmentation network is constructed, and lesion segmentation prediction is performed using the input of each preprocessed image set to obtain a head and neck tumor prediction probability map for each preprocessed image set. Combining the ground truth classification tag of each pixel of the head and neck tumor lesion in each preprocessed image set, a cross-entropy Dessell weighted loss function is constructed, and the network is optimized and trained using a stochastic gradient descent algorithm to obtain a trained lesion image segmentation network. Real-time acquired head and neck tumor PET-CT images are then used to predict and segment the lesion region using the trained lesion image segmentation network, and probability thresholds are determined to obtain the real-time pixel range of the head and neck tumor lesion region. This invention utilizes the complementary information between multiple modalities to improve the accuracy of pixel region segmentation prediction for head and neck tumor lesions.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and in particular relates to a method for image segmentation of head and neck tumor lesion areas and a computer-readable medium. Background Technology

[0002] Head and neck tumors are the fifth most common type of cancer worldwide. Currently, PET-CT, a comprehensive imaging modality, combines positron emission tomography (PET) and computed tomography (CT). PET provides molecular information such as the function and metabolism of the lesions, while CT provides precise anatomical localization, thus aiding in the diagnosis of head and neck tumors. In clinical practice, doctors primarily assess tumor conditions by observing the location, shape, size, and boundaries of the lesions to formulate treatment plans. Segmentation of head and neck tumor lesions assists doctors in clinical diagnosis and is therefore an important issue in medical image analysis. The precise segmentation of head and neck tumor lesions in PET-CT images remains to be solved, mainly for the following reasons:

[0003] PET and CT images provide different biological and anatomical information; however, effectively fusing these two different modalities remains a challenging problem.

[0004] Due to varying degrees of lesion, head and neck tumors exhibit significant heterogeneity in shape, size, and intensity. The tumor may have blurred boundaries or overlapping areas with surrounding tissues, which increases the complexity of segmentation.

[0005] Some lesions have indistinct boundaries, and different radiologists may label different areas of the lesion inconsistently. In addition, manual labeling relies on the expertise and experience of imaging specialists, is time-consuming and labor-intensive, and may introduce subjective differences among physicians.

[0006] Therefore, establishing an accurate automatic segmentation method for head and neck tumor lesions is of great significance for clinical diagnosis and disease management. In recent years, deep learning-based methods have demonstrated good performance in solving various computer vision problems (such as image classification, object detection, and semantic segmentation). Many methods have been applied to head and neck tumor lesion segmentation and have achieved impressive segmentation results.

[0007] Convolutional neural networks, exemplified by U-Net, feature a classic symmetric encoder and decoder structure and preserve high-resolution details through skip connections. Due to their strong feature extraction and inductive bias capabilities on smaller datasets, they have achieved remarkable results in medical image segmentation tasks. The advantage of multimodal medical images lies in their ability to provide diverse information, leading to more accurate lesion identification. However, simply stitching together multimodal medical images and using this as input to a U-shaped network fails to fully utilize the complementary information between the different modalities. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention proposes a method for image segmentation of head and neck tumor lesion regions and a computer-readable medium.

[0009] The technical solution of this invention is an image segmentation method for head and neck tumor lesion regions, characterized in that:

[0010] A lesion image segmentation network is constructed. Each group of preprocessed images is input into the lesion image segmentation network for prediction to construct the cross-entropy Dessell weighted loss function. The trained lesion image segmentation network is obtained by optimizing the training through the stochastic gradient descent algorithm.

[0011] The predicted probability map of head and neck tumors in real-time PET-CT images is obtained by predicting and segmenting the lesion image after training the lesion image segmentation network. The pixel range of the head and neck tumor lesion area in the real-time PET-CT image is further determined by combining the probability threshold.

[0012] The technical solution of this invention specifically includes the following steps:

[0013] Step 1: Acquire multiple sets of original head and neck tumor PET-CT images. Perform cropping, normalization, stitching, and data enhancement processing on each set of original head and neck tumor PET-CT images in sequence to obtain pre-processed images for each set. Mark the true classification label of the head and neck tumor lesion for each pixel of each pre-processed image.

[0014] Step 2: Construct a lesion image segmentation network. Input each group of preprocessed images into the lesion image segmentation network to perform lesion segmentation prediction and obtain the head and neck tumor prediction probability map of each group of preprocessed images. Combine the true classification label of each pixel of the head and neck tumor lesion in each group of preprocessed images to construct the cross-entropy Dessell weighted loss function. Optimize the training through the stochastic gradient descent algorithm to obtain the trained lesion image segmentation network.

[0015] Step 3: The real-time acquired head and neck tumor PET-CT images are predicted and segmented by the trained lesion image segmentation network to obtain the head and neck tumor prediction probability map of the real-time head and neck tumor PET-CT images. The pixel range of the head and neck tumor lesion area in the real-time head and neck tumor PET-CT images is determined by combining the probability threshold.

[0016] Preferably, the lesion image segmentation network in step 2 includes:

[0017] Single-modal coding network, fusion coding network, single-modal decoding network, fusion decoding network;

[0018] The single-modal coding network performs feature extraction processing on each group of preprocessed images to obtain intermediate feature representations and final feature representations of each group of preprocessed images. The intermediate feature representations of each group of preprocessed images are output to the fusion coding network, and the final feature representations of each group of preprocessed images are output to the single-modal decoding network.

[0019] The fusion coding network performs feature fusion processing on the intermediate feature representations of each group of preprocessed images to obtain the fusion coding features of each group of preprocessed images, and outputs the fusion coding features of each group of preprocessed images to the fusion decoding network.

[0020] The single-mode decoding network decodes the final feature representation of each preprocessed image to obtain the decoded feature representation of each preprocessed image, and outputs the decoded feature representation of each preprocessed image to the fusion decoding network.

[0021] The fusion decoding network performs fusion decoding on the decoding feature representations of each group of preprocessed images to finally obtain the predicted lesion region segmentation image of each preprocessed image.

[0022] The single-modal coding network includes:

[0023] The first encoding module, the second encoding module, ..., the Kth encoding module, the bottleneck module;

[0024] The first encoding module, the second encoding module, ..., the Kth encoding module, and the bottleneck module are sequentially cascaded together.

[0025] The kth encoding module is composed of multiple convolutional modules and downsampling layers cascaded together, k∈[1,K];

[0026] The bottleneck module is composed of multiple convolutional modules;

[0027] The fusion coding network includes:

[0028] The first fusion coding module, the second fusion coding module, ..., the Kth fusion coding module, and the bottleneck module;

[0029] The first fusion coding module, the second fusion coding module, ..., the Kth fusion coding module, and the bottleneck module are sequentially cascaded together.

[0030] The kth fusion coding module is composed of a multi-layer convolutional module, an attention module, and a downsampling layer cascaded in sequence, where k∈[1,K].

[0031] The bottleneck module is composed of multiple convolutional modules;

[0032] The single-mode decoding network includes:

[0033] The first decoding module, the second decoding module, ..., the Kth decoding module;

[0034] The first decoding module, the second decoding module, ..., the Kth decoding module are cascaded sequentially.

[0035] The kth decoding module is composed of an upsampling layer, a splicing layer, and a multi-layer convolutional module cascaded in sequence, where k∈[1,K].

[0036] The fusion decoding network includes:

[0037] The first fusion decoding module, the second fusion decoding module, ..., the Kth fusion decoding module;

[0038] The first fusion decoding module, the second fusion decoding module, ..., the Kth fusion decoding module are cascaded sequentially.

[0039] The kth fusion decoding module is composed of an upsampling layer, a splicing layer, and an inverted bottleneck convolution module cascaded in sequence, where k∈[1,K].

[0040] The first encoding module passes each modality of each preprocessed image through a multi-layer convolution module to obtain the first-stage features of each modality of each preprocessed image, and outputs the first-stage features of each modality of each preprocessed image to the first downsampling layer, the first fusion encoding module, and the first decoding module.

[0041] After outputting the first-stage features of each modality of each preprocessed image to the first downsampling layer, the downsampling features of each modality of each preprocessed image in the first stage are obtained. The downsampling features of each modality of each preprocessed image in the first stage are then output to the second encoding module.

[0042] If k∈[2,K-1], the k-th encoding module extracts the downsampled features of each modality at stage k-1 of each preprocessed image through a multi-layer convolution module to obtain the k-th stage features of each modality of each preprocessed image, and outputs the k-th stage features of each modality of each preprocessed image to the k-th downsampling layer, the k-th fusion encoding module, and the k-th decoding module respectively.

[0043] After outputting the k-th stage features of each modality of each preprocessed image to the k-th downsampling layer, the downsampling features of each modality of each preprocessed image at the k-th stage are obtained. The downsampling features of each modality of each preprocessed image at the k-th stage are then output to the (k+1)-th encoding module.

[0044] The Kth encoding module extracts the downsampled features of each modality in the (K-1)th stage of each preprocessed image through a multi-layer convolution module to obtain the Kth stage features of each modality in each preprocessed image. The Kth stage features of each modality in each preprocessed image are then output to the Kth downsampling layer, the Kth fusion encoding module, and the Kth decoding module, respectively.

[0045] After outputting the features of each modality at stage K of each preprocessed image to the Kth downsampling layer, the downsampling features of each modality at stage K of each preprocessed image are obtained, and the downsampling features of each modality at stage K of each preprocessed image are output to the bottleneck module.

[0046] The bottleneck module extracts the downsampled features of each modality at stage K of each preprocessed image through a multi-layer convolution module to obtain the bottleneck layer features of each modality of each preprocessed image, and outputs the bottleneck layer features of each modality of each preprocessed image to the Kth decoding module.

[0047] The first fusion coding module concatenates the modalities of each preprocessed image in the channel direction and passes them through a multi-layer convolution module to obtain the first-stage features of each preprocessed image. The first-stage features of each preprocessed image and the first-stage features of each modality of each preprocessed image are input into the first attention module to obtain the first-stage fusion features of each preprocessed image. The first-stage fusion features of each preprocessed image are output to the first downsampling layer to obtain the first-stage downsampled fusion features of each preprocessed image. The first-stage downsampled fusion features of each preprocessed image are output to the second fusion coding network.

[0048] If k∈[2,K-1], the k-th fusion coding module takes the (k-1)-th stage fusion features of each group of preprocessed images through a multi-layer convolution module to obtain the k-th stage features of each group of preprocessed images. The k-th stage features of each group of preprocessed images and the k-th stage features of each modality of each group of preprocessed images are input to the k-th attention module to obtain the k-th stage fusion features of each group of preprocessed images. The k-th stage fusion features of each group of preprocessed images are output to the k-th downsampling layer to obtain the k-th stage downsampled fusion features of each group of preprocessed images. The k-th stage downsampled fusion features of each group of preprocessed images are output to the (k+1)-th fusion coding network.

[0049] The Kth fusion coding module takes the K-1th stage fusion features of each group of preprocessed images and passes them through a multi-layer convolution module to obtain the Kth stage features of each group of preprocessed images. The Kth stage features of each group of preprocessed images and the Kth stage features of each modality of each group of preprocessed images are input to the Kth attention module to obtain the Kth stage fusion features of each group of preprocessed images. The Kth stage fusion features of each group of preprocessed images are output to the Kth downsampling layer to obtain the Kth stage downsampled fusion features of each group of preprocessed images. The Kth stage downsampled fusion features of each group of preprocessed images are output to the bottleneck module.

[0050] The bottleneck module extracts the downsampled fusion features of the Kth stage of each preprocessed image through a multi-layer convolution module to obtain the bottleneck layer fusion features of each preprocessed image, and outputs the bottleneck layer fusion features of each preprocessed image to the Kth fusion decoding module.

[0051] The Kth decoding module upsamples the bottleneck layer features of each modality of each preprocessed image to obtain the upsampled features of the Kth stage of each modality of each preprocessed image. The upsampled features of the Kth stage of each modality of each preprocessed image are concatenated with the features of the Kth stage of each modality of each preprocessed image by channel dimension. Then, the upsampled concatenated features of the Kth stage of each modality of each preprocessed image are obtained by passing through a multi-layer convolution module. The upsampled concatenated features of the Kth stage of each modality of each preprocessed image are output to the (K-1)th decoding module and the (K-1)th fusion decoding module, respectively.

[0052] If k∈[2,K-1], the k-th decoding module upsamples the upsampled stitching features of each modality at stage k+1 of each preprocessed image to obtain the upsampled features of each modality at stage k of each preprocessed image. The upsampled features of each modality at stage k of each preprocessed image are then stitched together with the features of each modality at stage k of each preprocessed image along the channel dimension. The upsampled stitching features of each modality at stage k of each preprocessed image are then obtained through a multi-layer convolution module. The upsampled stitching features of each modality at stage k of each preprocessed image are then output to the (k-1)-th decoding module and the (k-1)-th fusion decoding module, respectively.

[0053] The first decoding module upsamples the upsampled stitching features of each modality in the second stage of each preprocessed image to obtain the upsampled features of each modality in the first stage of each preprocessed image. It then stitches the upsampled features of each modality in the first stage of each preprocessed image with the features of each modality in the first stage of each preprocessed image along the channel dimension. Finally, it obtains the upsampled stitching features of each modality in the first stage of each preprocessed image through a multi-layer convolution module and outputs the upsampled stitching features of each modality in the first stage of each preprocessed image to the first fusion decoding module.

[0054] The Kth fusion decoding module upsamples the bottleneck layer fusion features of each group of preprocessed images to obtain the Kth stage upsampled fusion features of each group of preprocessed images. It then concatenates the Kth stage upsampled fusion features of each group of preprocessed images with the Kth stage upsampled stitching features of each modality of each group of preprocessed images along the channel dimension. Finally, it passes through the Kth inverted bottleneck convolution module to obtain the Kth stage upsampled stitching fusion features of each group of preprocessed images. Finally, it outputs the Kth stage upsampled stitching fusion features of each group of preprocessed images to the (K-1)th fusion decoding module.

[0055] If k∈[2,K-1], the k-th fusion decoding module upsamples the upsampled splicing fusion features of the (k+1)-th stage of each group of preprocessed images to obtain the upsampled fusion features of each modality at the k-th stage of each group of preprocessed images. The upsampled fusion features of the k-th stage of each group of preprocessed images are spliced ​​together with the upsampled splicing features of each modality at the k-th stage of each group of preprocessed images along the channel dimension. Then, the upsampled splicing fusion features of the k-th stage of each group of preprocessed images are obtained through the k-th inverted bottleneck convolution module. Finally, the upsampled splicing fusion features of the k-th stage of each group of preprocessed images are output to the (k-1)-th fusion decoding module.

[0056] The first fusion decoding module upsamples the second-stage upsampled splicing fusion features of each group of preprocessed images to obtain the first-stage upsampled fusion features of each modality of each group of preprocessed images. The first-stage upsampled fusion features of each group of preprocessed images are spliced ​​with the first-stage upsampled splicing features of each modality of each group of preprocessed images along the channel dimension. Then, the first inverted bottleneck convolution module obtains the first-stage upsampled splicing fusion features of each group of preprocessed images. The first-stage upsampled splicing fusion features of each group of preprocessed images are convolved to obtain the head and neck tumor prediction probability map of each group of preprocessed images.

[0057] The cross-entropy Dessell weighted loss function described in step 2 is defined as follows:

[0058] Loss=α×Lce +β×L dice

[0059] Where Loss represents the cross-entropy Dessian weighted loss function, L ce L represents the cross-entropy loss function. dice Let represent the Descein loss function, α represent the cross-entropy loss weights, and β represent the Descein loss weights;

[0060] The cross-entropy loss function is defined as follows:

[0061]

[0062] Where NUM represents the number of preprocessed images, M represents the number of rows in the i-th preprocessed image group, N represents the number of columns in the i-th preprocessed image group, and g i,(x,y) This represents the true classification label of the head and neck tumor lesion in the x-th row and y-th column of the i-th preprocessed image. If g i,(x,y) =0 indicates a normal pixel area; if g i,(x,y) =1 indicates the number of pixels in the lesion area, p i,(x,y) ∈[0,1],p i,(x,y) This represents the predicted probability value of the pixel in the x-th row and y-th column of the head and neck tumor prediction probability map of the i-th preprocessed image calculated by the segmentation method, which belongs to the head and neck tumor lesion region classification.

[0063] The Dess loss function is defined as follows:

[0064]

[0065] Where δ∈[0,1], δ represents the adjustable coefficient of gradient propagation;

[0066] Preferably, step 3 involves combining probability thresholds to determine the pixel range of the head and neck tumor lesion region in the real-time head and neck tumor PET-CT image, as detailed below:

[0067] Pixels whose predicted probability value for belonging to the head and neck tumor lesion region in the head and neck tumor prediction probability map of the real-time head and neck tumor PET-CT image is greater than the probability threshold are selected as lesion region pixels, and the pixel range of the head and neck tumor lesion region in the real-time head and neck tumor PET-CT image is further obtained.

[0068] The present invention also provides a computer-readable medium storing a computer program executed by an electronic device, wherein when the computer program is run on the electronic device, it performs the steps of the head and neck tumor lesion region image segmentation method.

[0069] This invention provides a method to improve the accuracy of pixel region segmentation prediction for head and neck tumor lesions. By jointly achieving feature fusion in the encoding and decoding stages, it achieves good segmentation results for head and neck tumor lesion regions of different sizes, and when the difference between lesion regions and normal regions is not significant, which can help doctors more accurately determine the lesion region. Attached Figure Description

[0070] Figure 1 : Flowchart of the method according to an embodiment of the present invention.

[0071] Figure 2 : Neural network structure diagram of an embodiment of the present invention. Detailed Implementation

[0072] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0073] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0074] The following is combined Figure 1-2 The technical solution of this invention is a method for image segmentation of head and neck tumor lesion regions, as detailed below:

[0075] like Figure 1 The diagram shown is a flowchart of the method of the present invention.

[0076] Step 1: Acquire multiple sets of original head and neck tumor PET-CT images, and perform cropping, normalization, and data enhancement processing on each set of original head and neck tumor PET-CT images in sequence to obtain preprocessed images for each set, and label the true classification label of the head and neck tumor lesion for each pixel of each preprocessed image.

[0077] Step 2: Construct a lesion image segmentation network. Input each group of preprocessed images into the lesion image segmentation network to perform lesion segmentation prediction and obtain the head and neck tumor prediction probability map of each group of preprocessed images. Combine the true classification label of each pixel of the head and neck tumor lesion in each group of preprocessed images to construct the cross-entropy Dessell weighted loss function. Optimize the training through the stochastic gradient descent algorithm to obtain the trained lesion image segmentation network.

[0078] like Figure 2 The diagram shown is a neural network structure diagram of the present invention.

[0079] The lesion image segmentation network in step 2 includes:

[0080] The first encoding module, the second encoding module, ..., the Kth encoding module, the bottleneck module;

[0081] The first encoding module, the second encoding module, ..., the Kth encoding module, and the bottleneck module are sequentially cascaded together.

[0082] The kth encoding module is composed of multiple convolutional modules and downsampling layers cascaded together, k∈[1,K];

[0083] The bottleneck module is composed of multiple convolutional modules;

[0084] The fusion coding network includes:

[0085] The first fusion coding module, the second fusion coding module, ..., the Kth fusion coding module, and the bottleneck module;

[0086] The first fusion coding module, the second fusion coding module, ..., the Kth fusion coding module, and the bottleneck module are sequentially cascaded together.

[0087] The kth fusion coding module is composed of a multi-layer convolutional module, an attention module, and a downsampling layer cascaded in sequence, where k∈[1,K].

[0088] The bottleneck module is composed of multiple convolutional modules;

[0089] The single-mode decoding network includes:

[0090] The first decoding module, the second decoding module, ..., the Kth decoding module;

[0091] The first decoding module, the second decoding module, ..., the Kth decoding module are cascaded sequentially.

[0092] The kth decoding module is composed of an upsampling layer, a splicing layer, and a multi-layer convolutional module cascaded in sequence, where k∈[1,K].

[0093] The fusion decoding network includes:

[0094] The first fusion decoding module, the second fusion decoding module, ..., the Kth fusion decoding module;

[0095] The first fusion decoding module, the second fusion decoding module, ..., the Kth fusion decoding module are cascaded sequentially.

[0096] The kth fusion decoding module is composed of an upsampling layer, a splicing layer, and an inverted bottleneck convolution module cascaded in sequence, where k∈[1,K].

[0097] The first encoding module passes each modality of each preprocessed image through a multi-layer convolution module to obtain the first-stage features of each modality of each preprocessed image, and outputs the first-stage features of each modality of each preprocessed image to the first downsampling layer, the first fusion encoding module, and the first decoding module.

[0098] After outputting the first-stage features of each modality of each preprocessed image to the first downsampling layer, the downsampling features of each modality of each preprocessed image in the first stage are obtained. The downsampling features of each modality of each preprocessed image in the first stage are then output to the second encoding module.

[0099] If k∈[2,K-1], the k-th encoding module extracts the downsampled features of each modality at stage k-1 of each preprocessed image through a multi-layer convolution module to obtain the k-th stage features of each modality of each preprocessed image, and outputs the k-th stage features of each modality of each preprocessed image to the k-th downsampling layer, the k-th fusion encoding module, and the k-th decoding module respectively.

[0100] After outputting the k-th stage features of each modality of each preprocessed image to the k-th downsampling layer, the downsampling features of each modality of each preprocessed image at the k-th stage are obtained. The downsampling features of each modality of each preprocessed image at the k-th stage are then output to the (k+1)-th encoding module.

[0101] The Kth encoding module extracts the downsampled features of each modality in the (K-1)th stage of each preprocessed image through a multi-layer convolution module to obtain the Kth stage features of each modality in each preprocessed image. The Kth stage features of each modality in each preprocessed image are then output to the Kth downsampling layer, the Kth fusion encoding module, and the Kth decoding module, respectively.

[0102] After outputting the features of each modality at stage K of each preprocessed image to the Kth downsampling layer, the downsampling features of each modality at stage K of each preprocessed image are obtained, and the downsampling features of each modality at stage K of each preprocessed image are output to the bottleneck module.

[0103] The bottleneck module extracts the downsampled features of each modality at stage K of each preprocessed image through a multi-layer convolution module to obtain the bottleneck layer features of each modality of each preprocessed image, and outputs the bottleneck layer features of each modality of each preprocessed image to the Kth decoding module.

[0104] The first fusion coding module concatenates the modalities of each preprocessed image in the channel direction and passes them through a multi-layer convolution module to obtain the first-stage features of each preprocessed image. The first-stage features of each preprocessed image and the first-stage features of each modality of each preprocessed image are input into the first attention module to obtain the first-stage fusion features of each preprocessed image. The first-stage fusion features of each preprocessed image are output to the first downsampling layer to obtain the first-stage downsampled fusion features of each preprocessed image. The first-stage downsampled fusion features of each preprocessed image are output to the second fusion coding network.

[0105] If k∈[2,K-1], the k-th fusion coding module takes the (k-1)-th stage fusion features of each group of preprocessed images through a multi-layer convolution module to obtain the k-th stage features of each group of preprocessed images. The k-th stage features of each group of preprocessed images and the k-th stage features of each modality of each group of preprocessed images are input to the k-th attention module to obtain the k-th stage fusion features of each group of preprocessed images. The k-th stage fusion features of each group of preprocessed images are output to the k-th downsampling layer to obtain the k-th stage downsampled fusion features of each group of preprocessed images. The k-th stage downsampled fusion features of each group of preprocessed images are output to the (k+1)-th fusion coding network.

[0106] The Kth fusion coding module takes the K-1th stage fusion features of each group of preprocessed images and passes them through a multi-layer convolution module to obtain the Kth stage features of each group of preprocessed images. The Kth stage features of each group of preprocessed images and the Kth stage features of each modality of each group of preprocessed images are input to the Kth attention module to obtain the Kth stage fusion features of each group of preprocessed images. The Kth stage fusion features of each group of preprocessed images are output to the Kth downsampling layer to obtain the Kth stage downsampled fusion features of each group of preprocessed images. The Kth stage downsampled fusion features of each group of preprocessed images are output to the bottleneck module.

[0107] The bottleneck module extracts the downsampled fusion features of the Kth stage of each preprocessed image through a multi-layer convolution module to obtain the bottleneck layer fusion features of each preprocessed image, and outputs the bottleneck layer fusion features of each preprocessed image to the Kth fusion decoding module.

[0108] The Kth decoding module upsamples the bottleneck layer features of each modality of each preprocessed image to obtain the upsampled features of the Kth stage of each modality of each preprocessed image. The upsampled features of the Kth stage of each modality of each preprocessed image are concatenated with the features of the Kth stage of each modality of each preprocessed image by channel dimension. Then, the upsampled concatenated features of the Kth stage of each modality of each preprocessed image are obtained by passing through a multi-layer convolution module. The upsampled concatenated features of the Kth stage of each modality of each preprocessed image are output to the (K-1)th decoding module and the (K-1)th fusion decoding module, respectively.

[0109] If k∈[2,K-1], the k-th decoding module upsamples the upsampled stitching features of each modality at stage k+1 of each preprocessed image to obtain the upsampled features of each modality at stage k of each preprocessed image. The upsampled features of each modality at stage k of each preprocessed image are then stitched together with the features of each modality at stage k of each preprocessed image along the channel dimension. The upsampled stitching features of each modality at stage k of each preprocessed image are then obtained through a multi-layer convolution module. The upsampled stitching features of each modality at stage k of each preprocessed image are then output to the (k-1)-th decoding module and the (k-1)-th fusion decoding module, respectively.

[0110] The first decoding module upsamples the upsampled stitching features of each modality in the second stage of each preprocessed image to obtain the upsampled features of each modality in the first stage of each preprocessed image. It then stitches the upsampled features of each modality in the first stage of each preprocessed image with the features of each modality in the first stage of each preprocessed image along the channel dimension. Finally, it obtains the upsampled stitching features of each modality in the first stage of each preprocessed image through a multi-layer convolution module and outputs the upsampled stitching features of each modality in the first stage of each preprocessed image to the first fusion decoding module.

[0111] The Kth fusion decoding module upsamples the bottleneck layer fusion features of each group of preprocessed images to obtain the Kth stage upsampled fusion features of each group of preprocessed images. It then concatenates the Kth stage upsampled fusion features of each group of preprocessed images with the Kth stage upsampled stitching features of each modality of each group of preprocessed images along the channel dimension. Finally, it passes through the Kth inverted bottleneck convolution module to obtain the Kth stage upsampled stitching fusion features of each group of preprocessed images. Finally, it outputs the Kth stage upsampled stitching fusion features of each group of preprocessed images to the (K-1)th fusion decoding module.

[0112] If k∈[2,K-1], the k-th fusion decoding module upsamples the upsampled splicing fusion features of the (k+1)-th stage of each group of preprocessed images to obtain the upsampled fusion features of each modality at the k-th stage of each group of preprocessed images. The upsampled fusion features of the k-th stage of each group of preprocessed images are spliced ​​together with the upsampled splicing features of each modality at the k-th stage of each group of preprocessed images along the channel dimension. Then, the upsampled splicing fusion features of the k-th stage of each group of preprocessed images are obtained through the k-th inverted bottleneck convolution module. Finally, the upsampled splicing fusion features of the k-th stage of each group of preprocessed images are output to the (k-1)-th fusion decoding module.

[0113] The first fusion decoding module upsamples the second-stage upsampled splicing fusion features of each group of preprocessed images to obtain the first-stage upsampled fusion features of each modality of each group of preprocessed images. The first-stage upsampled fusion features of each group of preprocessed images are spliced ​​with the first-stage upsampled splicing features of each modality of each group of preprocessed images along the channel dimension. Then, the first inverted bottleneck convolution module obtains the first-stage upsampled splicing fusion features of each group of preprocessed images. The first-stage upsampled splicing fusion features of each group of preprocessed images are convolved to obtain the head and neck tumor prediction probability map of each group of preprocessed images.

[0114] The cross-entropy Dessell weighted loss function described in step 2 is defined as follows:

[0115] Loss=α×L ce +β×L dice

[0116] Where Loss represents the cross-entropy Dessian weighted loss function, L ce L represents the cross-entropy loss function. dice Let represent the Descein loss function, α = 0.5, β = 0.5;

[0117] The cross-entropy loss function is defined as follows:

[0118]

[0119] Where NUM = 450, M = 144, N = 144, g i,(x,y) This represents the true classification label of the head and neck tumor lesion in the x-th row and y-th column of the i-th preprocessed image. If g i,(x,y) =0 indicates a normal pixel area; if g i,(x,y) =1 indicates the number of pixels in the lesion area, p i,(x,y) ∈[0,1],p i,(x,y) This represents the predicted probability value of the pixel in the x-th row and y-th column of the head and neck tumor prediction probability map of the i-th preprocessed image calculated by the segmentation method, which belongs to the head and neck tumor lesion region classification.

[0120] The Dess loss function is defined as follows:

[0121]

[0122] Where δ∈[0,1],δ=0.00005;

[0123] Step 3: The real-time acquired head and neck tumor PET-CT images are predicted and segmented by the trained lesion image segmentation network to obtain the head and neck tumor prediction probability map of the real-time head and neck tumor PET-CT images. The pixel range of the head and neck tumor lesion area in the real-time head and neck tumor PET-CT images is determined by combining the probability threshold.

[0124] Step 3 involves combining probability thresholds to determine the pixel range of the head and neck tumor lesion region in the real-time head and neck tumor PET-CT image, as detailed below:

[0125] Pixels whose predicted probability value for belonging to the head and neck tumor lesion region category in the real-time head and neck tumor PET-CT image is greater than the probability threshold are selected as lesion region pixels, and the pixel range of the head and neck tumor lesion region in the real-time head and neck tumor PET-CT image is further obtained.

[0126] The segmentation results obtained by each method using the Descein coefficient as the metric are as follows:

[0127] Ours UNet Att-UNet UNETR SwinUNet Main tumor region Dice 0.7931 0.7868 0.7639 0.7967 0.7847 Lymph node area Dice 0.7384 0.7072 0.6910 0.6932 0.6891 Average Dice 0.7657 0.7474 0.7275 0.7450 0.7369

[0128] The Dess coefficient is defined as follows:

[0129]

[0130] The experimental results above show that the average Dess coefficient of this method is significantly higher than other methods when predicting the lesion region of head and neck tumors in PET-CT images. Specifically, the Dess coefficient of the main tumor region is higher than UNet, Att-UNet, and SwinUNet, but slightly lower than UNETR (0.0036); the Dess coefficient of the lymph node region is significantly higher than UNet (0.0312), Att-UNet (0.0474), UNETR (0.0452), and SwinUNet (0.0493). Because this method performs feature fusion in both the encoding and decoding stages, it effectively improves the network's ability to identify lymph node regions in head and neck tumor PET-CT images, thereby improving the segmentation effect of the entire head and neck tumor lesion region.

[0131] A specific embodiment of the present invention also provides a computer-readable medium.

[0132] The computer-readable medium is a server workstation;

[0133] The server workstation stores a computer program executed by the electronic device. When the computer program runs on the electronic device, it causes the electronic device to execute the steps of the stereo SAR data optimization method according to the embodiments of the present invention.

[0134] It should be understood that any parts not described in detail in this specification belong to the prior art.

[0135] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A method for image segmentation of head and neck tumor lesion regions, characterized in that: Multiple sets of raw head and neck tumor PET-CT images were acquired. Each set of raw head and neck tumor PET-CT images was preprocessed sequentially to obtain a preprocessed image. The true classification label of the head and neck tumor lesion for each pixel of each preprocessed image was marked. A lesion image segmentation network is constructed. Each group of preprocessed images is input into the lesion image segmentation network for lesion segmentation prediction, and a head and neck tumor prediction probability map is obtained for each group of preprocessed images. Combined with the true classification label of each pixel of head and neck tumor lesion in each group of preprocessed images, a cross-entropy Dessell weighted loss function is constructed. The trained lesion image segmentation network is obtained by optimizing the training through the stochastic gradient descent algorithm. The real-time acquired head and neck tumor PET-CT images are predicted and segmented by a trained lesion image segmentation network to obtain a head and neck tumor prediction probability map of the real-time head and neck tumor PET-CT images. The pixel range of the head and neck tumor lesion area in the real-time head and neck tumor PET-CT images is determined by combining the probability threshold. The lesion image segmentation network includes: Single-modal coding network, fusion coding network, single-modal decoding network, fusion decoding network; The single-modal coding network performs feature extraction processing on each group of preprocessed images to obtain intermediate feature representations and final feature representations of each group of preprocessed images. The intermediate feature representations of each group of preprocessed images are output to the fusion coding network, and the final feature representations of each group of preprocessed images are output to the single-modal decoding network. The fusion coding network performs feature fusion processing on the intermediate feature representations of each group of preprocessed images to obtain the fusion coding features of each group of preprocessed images, and outputs the fusion coding features of each group of preprocessed images to the fusion decoding network. The single-mode decoding network decodes the final feature representation of each preprocessed image to obtain the decoded feature representation of each preprocessed image, and outputs the decoded feature representation of each preprocessed image to the fusion decoding network. The fusion decoding network performs fusion decoding on the decoding feature representations of each group of preprocessed images to finally obtain the predicted lesion region segmentation image of each preprocessed image.

2. The image segmentation method for head and neck tumor lesion regions according to claim 1, characterized in that, Multiple sets of raw head and neck tumor PET-CT images were acquired. Each set of raw head and neck tumor PET-CT images was preprocessed sequentially to obtain preprocessed images. The true classification label of the head and neck tumor lesion for each pixel in each preprocessed image was labeled, including: Multiple sets of raw head and neck tumor PET-CT images were acquired. Each set of raw head and neck tumor PET-CT images was then cropped, normalized, and augmented sequentially to obtain preprocessed images. The true classification label of the head and neck tumor lesion for each pixel in each preprocessed image was then labeled.

3. The image segmentation method for head and neck tumor lesion regions according to claim 1, characterized in that: The single-modal coding network includes: The first encoding module, the second encoding module, ..., the Kth encoding module, the bottleneck module; The first encoding module, the second encoding module, ..., the Kth encoding module, and the bottleneck module are sequentially cascaded together. The kth encoding module is composed of multiple convolutional modules and downsampling layers cascaded in sequence, k∈[1,K]; The bottleneck module is composed of multiple convolutional modules; The fusion coding network includes: The first fusion coding module, the second fusion coding module, ..., the Kth fusion coding module, and the bottleneck module; The first fusion coding module, the second fusion coding module, ..., the Kth fusion coding module, and the bottleneck module are sequentially cascaded together. The kth fusion coding module is composed of a multi-layer convolutional module, an attention module, and a downsampling layer cascaded in sequence, where k∈[1,K]. The bottleneck module is composed of multiple convolutional modules; The single-mode decoding network includes: The first decoding module, the second decoding module, ..., the Kth decoding module; The first decoding module, the second decoding module, ..., the Kth decoding module are cascaded sequentially. The kth decoding module is composed of an upsampling layer, a splicing layer, and a multi-layer convolutional module cascaded in sequence, where k∈[1,K]. The fusion decoding network includes: The first fusion decoding module, the second fusion decoding module, ..., the Kth fusion decoding module; The first fusion decoding module, the second fusion decoding module, ..., the Kth fusion decoding module are cascaded sequentially. The k-th fusion decoding module is composed of an upsampling layer, a splicing layer, and an inverted bottleneck convolution module cascaded in sequence, k∈[1,K].

4. The image segmentation method for head and neck tumor lesion areas according to claim 3, characterized in that: The first encoding module passes each modality of each preprocessed image through a multi-layer convolution module to obtain the first-stage features of each modality of each preprocessed image, and outputs the first-stage features of each modality of each preprocessed image to the first downsampling layer, the first fusion encoding module, and the first decoding module. After outputting the first-stage features of each modality of each preprocessed image to the first downsampling layer, the downsampling features of each modality of each preprocessed image in the first stage are obtained. The downsampling features of each modality of each preprocessed image in the first stage are then output to the second encoding module. If k∈[2,K-1], the k-th encoding module extracts the downsampled features of each modality at stage k-1 of each preprocessed image through a multi-layer convolution module to obtain the k-th stage features of each modality of each preprocessed image, and outputs the k-th stage features of each modality of each preprocessed image to the k-th downsampling layer, the k-th fusion encoding module, and the k-th decoding module respectively. After outputting the k-th stage features of each modality of each preprocessed image to the k-th downsampling layer, the downsampling features of each modality of each preprocessed image at the k-th stage are obtained. The downsampling features of each modality of each preprocessed image at the k-th stage are then output to the (k+1)-th encoding module. The Kth encoding module extracts the downsampled features of each modality in the (K-1)th stage of each preprocessed image through a multi-layer convolution module to obtain the Kth stage features of each modality in each preprocessed image. The Kth stage features of each modality in each preprocessed image are then output to the Kth downsampling layer, the Kth fusion encoding module, and the Kth decoding module, respectively. After outputting the features of each modality at stage K of each preprocessed image to the Kth downsampling layer, the downsampling features of each modality at stage K of each preprocessed image are obtained, and the downsampling features of each modality at stage K of each preprocessed image are output to the bottleneck module. The bottleneck module extracts the downsampled features of each modality at stage K of each preprocessed image through a multi-layer convolution module to obtain the bottleneck layer features of each modality of each preprocessed image, and outputs the bottleneck layer features of each modality of each preprocessed image to the Kth decoding module. The first fusion coding module concatenates the modalities of each preprocessed image in the channel direction and passes them through a multi-layer convolution module to obtain the first-stage features of each preprocessed image. The first-stage features of each preprocessed image and the first-stage features of each modality of each preprocessed image are input into the first attention module to obtain the first-stage fusion features of each preprocessed image. The first-stage fusion features of each preprocessed image are output to the first downsampling layer to obtain the first-stage downsampled fusion features of each preprocessed image. The first-stage downsampled fusion features of each preprocessed image are output to the second fusion coding network. If k∈[2,K-1], the k-th fusion coding module takes the (k-1)-th stage fusion features of each group of preprocessed images through a multi-layer convolution module to obtain the k-th stage features of each group of preprocessed images. The k-th stage features of each group of preprocessed images and the k-th stage features of each modality of each group of preprocessed images are input to the k-th attention module to obtain the k-th stage fusion features of each group of preprocessed images. The k-th stage fusion features of each group of preprocessed images are output to the k-th downsampling layer to obtain the k-th stage downsampled fusion features of each group of preprocessed images. The k-th stage downsampled fusion features of each group of preprocessed images are output to the (k+1)-th fusion coding network. The Kth fusion coding module takes the K-1th stage fusion features of each group of preprocessed images and passes them through a multi-layer convolution module to obtain the Kth stage features of each group of preprocessed images. The Kth stage features of each group of preprocessed images and the Kth stage features of each modality of each group of preprocessed images are input to the Kth attention module to obtain the Kth stage fusion features of each group of preprocessed images. The Kth stage fusion features of each group of preprocessed images are output to the Kth downsampling layer to obtain the Kth stage downsampled fusion features of each group of preprocessed images. The Kth stage downsampled fusion features of each group of preprocessed images are output to the bottleneck module. The bottleneck module extracts the downsampled fusion features of the Kth stage of each preprocessed image through a multi-layer convolution module to obtain the bottleneck layer fusion features of each preprocessed image, and outputs the bottleneck layer fusion features of each preprocessed image to the Kth fusion decoding module. The Kth decoding module upsamples the bottleneck layer features of each modality of each preprocessed image to obtain the upsampled features of the Kth stage of each modality of each preprocessed image. The upsampled features of the Kth stage of each modality of each preprocessed image are concatenated with the features of the Kth stage of each modality of each preprocessed image by channel dimension. Then, the upsampled concatenated features of the Kth stage of each modality of each preprocessed image are obtained by passing through a multi-layer convolution module. The upsampled concatenated features of the Kth stage of each modality of each preprocessed image are output to the (K-1)th decoding module and the (K-1)th fusion decoding module, respectively. If k∈[2,K-1], the k-th decoding module upsamples the upsampled stitching features of each modality at stage k+1 of each preprocessed image to obtain the upsampled features of each modality at stage k of each preprocessed image. The upsampled features of each modality at stage k of each preprocessed image are then stitched together with the features of each modality at stage k of each preprocessed image along the channel dimension. The upsampled stitching features of each modality at stage k of each preprocessed image are then obtained through a multi-layer convolution module. The upsampled stitching features of each modality at stage k of each preprocessed image are then output to the (k-1)-th decoding module and the (k-1)-th fusion decoding module, respectively. The first decoding module upsamples the upsampled stitching features of each modality in the second stage of each preprocessed image to obtain the upsampled features of each modality in the first stage of each preprocessed image. It then stitches the upsampled features of each modality in the first stage of each preprocessed image with the features of each modality in the first stage of each preprocessed image along the channel dimension. Finally, it obtains the upsampled stitching features of each modality in the first stage of each preprocessed image through a multi-layer convolution module and outputs the upsampled stitching features of each modality in the first stage of each preprocessed image to the first fusion decoding module. The Kth fusion decoding module upsamples the bottleneck layer fusion features of each group of preprocessed images to obtain the Kth stage upsampled fusion features of each group of preprocessed images. It then concatenates the Kth stage upsampled fusion features of each group of preprocessed images with the Kth stage upsampled stitching features of each modality of each group of preprocessed images along the channel dimension. Finally, it passes through the Kth inverted bottleneck convolution module to obtain the Kth stage upsampled stitching fusion features of each group of preprocessed images. Finally, it outputs the Kth stage upsampled stitching fusion features of each group of preprocessed images to the (K-1)th fusion decoding module. If k∈[2,K-1], the k-th fusion decoding module upsamples the upsampled splicing fusion features of the (k+1)-th stage of each group of preprocessed images to obtain the upsampled fusion features of each modality at the k-th stage of each group of preprocessed images. The upsampled fusion features of the k-th stage of each group of preprocessed images are spliced ​​together with the upsampled splicing features of each modality at the k-th stage of each group of preprocessed images along the channel dimension. Then, the upsampled splicing fusion features of the k-th stage of each group of preprocessed images are obtained through the k-th inverted bottleneck convolution module. Finally, the upsampled splicing fusion features of the k-th stage of each group of preprocessed images are output to the (k-1)-th fusion decoding module. The first fusion decoding module upsamples the second-stage upsampled splicing fusion features of each preprocessed image group to obtain the first-stage upsampled fusion features of each modality of each preprocessed image group. The first-stage upsampled fusion features of each preprocessed image group are then spliced ​​with the first-stage upsampled splicing features of each modality of each preprocessed image group along the channel dimension. The first inverted bottleneck convolution module then obtains the first-stage upsampled splicing fusion features of each preprocessed image group. Finally, the first-stage upsampled splicing fusion features of each preprocessed image group are convolved to obtain the head and neck tumor prediction probability map of each preprocessed image group.

5. The image segmentation method for head and neck tumor lesion areas according to claim 4, characterized in that: The cross-entropy Dessell weighted loss function described in step 2 is defined as follows: in, This represents the cross-entropy Dessell weighted loss function. Represents the cross-entropy loss function. Denotes the Desce loss function. The weights represent the cross-entropy loss. This indicates the weight of the Des loss; The cross-entropy loss function is defined as follows: in, Indicates the number of preprocessed images. This indicates the row number of the i-th preprocessed image group. This indicates the column number of the i-th preprocessed image group. This represents the true classification label of the head and neck tumor lesion in the x-th row and y-th column of the i-th preprocessed image. These are normal region pixels. These are the pixels representing the lesion area. ∈[0,1], This represents the predicted probability value of the pixel in the x-th row and y-th column of the head and neck tumor prediction probability map of the i-th preprocessed image calculated by the segmentation method, which belongs to the head and neck tumor lesion region classification. The Dess loss function is defined as follows: in, ∈[0, 1], This represents the adjustable coefficient of gradient propagation.

6. The image segmentation method for head and neck tumor lesion regions according to claim 5, characterized in that: Step 3 involves combining probability thresholds to determine the pixel range of the head and neck tumor lesion region in the real-time head and neck tumor PET-CT image, as detailed below: Pixels whose predicted probability value for belonging to the head and neck tumor lesion region in the head and neck tumor prediction probability map of the real-time head and neck tumor PET-CT image is greater than the probability threshold are selected as lesion region pixels, and the pixel range of the head and neck tumor lesion region in the real-time head and neck tumor PET-CT image is further obtained.

7. The image segmentation method for head and neck tumor lesion regions according to claim 6, characterized in that... ： A computer-readable medium, characterized in that, It stores a computer program executed by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of the method as described in any one of claims 1-6.

Images