PET-CT medical image automatic segmentation method based on weak supervised learning

Abstract

The invention discloses a PET-CT medical image automatic segmentation method based on weak supervised learning, and the method comprises the steps: firstly obtaining original images of PET and CT modes, and carrying out the registration and resampling; secondly, establishing a PET-CT medical image segmentation network model based on weak supervised learning; then, establishing a point annotation data set based on the diffuse large B-cell lymphoma, and performing pseudo-label generation on the data set by using a multi-modal fusion pseudo-label generation algorithm; then, the PET image, the CT image and the pseudo tag are sent to the segmentation network model for training, and optimal parameters are stored; and finally, performing prediction segmentation on the target focus by using the optimal model. According to the method, the segmentation network model is trained by using the labels in a weak form, so that the dependence of an existing medical image segmentation algorithm on accurate annotation data is reduced.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of medical image processing, and particularly relates to a PET-CT medical image automatic segmentation method based on weakly supervised learning. BACKGROUND

[0002] Diffuse large B-cell lymphoma (DLBCL) is the most common aggressive non-Hodgkin lymphoma, characterized by rapid proliferation of tumor cells, forming large heterogeneous B lymphocytes. These cells infiltrate lymph nodes, spleen, bone marrow and other lymphoid tissues extensively. Accurate delineation of DLBCL in PET-CT (Positron Emission Computed Tomography-Computed Tomography) images is crucial for the prognosis evaluation of patients.

[0003] However, due to the systemic spread of DLBCL and the high heterogeneity of lesions, accurate demarcation of lesion regions is a very challenging task. Moreover, in the segmentation of DLBCL, due to factors such as different sizes of lesions, low resolution of PET (Positron Emission Computed Tomography), low contrast between normal tissue and tumor in CT (Computed Tomography) images, and blurred tumor boundaries, the annotation process requires professional medical knowledge and is time-consuming. Therefore, developing computer-aided DLBCL automatic segmentation technology will effectively alleviate the difficulties in the annotation work.

[0004] In recent years, deep learning technology has provided a possibility to solve this problem. Although deep learning technology has greatly improved the accuracy of lymphoma segmentation compared to traditional methods and has shown significant potential for clinical application, its training process usually requires a large amount of accurate pixel-level annotation data. Based on the limitations of sample quantity and annotation cost, how to utilize rough medical image labels to achieve diffuse large B-cell lymphoma segmentation has become an important direction of current research. SUMMARY

[0005] The present application aims to at least partially solve one of the technical problems existing in the related art.

[0006] One object of the present application is to provide a PET-CT medical image automatic segmentation method based on weakly supervised learning, which utilizes weaker labels to train the segmentation network model, thereby reducing the dependence of existing medical image segmentation algorithms on accurate annotation data.

[0007] Another object of the present application is to provide a PET-CT medical image automatic segmentation method based on weakly supervised learning, comprising the following steps:

[0008] S1, acquire PET images and CT images, and pre-process them;

[0009] S2, add weak annotation information containing only lesion point annotations to the PET images and CT images, and then generate pseudo-labels based on the weak annotation information using a multi-modal fusion pseudo-label generation algorithm; construct a weakly supervised data set using the acquired PET images, CT images, and corresponding pseudo-labels;

[0010] S3, construct an image segmentation network with an encoder-decoder architecture, wherein the encoder includes multiple convolutional layers and down-sampling layers, the decoder includes multiple convolutional layers and up-sampling layers, a neck Transformer module based on a learnable sparse mechanism is designed at the deepest layer of the encoder, multi-scale skip connections are used between the encoder and the decoder, and each convolutional layer of the image segmentation network uses an adaptive multi-kernel convolution module; the PET images and CT images are processed by the image segmentation network to output a segmentation result of a target region;

[0011] S4, train the image segmentation network using the constructed weakly supervised data set to obtain a trained PET-CT medical image automatic segmentation model;

[0012] S5, input PET images and CT images to be segmented into the PET-CT medical image automatic segmentation model to predict a segmentation image of a target lesion.

[0013] The further preferred technical solution of the present application is that in step S1, the acquired PET images and CT images are pre-processed, specifically:

[0014] Align the original images of the two modalities to the spatial coordinate position, and resample them to the same resolution one by one corresponding to the lesion annotation files;

[0015] Perform windowing processing on the CT images and denoising processing on the PET images, and finally perform normalization.

[0016] As a preferred, in step S2, the weak annotation information containing only lesion point annotations is added to the PET images and CT images, and then pseudo-labels are generated based on the weak annotation information using a multi-modal fusion pseudo-label generation algorithm; specifically:

[0017] Based on the PET images and CT images of diffuse large B-cell lymphoma, one annotation point is provided on each of the six faces of the tumor, and one point annotation is provided inside the tumor as the weak annotation information containing lesion point annotations;

[0018] An image fusion strategy with adjustable weights is used to fuse the PET images and CT images;

[0019] The geodesic distance transformation method combined with the intensity information and spatial distance of the fused image is used to calculate the distance between voxels by balancing the weights of the two distances, which is expressed as:

[0020] ;

[0021] wherein, is the spatial distance, is the image gradient, is the geodesic distance between voxels, is the weight parameter;

[0022] The pseudo-label is generated according to the geodesic distance.

[0023] As preferred, the image segmentation network constructed in step S3 first extracts low-level detail features of the PET image and the CT image respectively, and splices the extracted features along the channel dimension through an image fusion module to obtain a multi-modal fusion feature map; the multi-modal fusion feature map gradually extracts high-level semantic features of the multi-modal fusion feature map through an encoder.

[0024] As preferred, each convolutional layer of the image segmentation network in step S3 adopts an adaptive multi-kernel convolution module, which combines an attention weighting module and a multi-branch convolution module.

[0025] wherein, the multi-branch convolution module has k three-dimensional convolution kernel branches with the same spatial dimension but different parameters, and the attention weighting module weights different three-dimensional convolution kernel branches; in each convolution processing stage, the convolution output of the th three-dimensional convolution kernel branch is , and the input is gradually updated through global pooling, convolution, normalization and activation operations to output an attention mask, which is expressed as:

[0026] ;

[0027] The attention mask corresponding to the th convolution kernel branch is expressed as

[0028] ;

[0029] wherein, is the output feature map of the th three-dimensional convolution kernel branch.

[0030] As preferred, the neck Transformer module of the deepest layer design of the encoder in step S3 is based on a learnable sparse mechanism, which first realizes patch segmentation and flattening through 3D convolution, and then adopts a hierarchical mixed attention strategy to selectively introduce a learnable sparse mechanism according to the position of the layer, and first generates three feature mapping matrices corresponding to the query, key and value through a full connection layer 、 and , denoted as:

[0031] ;

[0032] ;

[0033] ;

[0034] is the input feature, and secondly, the original full connection attention score is calculated as:

[0035] ;

[0036] wherein, is the batch index, is the th head in the multi-head attention, is the dimension of each head, is the sequence position of , and is the sequence position of ;

[0037] Then, the learnable sparsification is performed, and for each , only the most relevant keys are retained, and then:

[0038] ;

[0039] ;

[0040] ;

[0041] wherein, is a learnable parameter, and is constrained to 0-1 through , is a sparse proportion parameter, is the length of the input flattened sequence, represents the index set of the most relevant keys for the th query position; and the sparse attention score matrix is:

[0042] ;

[0043] After the softmax function activation is performed on the , the attention output is:

[0044] ;

[0045] wherein, represents the attention weight of the position to the position , represents the Value vector of the position , represents the final output feature vector of the position , and finally the output is converted into a format suitable for network processing through a dimension reshaping operation. As preferred, the image segmentation network is trained by using the constructed weakly supervised data set in step S4, specifically:

[0046] The weakly supervised data set constructed in step S2 is divided into a training set, a validation set and a test set according to a ratio of 6:2:2;

[0047] The divided training set is sent into the image segmentation network, and according to the image input in the training set, the corresponding target segmentation region output is calculated, compared with the pseudo label, the error is calculated by using the loss function and back propagation, and the weight of the network is updated;

[0048] The validation set is used for validation at the end of each epoch, the performance of the model on unseen data is evaluated, the hyperparameters are adjusted according to the loss and performance indicators of the validation set, and the best network parameters are saved according to the performance of the validation set.

[0049] The present application further provides a non-transitory computer readable storage medium having computer instructions stored thereon, the computer instructions causing a computer to execute the PET-CT medical image automatic segmentation method based on weakly supervised learning.

[0050] The present application further provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus, the processor calls the logical instructions in the memory to execute the PET-CT medical image automatic segmentation method based on weakly supervised learning.

[0051] The present application further provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus, the processor calls the logical instructions in the memory to execute the PET-CT medical image automatic segmentation method based on weakly supervised learning.

[0052] ​​In still another aspect, the present application provides a computer program product, which comprises a computer program stored on a non-transitory computer-readable storage medium, and the computer program is executed by a processor, and the computer executes the PET-CT medical image automatic segmentation method based on weakly supervised learning described above.

[0053] Beneficial effects: the PET-CT medical image automatic segmentation method based on weakly supervised learning of the present application combines point annotation strategy with multi-modal data fusion, fully utilizes the complementary advantages of PET and CT modalities, and generates high-quality pseudo labels through a multi-modal pseudo label generation algorithm. The fusion of PET and CT images provides more comprehensive tumor metabolism and anatomical structure information for the model, so that the performance of the segmentation task is significantly improved.

[0054] In order to further enhance the performance of the network, the proposed multi-modal fusion network introduces two innovative modules: an adaptive multi-kernel convolution module and a neck Transformer module based on a learnable sparse mechanism. The adaptive multi-kernel convolution module automatically learns different feature extraction modes in the training process through different branch convolution kernels, realizes content-adaptive feature extraction, and thus improves the segmentation accuracy of DLBCL lesions. The neck Transformer module based on the learnable sparse mechanism adaptively adjusts the sparse ratio through learnable parameters, reduces the computational overhead while maintaining key spatial dependencies. The module adopts a hierarchical sparse strategy: the first and last layers maintain full attention to ensure information integrity, and the middle layers apply sparse attention to improve efficiency. This module can selectively capture key long-range spatial dependencies, while maintaining computational efficiency, optimizing the correlation modeling of DLBCL multiple lesions and the lesion recognition accuracy in complex anatomical environments.

[0055] The network architecture of the present application relies on point annotation rather than precise pixel-level annotation under the weakly supervised learning framework, significantly reducing the demand for high-quality annotation data and reducing the cost and complexity of data annotation. This method has achieved satisfactory performance in medical image segmentation tasks and effectively reduces the strong dependence of traditional deep learning methods on accurate annotation data, providing an effective new approach to weakly supervised medical image segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 The flowchart of the PET-CT medical image automatic segmentation method based on weakly supervised learning;

[0057] Figure 2 The flowchart of the pseudo label generation algorithm for pseudo label generation;

[0058] Figure 3 The structure diagram of the image segmentation network;

[0059] Figure 4 Figure 1 is a structural schematic diagram of an adaptive multi-core convolution module;

[0060] Figure 5 Figure 2 is a structural schematic diagram of a neck Transformer module based on a learnable sparse mechanism;

[0061] Figure 6 Figure 3 is a comparison diagram of tumor images predicted by the method of the present application and other comparative tests. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all the embodiments, and they should not be understood as limiting the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application. In the description of the present application, it should be understood that the terms used are only for the purpose of description, and should not be understood as indicating or implying relative importance.

[0063] The following Figures 1-6 The present application provides a PET-CT medical image automatic segmentation method based on weakly supervised learning.

[0064] Embodiment 1: The present embodiment provides a PET-CT medical image automatic segmentation method based on weakly supervised learning.

[0065] Based on the problem of high demand for high-quality labeled data existing in traditional image segmentation technology, weakly supervised medical image segmentation provides a solution to this problem, which effectively alleviates the problem of scarcity of pixel-level annotation data by using a weaker form of label. The present application uses rough medical image labels to realize the segmentation of diffuse large B-cell lymphoma. The present application first generates pseudo labels based on the geodesic distance according to a multi-modal fusion pseudo label generation algorithm, and constructs a segmentation network model based on a convolutional neural network combined with an adaptive multi-core convolution module and a neck Transformer module based on a learnable sparse mechanism. The present application makes full use of PET-CT images, better captures the tumor metabolic characteristics and structural characteristics contained in different medical images, and effectively reduces the dependence on large-scale labeled data and reduces the labor cost.

[0066] The PET-CT medical image automatic segmentation method based on weakly supervised learning of the present application, as shown in Figure 1 The specific steps are as follows:

[0067] S1, acquire PET images and CT images, and pre-process them.

[0068] The original images of two modalities of positron emission computed tomography (PET) and computed tomography (CT) are acquired, the original images of the two modalities are registered to the spatial coordinate position for alignment, and are resampled to the same resolution one by one corresponding to the lesion drawing file; then the CT image is windowed, and the PET image is denoised, and finally normalized.

[0069] S2, generating pseudo-labels and constructing a data set.

[0070] As shown in Figure 2 , the pre-processed PET image and CT image are added with weak annotation information containing only lesion point annotations, that is, based on the PET image and CT image of diffuse large B-cell lymphoma, annotation points are provided on the six surfaces (upper, lower, front, back, left and right) of the tumor, and one point annotation is provided inside the tumor as weak annotation information containing lesion point annotations;

[0071] A weight-adjustable image fusion strategy is adopted to fuse the PET image and the CT image. The CT image can provide accurate anatomical structure information, and the PET image can reflect the functional metabolic information of the tissue.

[0072] A geodesic transform method combining image intensity information (functional information) and spatial distance (anatomical information) is used to calculate the distance between voxels by balancing the weights of the two distances, which is expressed as:

[0073] ;

[0074] wherein, is the spatial distance, is the image gradient, is the geodesic distance between voxels, is the weight parameter;

[0075] The pseudo-labels are generated according to the geodesic distance;

[0076] Then, the pseudo-labels are generated based on the weak annotation information using a multi-modal fusion pseudo-label generation algorithm; and a weakly supervised data set is constructed using the obtained PET image, CT image and corresponding pseudo-labels.

[0077] S3, constructing an image segmentation network.

[0078] As shown in Figure 3 , an image segmentation network with an encoder-decoder architecture is constructed, wherein the encoder includes 4 convolutional layers and a down-sampling layer, and the decoder includes 4 convolutional layers and an up-sampling layer.

[0079] First, low-level detail features of PET and CT images are extracted respectively, and after processing by a convolution block, the extracted features are spliced together along the channel dimension by an image fusion module to obtain a multi-modal fusion feature map, so as to fully utilize the complementary information of PET and CT images.

[0080] Each convolutional layer of the image segmentation network adopts an adaptive multi-kernel convolution module. As shown in Figure 4 , the adaptive multi-kernel convolution module combines an attention weighting module and a multi-branch convolution module.

[0081] Among them, the multi-branch convolution module has k three-dimensional convolution kernel branches with the same spatial dimension but different parameters, which learn to capture different feature patterns and spatial representations during the training process. The attention weighting module weights different three-dimensional convolution kernel branches; in each convolution processing stage, the convolution output of the th three-dimensional convolution kernel branch is , and the input is gradually updated by global pooling, convolution, normalization and activation operations to output an attention mask, denoted as:

[0082] ;

[0083] The attention mask corresponding to the th convolution kernel branch is denoted as , and the final output feature map is denoted as:

[0084] ;

[0085] Among them, denotes the output feature map of the th three-dimensional convolution kernel branch.

[0086] The deepest layer of the encoder is designed based on a neck Transformer module with a learnable sparse mechanism, which enables the network to focus on long-distance dependencies in the input feature map while reducing computational overhead, thereby improving the model's ability to capture complex patterns and subtle differences. As shown in Figure 5 , the neck Transformer module with a learnable sparse mechanism first performs patch segmentation and flattening through 3D convolution, and then adopts a hierarchical hybrid attention strategy to selectively introduce a learnable sparse mechanism according to the position of the layer. First, three feature mapping matrices , and corresponding to queries, keys and values are generated respectively through a fully connected layer, denoted as:

[0087] ;

[0088] ;

[0089] ;

[0090] is the input feature, secondly, the original full connection attention score is calculated as:

[0091] ;

[0092] where, is the batch index, is the th head in multi-head attention, is the dimension of each head, is the sequence position of , is the sequence position of ;

[0093] Then, the learnable sparsification is performed, for each , only the keys most relevant to it are kept, then:

[0094] ;

[0095] ;

[0096] ;

[0097] where, is a learnable parameter, and is constrained to 0-1 by , is the sparsity ratio parameter, is the length of the flattened input sequence, represents the index set of the keys most relevant to the th query position; the sparse attention score matrix is:

[0098] ;

[0099] After the softmax function activation on , the is obtained, and the attention output is:

[0100] ;

[0101] where, represents the attention weight of position to position , represents the attention weight of position Value vector of the Value vector, indicates the position the final output feature vector, and finally the output is converted into a format suitable for network processing through the operation of dimension reshaping.

[0102] In the decoding stage, a multi-scale skip connection is adopted between the encoder and the decoder, and the low-level detail features and high-level semantic features of the encoder are fused, so that the spatial details are better restored in the upsampling process, and an accurate target segmentation result is obtained.

[0103] S4, model training.

[0104] The weakly supervised data set constructed in step S2 is divided into a training set, a validation set and a test set in a ratio of 6:2:2;

[0105] The divided training set is input into the image segmentation network, and the corresponding target segmentation region output is calculated according to the image input in the training set, compared with the pseudo label, the error is calculated by using the loss function and back propagation, and the weight of the network is updated;

[0106] The validation set is used for validation at the end of each epoch, the performance of the model on unseen data is evaluated, the hyperparameters are adjusted according to the loss and performance indicators of the validation set, and the best network parameters are saved according to the performance of the validation set, and the trained PET-CT medical image automatic segmentation model is obtained;

[0107] S5, image segmentation.

[0108] The PET image and the CT image to be segmented are input into the PET-CT medical image automatic segmentation model, and the segmentation image of the target lesion is predicted.

[0109] The method according to the present application designs a simulation experiment and carries out five groups of comparison tests.

[0110] (1) Simulation conditions

[0111] The simulation experiment of the present application is carried out on an Ubuntu server, using CUDA 11.8 version. The server is equipped with a 24GB video memory NVIDIA RTX A5000 GPU. The software environment includes Python 3.8.10 and Pytorch 2.0.0.

[0112] (2) Simulation content

[0113] Comparison experiment one: based on the classical VNet segmentation network, the CT image is predicted and segmented;

[0114] Comparison experiment two: based on the classical VNet segmentation network, the PET image is predicted and segmented;

[0115] Comparative Experiment Three: Predictive segmentation of PET-CT images based on a classical VNet segmentation network;

[0116] Comparative Experiment Four: Predictive segmentation of PET-CT images using a segmentation network with an adaptive multi-kernel convolution module;

[0117] Comparative Experiment Five: Predictive segmentation of PET-CT images using a segmentation network with a neck Transformer module based on a learnable sparse mechanism;

[0118] The above five sets of comparative experiments are based on weakly supervised learning of medical image segmentation.

[0119] (3) Simulation results

[0120] The comparison chart of the tumor images predicted by the simulation segmentation is shown in Figure 6 The evaluation indexes Dice similarity coefficient and Jaccard similarity coefficient of the predictive segmentation performance of the above five experimental methods and the method proposed in the application on the same data set are given in the simulation experiment, which are used to quantitatively evaluate the performance of each method in the image segmentation task, and the evaluation results are shown in Table 1. The core significance of these two indexes is to measure the spatial overlap degree between the prediction result and the true label from the mathematical point of view, so as to objectively reflect the accuracy and robustness of the segmentation algorithm. The value range of these two indexes is [0, 1], and in these two indexes, the closer the result is to 1, the higher the similarity between the prediction result and the true label, and the better the segmentation performance.

[0121] Table 1 Segmentation performance of different experimental methods under the same PET-CT data set

[0122]

[0123] According to the comparison results of the evaluation indexes in Table 1, the PET-CT medical image automatic segmentation method based on weakly supervised learning proposed in the application has the best segmentation performance. Compared with other traditional methods, the proposed method not only can make full use of weakly supervised information to improve the segmentation accuracy, but also can significantly improve the recognition ability of the network to tumor or lesion area by combining PET and CT image data of two different modalities.

[0124] In summary, the PET-CT medical image automatic segmentation method based on weakly supervised learning provided by the present application adopts a weakly supervised learning framework, effectively makes up for the respective shortcomings of PET and CT two different modal image data by reasonably combining the two, and fully plays the advantages of the two, so that the model can more comprehensively understand the metabolic active area of the tumor and its morphological characteristics, thereby improving the accuracy of the segmentation result. The strategy of generating pseudo labels by point annotation reduces the dependence on a large amount of manually annotated data. The pseudo label generation method of the present application accurately calculates image gradient information and the like by using a geodesic transformation algorithm based on PET-CT image fusion, thereby providing a solid foundation for generating high-quality pseudo labels. In this way, the present application can effectively train the network with the generated pseudo labels under the condition of less data annotation, thereby avoiding the problem of insufficient supervision of the annotated data for further improving the performance of the model. The present application introduces an adaptive multi-kernel convolution module in the network architecture, which automatically learns different feature extraction modes in the training process through different branch convolution kernels, thereby realizing content-adaptive feature extraction. In addition, the present application also introduces a neck Transformer module based on a learnable sparse mechanism, which is designed to capture global context information in the image, especially in the deepest layer of the encoder, helping the model to understand the overall structure relationship when processing complex medical images, thereby avoiding the problem of insufficient local information and reducing the computational overhead while maintaining the key spatial dependency relationship. These innovative technologies enhance the model's perception of tumor characteristics, thereby improving the segmentation accuracy.

[0125] Embodiment 2: The present embodiment provides a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to execute a PET-CT medical image automatic segmentation method based on weakly supervised learning, the method comprising the following steps:

[0126] S1, obtaining PET images and CT images and pre-processing the same;

[0127] S2, adding weak annotation information containing only lesion point annotations to the PET images and CT images, and then generating pseudo labels based on the weak annotation information using a multi-modal fusion pseudo label generation algorithm; and constructing a weakly supervised data set using the obtained PET images, CT images and corresponding pseudo labels;

[0128] S3, construct an image segmentation network of an encoder-decoder architecture, wherein the encoder comprises a plurality of convolutional layers and down-sampling layers, the decoder comprises a plurality of convolutional layers and up-sampling layers, a neck Transformer module based on a learnable sparse mechanism is designed at the deepest layer of the encoder, multi-scale skip connections are adopted between the encoder and the decoder, and each convolutional layer of the image segmentation network adopts an adaptive multi-kernel convolution module; the PET image and the CT image are processed by the image segmentation network, and a segmentation result of a target region is output;

[0129] S4, train the image segmentation network by using the constructed weakly supervised dataset, and obtain a trained PET-CT medical image automatic segmentation model;

[0130] S5, input the PET image and the CT image to be segmented into the PET-CT medical image automatic segmentation model, and predict a segmentation image of a target lesion.

[0131] Embodiment 3: The electronic device can include a processor, a communications interface, a memory, and a communications bus, wherein the processor, the communications interface, and the memory can communicate with each other through the communications bus. The processor can invoke a logical instruction in the memory to execute a PET-CT medical image automatic segmentation method based on weakly supervised learning, which includes the following steps:

[0132] S1, obtain a PET image and a CT image, and perform preprocessing;

[0133] S2, add weak annotation information containing only lesion point annotations to the PET image and the CT image, and then generate pseudo labels based on the weak annotation information by using a multi-modal fusion pseudo label generation algorithm; a weakly supervised dataset is constructed by using the obtained PET image, CT image, and corresponding pseudo labels;

[0134] S3, construct an image segmentation network of an encoder-decoder architecture, wherein the encoder comprises a plurality of convolutional layers and down-sampling layers, the decoder comprises a plurality of convolutional layers and up-sampling layers, a neck Transformer module based on a learnable sparse mechanism is designed at the deepest layer of the encoder, multi-scale skip connections are adopted between the encoder and the decoder, and each convolutional layer of the image segmentation network adopts an adaptive multi-kernel convolution module; the PET image and the CT image are processed by the image segmentation network, and a segmentation result of a target region is output;

[0135] S4, train the image segmentation network by using the constructed weakly supervised dataset, and obtain a trained PET-CT medical image automatic segmentation model;

[0136] S5. Input the PET image and CT image to be segmented into the PET-CT medical image automatic segmentation model to predict the segmented image of the target lesion.

[0137] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0138] Example 4: This example provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute an automatic segmentation method for PET-CT medical images based on weakly supervised learning. This method includes the following steps:

[0139] S1. Acquire PET and CT images and perform preprocessing;

[0140] S2. Add weak annotation information containing only lesion point annotations to the PET images and CT images, and then generate pseudo-labels based on the weak annotation information using a multimodal fusion pseudo-label generation algorithm; construct a weakly supervised dataset using the obtained PET images, CT images and corresponding pseudo-labels.

[0141] S3. Construct an image segmentation network with an encoder-decoder architecture, wherein the encoder includes multiple convolutional layers and downsampling layers, and the decoder includes multiple convolutional layers and upsampling layers. A neck Transformer module based on a learnable sparse mechanism is designed in the deepest layer of the encoder. A multi-scale skip connection is used between the encoder and the decoder. Each convolutional layer of the image segmentation network uses an adaptive multi-kernel convolutional module. The PET image and CT image are processed by the image segmentation network to output the segmentation result of the target region.

[0142] S4. The image segmentation network is trained using the constructed weakly supervised dataset to obtain a trained PET-CT medical image automatic segmentation model.

[0143] S5. Input the PET image and CT image to be segmented into the PET-CT medical image automatic segmentation model to predict the segmented image of the target lesion.

[0144] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0145] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic segmentation method for PET-CT medical images based on weakly supervised learning, characterized in that, Includes the following steps: S1. Acquire PET and CT images and perform preprocessing; S2. Add weak annotation information containing only lesion point annotations to the PET images and CT images, and then generate pseudo-labels based on the weak annotation information using a multimodal fusion pseudo-label generation algorithm; construct a weakly supervised dataset using the obtained PET images, CT images and corresponding pseudo-labels. S3. Construct an image segmentation network with an encoder-decoder architecture, wherein the encoder includes multiple convolutional layers and downsampling layers, and the decoder includes multiple convolutional layers and upsampling layers. A neck Transformer module based on a learnable sparse mechanism is designed in the deepest layer of the encoder. A multi-scale skip connection is used between the encoder and the decoder. Each convolutional layer of the image segmentation network uses an adaptive multi-kernel convolutional module. The PET image and CT image are processed by the image segmentation network to output the segmentation result of the target region. S4. The image segmentation network is trained using the constructed weakly supervised dataset to obtain a trained PET-CT medical image automatic segmentation model. S5. Input the PET image and CT image to be segmented into the PET-CT medical image automatic segmentation model to predict the segmented image of the target lesion.

2. The automatic segmentation method for PET-CT medical images based on weakly supervised learning according to claim 1, characterized in that, Step S1 involves preprocessing the acquired PET and CT images, specifically as follows: The original images of the two modalities are registered to spatial coordinates and aligned, and then resampled to the same resolution to correspond one-to-one with their respective lesion delineation files. Windowing was applied to CT images, noise reduction was applied to PET images, and finally, normalization was performed.

3. The automatic segmentation method for PET-CT medical images based on weakly supervised learning according to claim 1, characterized in that, In step S2, weak annotation information containing only lesion point annotations is added to the PET and CT images. Then, based on this weak annotation information, a multimodal fusion pseudo-label generation algorithm is used to generate pseudo-labels; specifically: Based on PET and CT images of diffuse large B-cell lymphoma, an annotation point is provided on each of the six planes of the tumor, and an annotation point is provided inside the tumor as weak annotation information containing lesion point annotations. A weighted image fusion strategy is used to fuse PET and CT images; A geodesic transform method combining intensity information and spatial distance from the fused image is used to calculate the distance between voxels by balancing the weights of the two distances using parameters, expressed as: ； in, For spatial distance, For image gradient, The geodesic distance between voxels. These are weight parameters; Pseudo-labels are generated based on geodesic distances.

4. The automatic segmentation method for PET-CT medical images based on weakly supervised learning according to claim 1, characterized in that, In step S3, the image segmentation network first extracts low-level detail features from PET and CT images respectively, and then stitches the extracted features together along the channel dimension through the image fusion module to obtain a multimodal fusion feature map. The multimodal fusion feature map is then gradually extracted into high-level semantic features by the encoder.

5. The automatic segmentation method for PET-CT medical images based on weakly supervised learning according to claim 1, characterized in that, Each convolutional layer of the image segmentation network described in step S3 uses an adaptive multi-kernel convolutional module, which combines an attention weighting module and a multi-branch convolutional module. The multi-branch convolution module has k three-dimensional convolutional kernel branches with the same spatial dimension but different parameters, and the attention weighting module weights the different three-dimensional convolutional kernel branches; in each convolutional processing stage, for the first... The convolution output of each 3D convolutional kernel branch is ,enter After global pooling, convolution, normalization, and activation operations, the output is gradually updated to an attention mask, represented as: ； Indicates the first The attention mask corresponding to each convolutional kernel branch, and the final output feature map are represented as follows: ； in, Indicates the first The output feature map of each three-dimensional convolutional kernel branch.

6. The automatic segmentation method for PET-CT medical images based on weakly supervised learning according to claim 1, characterized in that, In step S3, the deepest layer of the encoder, the neck Transformer module based on a learnable sparse mechanism, first performs patch segmentation and flattening through 3D convolution. Then, it adopts a hierarchical hybrid attention strategy, selectively introducing a learnable sparse mechanism according to the layer position. First, it generates three feature mapping matrices corresponding to the query, key, and value through fully connected layers. , and , represented as: ； ； ； Using the input features, the original fully connected attention score is then calculated as follows: ； in, For batch index, The first in the attention of the bulls Size, For each head dimension, for sequence position, for The sequence position; Then, learnable sparsity is performed for each Only keep the ones most relevant to it. If there are 1 key, then: ； ； ； in, As a learnable parameter, and through Constrain it to 0-1. It is the sparsity ratio parameter. It is the length of the flattened input sequence. Indicates the first The most relevant query location The set of indices for each key; the sparse attention score matrix is: ； right After activation by the softmax function, we get The attention output is: ； in, Indicates position Position Attention weights Indicates position The Value vector, Indicates position The final output feature vector is then transformed into a format suitable for network processing through dimension reshaping.

7. The automatic segmentation method for PET-CT medical images based on weakly supervised learning according to claim 1, characterized in that, Step S4, which involves training the image segmentation network using the constructed weakly supervised dataset, specifically involves: The weakly supervised dataset constructed in step S2 is divided into a training set, a validation set, and a test set in a ratio of 6:2:2; The divided training set is fed into the image segmentation network. Based on the image input in the training set, the corresponding target segmentation region output is calculated, compared with the pseudo label, and the error is calculated using the loss function and backpropagated to update the network weights. At the end of each epoch, the model is validated using a validation set to evaluate its performance on unseen data. Hyperparameters are then tuned based on the loss and performance metrics of the validation set, and the optimal network parameters are saved based on the performance of the validation set.

8. A non-transitory computer-readable storage medium, characterized in that, It stores computer instructions that cause the computer to execute the PET-CT medical image automatic segmentation method based on weakly supervised learning as described in any one of claims 1-7.

9. An electronic device, characterized in that, include: The system includes a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The processor calls logical instructions from the memory to execute the PET-CT medical image automatic segmentation method based on weakly supervised learning as described in any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer performs the automatic segmentation method for PET-CT medical images based on weakly supervised learning as described in any one of claims 1-7.

Images