A segmentation method and system based on multi-task assistance and global semantic guidance

Abstract

The application discloses a kind of segmentation method and system based on multi-task auxiliary and global semantic guidance, belong to image processing technical field.The application creates the advantages in that multi-task learning is introduced, especially multi-task cross scale transformer, effectively solve the problem of lesion size difference in lymphoma PET / CT segmentation, so that network can adapt to different size lesions.Change old multi-task workflow, especially classification guided dynamic convolution, improve the combination ability of segmentation network to classification task result, enhance robustness.Use feature-level fusion UNet as backbone network, effectively handle the image appearance difference between PET and CT, improve the performance of network in lymphoma PET / CT segmentation task.Through multi-task supervision and the design of comprehensive loss function, fully consider pixel regression, classification and segmentation task, so that network learns more comprehensively in training process and improves segmentation performance.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and relates to a segmentation method and system based on multi-task assistance and global semantic guidance. Background Technology

[0002] In the background of PET / CT segmentation of lymphoma, the main technology involves the combined application of two major medical imaging techniques: positron emission tomography (PET) and computed tomography (CT). This combined application can provide more comprehensive information for cancer diagnosis and treatment planning. Specifically, PET imaging uses injected radioactive tracers to reflect the metabolic activity of tissues, while CT imaging provides anatomical information about tissue structure and location. Lymphoma, as one of the most common cancers worldwide, is particularly important for accurate segmentation and localization. Manually annotating lymphoma lesions is both time-consuming and subject to observer variability, making automated lymphoma segmentation methods crucial. However, existing techniques face two main challenges in handling lymphoma PET / CT segmentation: first, the size variation of lesions in the image may lead to oversegmentation or lesion omission; second, additional assessment of the presence of lesions in the entire image can improve segmentation performance, but requires appropriate task integration. Therefore, there is an urgent need to propose new methods to address these technical challenges and improve the accuracy and adaptability of lymphoma segmentation.

[0003] Defects and shortcomings of existing technology:

[0004] 1. Ignoring differences in lesion size: Current techniques often struggle to handle variations in lesion size within images when segmenting lymphomas. Smaller lesions are easily overlooked, while larger lesions may be over-segmented, affecting segmentation accuracy.

[0005] 2. Accumulated errors caused by cascaded multitasking: Existing technologies often use classification prediction to determine whether the entire image contains lesions before segmenting the lesions. This approach causes errors from the classification task to accumulate in the segmentation task and cannot be corrected by a loss function. Summary of the Invention

[0006] The purpose of this invention is to solve the problems in the prior art and provide a segmentation method and system based on multi-task assistance and global semantic guidance.

[0007] To achieve the above objectives, the present invention employs the following technical solution:

[0008] The present invention proposes a segmentation method based on multi-task assistance and global semantic guidance, comprising the following steps:

[0009] Acquire medical cross-sectional images after multi-task cross-scale transformer, classification-guided dynamic convolution, and preprocessing.

[0010] The UNet with feature-level fusion is used as the backbone network, and a segmentation network is formed by combining a multi-task cross-scale transformer and classification-guided dynamic convolution.

[0011] The preprocessed medical cross-sectional image is input into the segmentation network to obtain a segmented image of systemic lymphoma, thereby achieving image segmentation.

[0012] Preferably, the method for obtaining the preprocessed medical cross-sectional image is as follows:

[0013] The electron emission computed tomography (EECT) images and computed tomography (CT) images were normalized, and then rigid image registration and cropping were performed to obtain preprocessed medical cross-sectional images.

[0014] Preferably, the implementation method of the multi-task cross-scale transformer is as follows:

[0015] The decoder part of the backbone network generates image features at different resolutions. By generating query vectors at high resolution and key vectors and value vectors at low resolution, cross-scale attention at different scales is calculated to obtain the percentage of lesions in the region corresponding to each feature point, forming a regression prediction map.

[0016] The cls token is inserted at the end of the query vector. While calculating the attention cross attention, the attention between the cls token and the key vector and the value vector is also calculated to predict the classification result and obtain the classification features.

[0017] Preferably, the implementation method of classification-guided dynamic convolution is as follows:

[0018] By classifying feature F cls The parameter generator G(·) generates the first dynamic convolution parameters W. dyn Second dynamic convolution parameter b dyn Using the first dynamic convolution parameter W dyn Second dynamic convolution parameter b dyn For fusion feature F fus Perform convolution operations to obtain the final segmentation result P. fus .

[0019] Preferably, the input to the parameter generator is the fused feature F. fus After average pooling aggregation, and with classification feature F cls The connection was obtained.

[0020] Preferably, the backbone network processing method is as follows:

[0021] Connect consecutive PET slices and CT images in the same dimension to form the first input I. PET Second input I CT An independent encoder is used for the first input I. PET Second input I CT The segments are encoded, then concatenated along the channel dimension and fed into the decoding block for reconstruction, ultimately yielding the segmentation result.

[0022] Preferably, the segmentation network uses cross-entropy loss and Dice loss to supervise the final segmentation result.

[0023] This invention proposes a segmentation system based on multi-task assistance and global semantic guidance, comprising:

[0024] The information acquisition module is used to acquire medical cross-sectional images after multi-task cross-scale transformer, classification-guided dynamic convolution, and preprocessing.

[0025] The network construction module is used to take the feature-level fused UNet as the backbone network and combine it with multi-task cross-scale transformer and classification-guided dynamic convolution to form a segmentation network.

[0026] The image processing module is used to input the preprocessed medical cross-sectional image into the segmentation network to obtain a segmented image of systemic lymphoma, thereby achieving image segmentation.

[0027] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement steps of a segmentation method based on multi-task assistance and global semantic guidance.

[0028] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a segmentation method based on multi-task assistance and global semantic guidance.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] This invention proposes a segmentation method based on multi-task assistance and global semantic guidance. The advantages of this invention lie in the introduction of multi-task learning, particularly a multi-task cross-scale transformer, which effectively addresses the issue of lesion size differences in lymphoma PET / CT segmentation, enabling the network to adapt to lesions of varying sizes. It changes the traditional multi-task workflow, especially the classification-guided dynamic convolution, improving the segmentation network's ability to integrate classification task results and enhancing robustness. Employing a feature-level fusion-based UNet as the backbone network effectively handles the image appearance differences between PET and CT, improving the network's performance in lymphoma PET / CT segmentation. Therefore, this invention achieves higher accuracy, adaptability, and robustness in lymphoma PET / CT segmentation.

[0031] Furthermore, by designing a multi-task supervision and comprehensive loss function, pixel regression, classification, and segmentation tasks are fully considered, enabling the network to learn more comprehensively during training and improving segmentation performance.

[0032] This invention proposes a segmentation system based on multi-task assistance and global semantic guidance. By dividing the system into an information acquisition module, a network construction module, and an image processing module, image segmentation is achieved. The modular approach ensures that each module is independent, facilitating unified management of all modules. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart of the segmentation method based on multi-task assistance and global semantic guidance of the present invention.

[0035] Figure 2 This is a structural diagram of the systemic lymphoma segmentation network based on multi-task assistance and global semantic guidance of the present invention.

[0036] Figure 3 This is a block diagram of the multi-task cross-scale transformer of the present invention.

[0037] Figure 4 This is a multi-task prediction result diagram of lesions of different sizes throughout the body according to the present invention.

[0038] Figure 5 This is a diagram of the segmentation system based on multi-task assistance and global semantic guidance of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0040] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0041] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0042] In the description of the embodiments of the present invention, it should be noted that if terms such as "upper," "lower," "horizontal," or "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, terms such as "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0043] Furthermore, the use of the term "horizontal" does not imply that the component must be absolutely horizontal, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0044] In the description of the embodiments of the present invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0045] The present invention will now be described in further detail with reference to the accompanying drawings:

[0046] This invention proposes a segmentation method based on multi-task assistance and global semantic guidance, such as... Figure 1 As shown, it includes the following steps:

[0047] S1. Obtain medical cross-sectional images after multi-task cross-scale transformer, classification-guided dynamic convolution and preprocessing.

[0048] The method for obtaining preprocessed medical cross-sectional images is as follows:

[0049] The electron emission computed tomography (EECT) images and computed tomography (CT) images were normalized, and then rigid image registration and cropping were performed to obtain preprocessed medical cross-sectional images.

[0050] The implementation method of the multi-task cross-scale transformer is as follows:

[0051] The decoder part of the backbone network generates image features at different resolutions. By generating query vectors at high resolution and key vectors and value vectors at low resolution, cross-scale attention at different scales is calculated to obtain the percentage of lesions in the region corresponding to each feature point, forming a regression prediction map.

[0052] The cls token is inserted at the end of the query vector. While calculating the attention cross attention, the attention between the cls token and the key vector and the value vector is also calculated to predict the classification result and obtain the classification features.

[0053] The implementation method of classification-guided dynamic convolution is as follows:

[0054] By classifying feature F cls The parameter generator G(·) generates the first dynamic convolution parameters W. dyn Second dynamic convolution parameter b dyn Using the first dynamic convolution parameter W dyn Second dynamic convolution parameter b dyn For fusion feature F fus Perform convolution operations to obtain the final segmentation result P. fus .

[0055] The input to the parameter generator is the fused feature F fus After average pooling aggregation, and with classification feature F cls The connection was obtained.

[0056] S2. Use the UNet with feature-level fusion as the backbone network, and combine it with multi-task cross-scale transformer and classification-guided dynamic convolution to form a segmentation network.

[0057] The backbone network processing method is as follows:

[0058] Connect consecutive PET slices and CT images in the same dimension to form the first input I. PET Second input I CT An independent encoder is used for the first input I. PET Second input I CT The segments are encoded, then concatenated along the channel dimension and fed into the decoding block for reconstruction, ultimately yielding the segmentation result.

[0059] S3. Input the preprocessed medical cross-sectional image into the segmentation network to obtain the segmentation image of systemic lymphoma and achieve image segmentation.

[0060] The segmentation network uses cross-entropy loss and Dice loss to supervise the final segmentation result.

[0061] See Figure 2 The segmentation method proposed in this invention first performs image preprocessing based on medical knowledge, uses a feature-level fusion UNet as the backbone network, employs a multi-task cross-scale transformer to handle the size differences of lymphoma slices, and introduces classification-guided dynamic convolution to combine the prediction results of multiple tasks. These schemes, through multi-task supervision and training guided by a comprehensive loss function, jointly construct the network of this invention, aiming to improve the accuracy of lymphoma PET and CT segmentation. The detailed steps are as follows:

[0062] Step 1: Image Preprocessing Based on Medical Knowledge: First, normalization was performed on PET and CT images based on medical knowledge. Then, rigid image registration was performed. Since radiologists manually annotate PET images and maintain the same resolution between the CT images and the annotations, trilinear interpolation was used to resample the CT images. Finally, the center of the image projected onto the cross-section was cropped to obtain the cross-sectional image, which was used for subsequent training and inference of the segmentation network.

[0063] The subsequent segmentation network uses UNet, a feature-level fusion network, as its backbone to perform initial image segmentation. However, the segmentation effect of the backbone network is very limited. Therefore, this backbone network is designed with two modules: a multi-task cross-scale transformer (MCT) and a classification-guided dynamic convolution (CDC) module to post-process the segmentation results and improve segmentation accuracy. The backbone network and the two modules are described below according to the network processing flow.

[0064] Step 2: Feature-level fusion backbone network: The preprocessed PET and CT images are directly input into this backbone network to obtain coarse segmentation results. Due to the significant differences in the appearance of PET and CT images, this backbone network uses a feature-level fusion UNet as its backbone network.

[0065] The specific processing steps of the backbone network are as follows: By concatenating three consecutive PET slices and CT images in the channel dimension, two inputs (I... PET and I CT Using an independent encoder for I. PET and I CT The segments are encoded, then concatenated along the channel dimension and fed into the decoding block for reconstruction, yielding the final segmentation result. This feature-level fusion strategy helps improve the network's adaptability to different image appearances.

[0066] To further address the challenge of image processing with significant differences in appearance, a multi-task cross-scale transformer and classification-guided dynamic convolution are introduced. The combination of these two modules improves the performance of the backbone network and, as a post-processing step, refines the results. The multi-task cross-scale transformer guides pixel regression and overall image classification tasks by computing attention at different scales, thereby better capturing features of lesions of varying sizes. The classification-guided dynamic convolution dynamically generates convolution parameters based on the overall image context information to adapt to the PET ingestion value distribution of different images, improving the network's robustness.

[0067] Step 3, Multi-task Cross-scale Transformer: Since lymphoma is a systemic disease, slice sizes vary greatly. To reduce the impact of lesion size differences, this backbone network incorporates the MCT module, which is the core and most innovative module in the network. The MCT block computes multi-scale attention, guiding the pixel regression task to combine global contextual information from the classification task, thus improving the segmentation performance of the UNet backbone network. The MCT and backbone network operate in parallel; the output features of the MCT are fused with the output features of the backbone network to obtain fused features, and an additional classification result for the entire image is output.

[0068] like Figure 3 As shown, the core module MCT proposed by the backbone network of this invention is compared with traditional methods. MCT can avoid mutual interference between different tasks due to hard fusion, and at the same time has cross-scale attention computing capability, thus improving segmentation performance.

[0069] The specific implementation steps of MCT are as follows: First, MCT introduces a novel multi-task cross-scale transformer, which simultaneously performs pixel regression and overall image classification tasks at multiple scales. Specifically, the decoder part of the backbone network UNet can generate image features at different resolutions. This is achieved by generating query vectors at high resolution and key and value vectors at low resolution, and then calculating cross-scale attention at different scales. Afterwards, the percentage of lesions in the region corresponding to each feature point is predicted, forming a regression prediction map with feature F. regThe F1 feature of the backbone network is fused into F fus Cross-scale attention helps the network capture features of large lesions at low resolution and features of small lesions at high resolution.

[0070] Secondly, to mitigate the impact of large lesions, a classification task is introduced. This task involves inserting a cls token at the end of the query vector. While calculating cross-attention, the attention between the cls token and the key vector is also calculated, along with the attention between the value vector and the key vector, to predict the classification result P. cls And obtain the classification feature F cls The classification task determines the presence of lesions in an image through soft classification (not directly deciding whether to segment) and incorporates semantic information from the entire slice. This helps the network better handle the contextual information of large lesions in image classification, improving segmentation accuracy. This method only requires one attention computation to obtain the results of both tasks, reducing the negative impact of multi-task methods.

[0071] Step 4, Classification-Guided Dynamic Convolution: This CDC module dynamically adjusts the convolution parameters based on the prediction results of the classification task. Located at the end of the entire network process, it refines the segmentation results of the backbone network and outputs the final segmentation result.

[0072] The specific implementation steps of the CDC module are as follows: First, CDC uses dynamic convolution parameters to process the F after feature fusion. fus A parameter generator G(·) is introduced to generate dynamic convolution parameters, which consists of fully connected layers, ReLU, and softmax. Unlike traditional dynamic convolution, CDC also uses additional classification features F. cls The input to G(·) controls the convolution parameters. This is achieved by inputting F... cls Transform into P representing the semantics of the entire slice cls P cls It is used to guide the generation of dynamic convolution parameters. P cls When the value is close to 1 or 0, it indicates whether there is a lesion in the image, while when the value is close to 0.5, it indicates that the slice may be a difficult sample, which may lead to false detection or false negative.

[0073] Secondly, the dynamic convolution parameters W dyn and b dyn It is generated by a parameter generator G(·), whose input is F. fus After average pooling polymerization, it is combined with F cls Connection. Dynamic convolution uses the generated parameters W. dyn and b dyn For fusion feature F fusPerform convolution operations to obtain the final segmentation result P. fus By introducing CDC, the model can dynamically adjust the convolution parameters based on the classification results of each image, changing the way classification and segmentation tasks are combined.

[0074] Step 5: Multi-task is introduced to address multiple challenges in lymphoma PET / CT image segmentation. In order to enable the network to balance various tasks, different aspects of learning are supervised simultaneously to improve the model's performance and generalization ability.

[0075] The specific details of multi-task supervision are as follows: First, regarding the pixel regression task, for F... reg Downsampling is performed for M reg Used to guide F reg M's learning. reg Each pixel in the image represents the proportion of lesions in the ground truth mask M for the corresponding region.

[0076] Secondly, for the classification task, the model was designed to determine whether lesions exist in the slices. To enhance the supervision of positive samples, a weighted focal loss was used to supervise the classification branch. This loss function generates classification labels based on whether lesions are present in the slices, with larger weights applied when lesions are present and smaller weights applied when they are not.

[0077] For the segmentation task, the model supervises the final segmentation result by introducing cross-entropy loss and Dice loss. These two loss functions consider both the spatial location and pixel-level similarity of the segmentation results. Supervision of the segmentation task is based on the F1 score of features extracted from the backbone network, as the performance of the backbone network directly affects the final segmentation result.

[0078] Finally, the losses from the above tasks are combined to form a comprehensive total loss function. By comprehensively optimizing multiple tasks during training, the model can learn more comprehensive information from different aspects, improving its image understanding and segmentation performance.

[0079] like Figure 2The diagram illustrates the overall framework of the segmentation method. This invention enhances the robustness of segmentation for lesions of varying sizes and semantics. The overall framework comprises the synergistic effects of modules such as the UNet backbone network, a multi-task cross-scale transformer, and classification-guided dynamic convolutions, enabling the network to better process the features of lymphoma PET / CT images and improving segmentation performance. To verify the segmentation performance of this invention and the role of each module, clinical data was collected and labeled. This invention is implemented using the PyTorch framework, and the network was trained using the collected data. Compared with other methods in the same field, this method shows improvements in all performance metrics. Ablation experiments verified the role of each module in the invention, which will be described in detail below with examples:

[0080] First, data preparation: To verify the feasibility of this method, clinical data from 20 cases were collected, none of whom had received radiation therapy prior to data collection. PET and CT images were collected simultaneously for each case. Experienced radiologists manually annotated images using a semi-automatic thresholding method, using the maximum value on the PET image as a benchmark and annotating at 41% of the maximum value. In uncertain areas, the radiologists decided whether to retain the annotation based on the anatomical location on the CT image to avoid false positives. The resolution of the CT images was 0.78 × 0.78 × 0.78 mm. 3 The resolution of the PET image is 4.0×4.0×4.0mm. 3 Rigid image registration was performed first. Since radiologists manually annotate the PET images, the resolution of these manual annotations is also 4.0 × 4.0 × 4.0 mm. 3 .

[0081] To maintain the same resolution between the CT images and the annotations, trilinear interpolation was used to resample the CT images, achieving a resolution of 4.0 × 4.0 × 4.0 mm. 3 After registration, the image projected onto the cross-section is 144×144 pixels. To crop the background area, the cross-sectional image is centered and cropped to 112×112 pixels. After cropping, 180-230 cross-sectional images were obtained from each case. Ultimately, a total of 4241 images were obtained, of which 1701 images (40.1%) showed lesions. These images were then fed into the subsequent segmentation network for training and inference.

[0082] Second, experimental details and evaluation criteria: In the experiment, all 20 patient cases were randomly shuffled. Four patient cases were randomly selected for testing, and the remaining 16 patient cases were shuffled for training. After the cases were divided into training or testing sets, all lateral images of the cases were used only for training or testing. The entire network was optimized using the Adam optimizer with a learning rate of 0.001 and a batch size of 32. All methods were based on the PyTorch framework and trained for 300 iterations on an NVIDIA 3060 GPU. Due to the low performance of the classification head at the beginning of training, plain convolutions were used instead of dynamic convolutions for the first 25 epochs. Classification performance was evaluated using accuracy, recall, and precision. Segmentation performance was evaluated using the Dice coefficient, Hausdorff distance (HD), recall, and precision.

[0083] Third, comparative experiments: To demonstrate the segmentation performance of this method, the same training and test sets will be used to compare it with other existing methods. Depending on the working method, 2D or 3D images can be used to train the network. As shown in Table 1, this method achieved the highest scores in DSC, HD, sensitivity, and accuracy.

[0084] Table 1

[0085]

[0086] Compared to the previous four methods, this method combines PET and CT at the feature level, thus achieving greater improvement. Comparing this method to Zhong's method, it further improves upon this through cross-scale attention and multi-task assistance. Furthermore, it performs better than TransUNet, which introduces a single-scale transform. According to the analysis, TransUNet uses a transform as an encoder, but due to its large number of parameters, it suffers from overfitting.

[0087] As shown in Table 2, a multi-task cascaded version of the method was also set up, which has high recall but low precision because most of the training data for the segmentation network contains lesions. The classification network makes errors and filters out some small lesion images and all-negative images, which need to be learned and distinguished by the segmentation network. Furthermore, a hard multi-task method was compared. Instead of inserting a cls token, an additional classification branch was added to F5, F4, and F3. The canonical multi-task method cannot enable the segmentation task to learn the global semantic information of the auxiliary task, and is therefore more susceptible to data imbalance.

[0088] Table 2

[0089]

[0090] Fourth, ablation experiments: The performance of CDC or MCT was evaluated by replacing CDC with convolution or removing MCT and CDC. As shown in Table 3, combining MCT and CDC improved the backbone network by 7.0%, 3.9%, and 3.8% in Dice, recall, and precision, respectively, with an average HD of 1.74. The variance of each metric decreased to some extent, especially the variance of Dice and precision. Furthermore, using only MCT resulted in a significant drop in recall, while precision only saw a small improvement. Analysis suggests that the classification task's overemphasis on the context of imbalanced data negatively impacted the segmentation network. Therefore, CDC was designed to segment based on classification results. Additionally, partial results for each task are shown below. Figure 4 As shown, from left to right, the images are: original CT image, original PET image, pixel regression map and overall image classification results generated by MCT, backbone segmentation results, final segmentation results and ground truth.

[0091] As can be seen, the volume of lymphoma lesions from different body sites varies greatly, but this method exhibits good segmentation performance for lesions of different sizes. From MCT, the classification task can predict whether an image contains a lesion with a certain accuracy, and difficult images can be inferred from the predicted probability. From MCT, the pixel regression mask can predict the proportion of lesions in a region and distinguish healthy regions with high uptake values. Compared with backbone network segmentation, CDC segmentation is less affected by data imbalance and more closely approximates the ground truth. Furthermore, this method reduces the occurrence of false positives and false negatives.

[0092] Table 3

[0093]

[0094] This invention proposes a segmentation system based on multi-task assistance and global semantic guidance, such as... Figure 5 As shown, it includes an information acquisition module, a network construction module, and an image processing module;

[0095] The information acquisition module is used to acquire multi-task cross-scale transformer, classification-guided dynamic convolution and preprocessed medical cross-sectional images.

[0096] The network construction module is used to use the feature-level fused UNet as the backbone network, and combine it with multi-task cross-scale transformer and classification-guided dynamic convolution to form a segmentation network.

[0097] The image processing module is used to input the preprocessed medical cross-sectional image into the segmentation network to obtain a segmented image of systemic lymphoma, thereby achieving image segmentation.

[0098] The terminal device provided in this embodiment of the invention includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the various method embodiments described above. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the various device embodiments described above.

[0099] The computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention.

[0100] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0101] The processor may be a central processing unit (CPU), or other general-purpose processors, graphics processing units (GPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0102] The memory can be used to store the computer program and / or module. The processor implements various functions of the terminal device by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory.

[0103] If the modules / units integrated into the terminal device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0104] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A segmentation method based on multi-task assistance and global semantic guidance, characterized in that, Includes the following steps: Acquire medical cross-sectional images after multi-task cross-scale transformer, classification-guided dynamic convolution, and preprocessing. The UNet with feature-level fusion is used as the backbone network, and a segmentation network is formed by combining a multi-task cross-scale transformer and classification-guided dynamic convolution. The preprocessed medical cross-sectional image is input into the segmentation network to obtain a segmentation image of systemic lymphoma, thereby achieving image segmentation. The implementation method of the multi-task cross-scale transformer is as follows: The decoder part of the backbone network generates image features at different resolutions. A query vector is generated at high resolution, and key vectors and value vectors are generated at low resolution. Cross-scale attention at different scales is calculated to obtain the percentage of lesions in the region corresponding to each feature point, forming a regression prediction map. The cls token is inserted at the end of the query vector. While calculating cross-attention, the attention between the cls token and the key vector and the value vector is also calculated to predict the classification result and obtain the classification features. The implementation method of classification-guided dynamic convolution is as follows: using classification features F cls The parameter generator G(·) generates the first dynamic convolution parameters W. dyn Second dynamic convolution parameter b dyn Using the first dynamic convolution parameter W dyn Second dynamic convolution parameter b dyn For fusion feature F fus Perform convolution operations to obtain the final segmentation result P. fus The input to the parameter generator is the fused feature F. fus After average pooling aggregation, and with classification feature F cls The connection was obtained.

2. The segmentation method based on multi-task assistance and global semantic guidance according to claim 1, characterized in that, The method for obtaining preprocessed medical cross-sectional images is as follows: The electron emission computed tomography (EECT) images and computed tomography (CT) images were normalized, and then rigid image registration and cropping were performed to obtain preprocessed medical cross-sectional images.

3. The segmentation method based on multi-task assistance and global semantic guidance according to claim 1, characterized in that, The backbone network processing method is as follows: Connect consecutive PET slices and CT images in the same dimension to form the first input I. PET Second input I CT An independent encoder is used for the first input I. PET Second input I CT The segments are encoded, then concatenated along the channel dimension and fed into the decoding block for reconstruction, ultimately yielding the segmentation result.

4. The segmentation method based on multi-task assistance and global semantic guidance according to claim 1, characterized in that, The segmentation network uses cross-entropy loss and Dice loss to supervise the final segmentation result.

5. A segmentation system based on multi-task assistance and global semantic guidance, characterized in that, The segmentation method based on multi-task assistance and global semantic guidance as described in any one of claims 1 to 4 includes: The information acquisition module is used to acquire medical cross-sectional images after multi-task cross-scale transformer, classification-guided dynamic convolution, and preprocessing. The network construction module is used to take the feature-level fused UNet as the backbone network and combine it with multi-task cross-scale transformer and classification-guided dynamic convolution to form a segmentation network. The image processing module is used to input the preprocessed medical cross-sectional image into the segmentation network to obtain a segmented image of systemic lymphoma, thereby achieving image segmentation.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes a computer program, it implements the steps of the segmentation method based on multi-task assistance and global semantic guidance as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the segmentation method based on multi-task assistance and global semantic guidance as described in any one of claims 1 to 4.

Images