Focus segmentation method fusing PET (positron emission tomography) and CT (computed tomography) bimodal images

Abstract

The invention discloses a focus segmentation method fusing PET and CT bimodal images, and relates to the technical field of computers. The method comprises the following steps: acquiring focus image data, separating the focus image data to obtain CT data and PET data, and inputting the CT data and the PET data into a bimodal medical image segmentation model to obtain a preliminary segmentation result; a difference region is generated based on the preliminary segmentation result, difference region detection is carried out on the difference region, and click signal data is generated; the difference region is a region which is not marked as a focus in the preliminary segmentation result; training the bimodal medical image segmentation model for a preset number of times through the click signal data, the CT data and the PET data to obtain a segmentation result; and determining a segmentation result generated by the last training as a lesion image segmentation result. The method can improve the medical image lesion segmentation precision.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a lesion segmentation method for fusing PET and CT dual-modality images.

[0002] Background technology Lesions often exhibit irregular shapes and sizes, adhere to normal tissue, and have blurred edges, making them difficult to accurately distinguish with existing technologies. For example, the grayscale difference between a tumor and surrounding tissue is minimal, easily leading to segmentation boundary errors. Medical targets (such as blood vessels and organs) exhibit significant scale variations across images and exhibit multi-pose characteristics due to changes in imaging angle or patient position, making them difficult to adaptively process with traditional technologies. Differences in imaging parameters across different devices lead to variations in image contrast, impacting segmentation stability.

[0003] Currently, the existing technologies include fully automatic segmentation methods and single-modal analysis methods. The fully automatic segmentation method relies on a large amount of labeled data for training and lacks flexibility; single-modal analysis only uses a single modality of CT or PET for segmentation; user interaction is limited, and most interactive methods only support simple clicks and have limited interactive response capabilities; fixed fusion weights, traditional multimodal fusion uses preset static weights and cannot adapt to the needs of different regions.

[0004] The simple click interactive method in the existing technology results in low accuracy of lesion image segmentation. Summary of the Invention

[0005] Therefore, it is necessary to provide a lesion segmentation method that integrates PET and CT dual-modality images to address the above technical issues. This method can improve the accuracy of lesion image segmentation.

[0006] The present invention adopts the following technical solutions: The present invention provides a lesion segmentation method for fusing PET and CT dual-modality images, comprising: Acquire lesion image data, separate the lesion image data to obtain CT data and PET data, and input the CT data and PET data into a dual-modality medical image segmentation model to obtain preliminary segmentation results; Based on the preliminary segmentation results, a difference region is generated, and a difference region detection is performed on the difference region to generate click signal data; the difference region is the region not marked as a lesion in the preliminary segmentation results; The dual-modality medical image segmentation model is trained a preset number of times by clicking on the signal data, CT data, and PET data to obtain the segmentation results; The segmentation result generated by the last training is determined as the lesion image segmentation result.

[0007] Preferably, the dual-modality medical image segmentation model is trained a preset number of times by using the click signal data, CT data, and PET data to obtain a segmentation result, including: For each of the preset number of trainings, the click signal data is processed by the click encoding module to obtain click features, the CT data is processed by the CT input module to obtain CT features, and the PET data is processed by the PET input module to obtain PET features; In the PET downsampling module of each layer, the PET features and click features are fused through the multi-scale click attention module to obtain the enhanced PET features of each layer; Process the CT features through the CT downsampling module to obtain the CT features of each layer; The enhanced PET features of each layer and the CT features of each layer are fused through the feature fusion module to obtain the output fusion features of the PET downsampling module of each layer; The output fusion features of the last layer are processed through the bottleneck layer and multi-layer upsampling is performed through the upsampling module to obtain the segmentation result; the upsampling module and the PET downsampling module are connected by jumps.

[0008] Preferably, the method further comprises: For each of the preset number of trainings, after the training is completed, new click signal data is generated based on the segmentation results obtained after the training is completed; The generated new click signal data is used as the click signal data for the next round of training.

[0009] Preferably, the multi-scale click attention module includes channel attention, spatial attention and click gating mechanism. The multi-scale click attention module is: ; in, is the feature map, is the click feature, is the batch size, which indicates the number of samples processed at a time, is the number of feature channels, is the channel attention, is a click gating mechanism, Expressed as element-wise multiplication, for spatial attention; CT features are: ; in, Indicates the i layer CT Feature map, Indicates the i layer CT The feature extraction function of the path, for CT A set of network parameters for the path, For the i-1th layer CT Feature map; PET features are: ; in, Indicates the i layer PET Feature map, Represents the i-th layer PET The feature extraction function of the path, for PET A collection of network parameters for the route.

[0010] Preferably, the enhanced PET features of each layer and the CT features of each layer are fused by a feature fusion module to obtain the output fusion features of the PET downsampling module of each layer, specifically including: The CT features of each layer and the enhanced PET features of each layer are normalized to obtain normalized CT features and normalized PET features; Attention enhancement is performed on the normalized CT features and the normalized PET features to obtain attention-enhanced CT features and attention-enhanced PET features; the attention-enhanced CT features are: ; in, Indicates that the attention is enhanced i layer CT feature, Represents the normalized i-th layer CT feature, Indicates the i layer CT Attention map of features, Represents element-wise multiplication; The PET characteristics after attention enhancement are: ; in, Indicates that the attention is enhanced i layer PET feature, Represents the normalized i layer PET feature, Indicates the i layer PET Attention map of features, Represents element-wise multiplication; The attention-enhanced CT features and the attention-enhanced PET features are fused to obtain the output fused features.

[0011] Preferably, the output fusion features include shallow fusion features and deep fusion features; When the number of layers is less than the preset threshold, the output fusion feature is a shallow fusion feature, which is: ; in, for CT The initial weights of the features, for sigmoid activation function, It is the feature after shallow fusion; When the number of layers is greater than or equal to the preset threshold, the output fusion feature is deep feature fusion, and the deep fusion feature is: ; in, for PET Enhancement coefficient, is the residual coefficient, for CT Initial weights of features.

[0012] Preferably, the adaptive method of CT feature weights specifically includes: Initialize the CT feature weight to obtain the initialized CT feature weight; the initialized CT feature weight is: ; in, i is the number of layers; Dynamically adjust the initialized CT feature weights; the corresponding formula for dynamic adjustment is: ; in, is the overall loss function, is the learning rate, is the number of iterations, For the t The first iteration i The weight of the layer.

[0013] Preferably, the difference region detection includes morphological operations and SUV feature weight analysis, performing difference region detection on the difference region and generating click signal data, including: Perform morphological operations on the difference region to obtain the boundary region of the difference region; After obtaining the boundary area of ​​the difference area, the SUV feature weight analysis is performed on the difference area to obtain the weighted difference area; Return the target difference area based on the weighted difference area and boundary area; Random selection is performed based on the target difference area to obtain click signal data.

[0014] Preferably, the morphological operation includes a dilation operation, an erosion operation, and a boundary region extraction, and the morphological operation is performed on the difference region to obtain the boundary region of the difference region, specifically including: Perform a dilation operation on the difference area to obtain the dilation result; the formula corresponding to the dilation operation is: ; in, Indicates the difference area The expansion result, for The three-dimensional structural element, represents the expansion operation, Represented in three-dimensional structural elements Take the maximum value within; Perform an erosion operation on the difference area to obtain an erosion operation; the formula corresponding to the erosion operation is: ; in, Indicates the difference area The corrosion results, for The three-dimensional structural element, represents the corrosion operation, Represented in three-dimensional structural elements Take the minimum value within; The boundary area is calculated based on the erosion and dilation results; the calculation formula for the boundary area is: ; in, Indicates the difference area The boundary area, represents the image after the dilation operation, Represents the image after the erosion operation.

[0015] Preferably, the SUV feature weight analysis includes SUV feature weight calculation and weighted difference map calculation; performing SUV feature weight analysis on the difference area to obtain a weighted difference area specifically includes: The SUV feature weight calculation is performed on the difference area in the PET image to obtain the weight value of each position in the difference area; the corresponding formula for SUV feature weight calculation is: ; in, Indicates location The weight value at Indicates location The SUV value at Indicates the average SUV value of the difference area, represents the SUV standard deviation of the difference area, is a small positive number; Based on the weight value, a weighted difference map is calculated for the difference area to obtain a weighted difference map, and the weighted difference map is determined as the weighted difference area; the corresponding formula for calculating the weighted difference map is: ; in, represents the weighted difference map, Indicates the difference area, represents the boundary area, Represents the weighting factor for the boundary area.

[0016] The present invention provides a lesion segmentation method and device for fusing PET and CT dual-modality images, comprising: A segmentation module is used to obtain lesion image data, separate the lesion image data to obtain CT data and PET data, and input the CT data and PET data into a dual-modality medical image segmentation model to obtain preliminary segmentation results; A generation module is used to generate a difference region based on the preliminary segmentation result, perform difference region detection on the difference region, and generate click signal data; the difference region is the region not marked as a lesion in the preliminary segmentation result; A training module is used to train the dual-modality medical image segmentation model for a preset number of times using click signal data, CT data, and PET data to obtain a segmentation result; The determination module is used to determine the segmentation result generated by the last training as the lesion image segmentation result.

[0017] The present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the computer program implements the above-mentioned method for lesion segmentation by fusing PET and CT dual-modality images.

[0018] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned method for lesion segmentation by fusing PET and CT dual-modality images is implemented.

[0019] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects: Lesion image data is acquired and separated to obtain CT data and PET data. These data are then input into a dual-modality medical image segmentation model to obtain a preliminary segmentation result, leveraging the complementary nature of multimodal features to improve lesion localization accuracy. Based on the preliminary segmentation results, differential regions are generated, and differential region detection is performed on these differential regions to generate click signal data. Interactive segmentation enables the dual-modality medical image segmentation model to dynamically learn and correct intent. The dual-modality medical image segmentation model is trained a preset number of times using click signal data, CT data, and PET data to obtain a segmentation result. The segmentation boundaries are then iteratively optimized through a preset number of rounds. The segmentation result generated from the final training session is determined as the lesion image segmentation result. This method can improve lesion image segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0021] Figure 1 A schematic flow chart of a lesion segmentation method for fusing PET and CT dual-modality images provided by the present invention; Figure 2 A click operation diagram of the system provided by the present invention; Figure 3 Click to generate a preview of the mask provided by the system of the present invention; Figure 4 Click to generate a mask map for the system provided by the present invention; Figure 5 A diagram of the model structure provided by the present invention; Figure 6 A schematic diagram of a lesion segmentation device for fusing PET and CT dual-modality images provided by the present invention; Figure 7 A schematic diagram of a computer device for a lesion segmentation model method that integrates PET and CT dual-modality images provided by the present invention. DETAILED DESCRIPTION

[0022] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0023] Devices such as desktop computers, servers, and laptop computers that can execute the solution of the present invention are described below with the server as the execution subject for the sake of convenience.

[0024] The technical solutions provided by various embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0025] Figure 1 The figure is a flowchart of a lesion segmentation method for fusing PET and CT dual-modality images in the present invention, which specifically includes the following steps: S101: Acquire lesion image data, separate the lesion image data to obtain CT data and PET data, and input the CT data and PET data into a dual-modality medical image segmentation model to obtain a preliminary segmentation result.

[0026] Specifically, the CT data and PET data are fused to generate preliminary segmentation results.

[0027] S102: generating a difference region based on the preliminary segmentation result, performing difference region detection on the difference region, and generating click signal data; the difference region is the region not marked as a lesion in the preliminary segmentation result.

[0028] In an exemplary embodiment, difference area detection includes morphological operations and SUV feature weight analysis, and difference area detection is performed on the difference area to generate click signal data, including: performing morphological operations on the difference area to obtain the boundary area of ​​the difference area; after obtaining the boundary area of ​​the difference area, performing SUV feature weight analysis on the difference area to obtain a weighted difference area; returning the target difference area based on the weighted difference area and the boundary area; and performing random selection based on the target difference area to obtain click signal data.

[0029] In an exemplary embodiment, the morphological operation includes a dilation operation, an erosion operation, and a boundary region extraction. The morphological operation is performed on the difference region to obtain the boundary region of the difference region. Specifically, the morphological operation includes: performing a dilation operation on the difference region to obtain a dilation result. The formula corresponding to the dilation operation is shown in formula (1): (1); in, Indicates the difference area The expansion result, for The three-dimensional structural element, represents the expansion operation, Represented in three-dimensional structural elements Take the maximum value within; The difference area is corroded to obtain the corrosion operation; the formula corresponding to the corrosion operation is shown in formula (2): (2); in, Indicates the difference area The corrosion results, for The three-dimensional structural element, represents the corrosion operation, Represented in three-dimensional structural elements Take the minimum value within; The boundary area is calculated based on the corrosion results and the expansion results. The calculation formula of the boundary area is shown in formula (3): (3); in, Indicates the difference area The boundary area, represents the image after the dilation operation, Represents the image after the erosion operation.

[0030] In an exemplary embodiment, the SUV feature weight analysis includes SUV feature weight calculation and weighted difference map calculation; performing SUV feature weight analysis on the difference region to obtain a weighted difference region, specifically including: performing SUV feature weight calculation on the difference region in the PET image to obtain a weight value for each position in the difference region; the formula corresponding to the SUV feature weight calculation is shown in formula (4): (4); in, Indicates location The weight value at Indicates location The SUV value at Indicates the average SUV value of the difference area, represents the SUV standard deviation of the difference area, A small positive number.

[0031] Based on the weight value, a weighted difference map is calculated for the difference area to obtain a weighted difference map, and the weighted difference map is determined as the weighted difference area; the formula corresponding to the weighted difference map calculation is shown in formula (5): (5); in, represents the weighted difference map, Indicates the difference area, represents the boundary area, Represents the weighting factor for the boundary area S103: The dual-modality medical image segmentation model is trained for a preset number of times using the click signal data, the CT data, and the PET data to obtain a segmentation result.

[0032] The preset number is set according to specific engineering practice, for example, it is set to 2.

[0033] In an exemplary embodiment, the dual-modal medical image segmentation model is trained a preset number of times using click signal data, CT data, and PET data to obtain a segmentation result, including: for each of the preset number of trainings, processing the click signal data through a click encoding module to obtain click features, processing the CT data through a CT input module to obtain CT features, and processing the PET data through a PET input module to obtain PET features; in the PET downsampling module of each layer, the PET features and the click features are fused through a multi-scale click attention module to obtain enhanced PET features of each layer; the CT features are processed through a CT downsampling module to obtain CT features of each layer; the enhanced PET features of each layer and the CT features of each layer are fused through a feature fusion module to obtain output fusion features of the PET downsampling module of each layer; the output fusion features of the last layer are processed through a bottleneck layer, and multi-layer upsampling is performed through an upsampling module to obtain a segmentation result; the upsampling module and the PET downsampling module are connected through jumps.

[0034] like Figure 2 The figure shows a schematic diagram of the dual-modal medical image segmentation model provided by the present invention. The dual-modal medical image segmentation model input starts from the input, separates the CT data, PET data and click signal, and then obtains click features through click coding. The CT data is feature extracted and combined with residual coding to obtain CT features. The PET data is extracted to obtain PET features through PET feature extraction. After that, the click features of each layer are fused with the PET features of each layer or the features of the PET downsampling after the multi-scale click attention mechanism to obtain the corresponding main path features. After that, the CT features of each layer or the features of the CT downsampling module are fused with the main path features through the fusion feature. Finally, after the bottleneck layer and upsampling of each layer, the output prediction result is obtained.

[0035] CT features are extracted through the CT input module, PET features are extracted through the PET input module, CT feature downsampling is processed through the CT downsampling module, PET feature downsampling is processed through the PET downsampling module, click feature extraction is generated through the click encoding module, and feature fusion is performed through the feature fusion module.

[0036] The preset number of times is set according to engineering practice.

[0037] In an exemplary embodiment, the method further includes: for each of the preset number of trainings, after the training is completed, generating new click signal data based on the segmentation results obtained after the training is completed; and using the generated new click signal data as the click signal data for the next round of training.

[0038] Specifically, the first step is to extract CT features and PET features respectively through the dual-path structure, perform preliminary processing on the obtained features, and then perform a hierarchical feature fusion strategy to fuse the features of each layer accordingly. The CT feature is shown in formula (6):

[0039] (6); in, Indicates the i layer CT Feature map, Indicates the i layer CT The feature extraction function of the path, for CT A set of network parameters for the path, For the i-1th layer CT Feature map; The PET characteristics are shown in formula (7): (7); in, Indicates the i layer PET Feature map, Represents the i-th layer PET The feature extraction function of the path, for PET A collection of network parameters for the route.

[0040] Assume that the CT input is , PET input is .

[0041] In an exemplary embodiment, the enhanced PET features of each layer and the CT features of each layer are fused by a feature fusion module to obtain the output fusion features of each layer of the PET downsampling module, specifically including: normalizing the CT features of each layer and the enhanced PET features of each layer to obtain normalized CT features and normalized PET features; performing attention enhancement on the normalized CT features and the normalized PET features to obtain attention-enhanced CT features and attention-enhanced PET features; the attention-enhanced CT features are shown in formula (8): (8); in, Indicates that the attention is enhanced i layer CT feature, Represents the normalized i-th layer CT feature, Indicates the i layer CT Attention map of features, Represents element-wise multiplication.

[0042] The PET features after attention enhancement are shown in formula (9): (9); Indicates that the attention is enhanced i layer PET feature, Represents the normalized i layer PET feature, Indicates the i layer PET Attention map of features, Represents element-wise multiplication.

[0043] The attention-enhanced CT features and the attention-enhanced PET features are fused to obtain the output fused features.

[0044] In an exemplary embodiment, the output fusion feature includes a shallow fusion feature and a deep fusion feature; when the number of layers is less than a preset threshold, the output fusion feature is a shallow fusion feature, and the shallow fusion feature is shown in formula (10): (10); in, for CT The initial weights of the features, for sigmoid activation function, It is the feature after shallow fusion; When the number of layers is greater than or equal to the preset threshold, the output fusion feature is deep feature fusion, and the deep fusion feature is shown in formula (11): (11); in, for PET Enhancement coefficient, is the residual coefficient, for CT Initial weights of features.

[0045] In an exemplary embodiment, the adaptive method of the CT feature weight specifically includes: initializing the CT feature weight to obtain the initialized CT feature weight; the initialized CT feature weight is shown in formula (12): (12); Where i is the number of layers; the initialized CT feature weights are dynamically adjusted; the formula corresponding to the dynamic adjustment is shown in formula (13): (13); in, is the overall loss function, is the learning rate, is the number of iterations, is the tth iteration i The weight of the layer.

[0046] Specifically, the second step is to implement a hierarchical feature fusion strategy, fuse the features of each layer accordingly, and normalize the features first, which are: 、 ; Then, attention enhancement is performed, which are and ; Finally, the enhanced features are and .in, Indicates that the attention is enhanced i Layer CT features; Indicates that the attention is enhanced i Layer PET characteristics; represents element-wise multiplication (Hadamard product); + represents residual connection, which ensures effective transmission of information and stable flow of gradients.

[0047] In an exemplary embodiment, the multi-scale click attention module includes channel attention, spatial attention, and click gating mechanisms. The formula corresponding to the multi-scale click attention module is shown in Formula (14): (14); in, is the feature map, is the click feature, is the batch size, which indicates the number of samples processed at a time, is the number of feature channels, is the channel attention, is a click gating mechanism, Expressed as element-wise multiplication, For spatial attention.

[0048] Specifically, in the third step: First, the multi-scale click attention module receives two inputs, namely feature maps: And click characteristics: ,in, is the batch size, which indicates the number of samples processed at a time, The number of feature channels represents the channel dimension of the feature map, is the number of click feature channels, are spatial dimensions, representing the height, width, and depth of the feature map, respectively. Secondly, a triple attention mechanism is implemented, comprising channel attention, spatial attention, and click gating. Finally, multi-scale feature transfer is performed, which includes scale adaptation and hierarchical attention fusion.

[0049] The overall feature enhancement effect is shown in formula (15): (15); in, is the weight coefficient of each scale, indicating the importance of features at different scales. is the number of network layers, indicating the number of scales, Represents enhanced multi-scale features.

[0050] S104: Determine the segmentation result generated by the last training as the lesion image segmentation result.

[0051] The final segmentation mask will be generated after a specified number of times or when the best segmentation effect is achieved.

[0052] The present invention achieves more accurate medical image segmentation by innovatively combining CT and PET image features and fusing interactive click signals. The method introduces an adaptive feature fusion mechanism and a multi-scale click attention module. In addition, an improved difference area detection method is proposed based on the click point selection strategy. By combining morphological operations and the SUV features of PET images, more accurate lesion area positioning is achieved. This method not only considers simple pixel-level differences, but also introduces boundary area enhancement and SUV value weighting mechanisms, which significantly improves the accuracy of click position selection in interactive segmentation. Finally, the present invention designs a visualization interface software based on the method of the present invention.

[0053] The input data is separated to obtain CT data, PET data, and click signal data respectively. The click signal data obtains click features through the click encoding module, the CT data obtains CT features through the CT input module, and the PET data obtains PET features through the PET input module. After obtaining the relevant features, the PET features in the PET downsampling module of each layer will first be fused with the click features through the multi-scale click attention module to obtain the main path features. The main path features of each layer and the CT features obtained by the CT downsampling module are then combined with the CT features obtained by the CT downsampling module through the feature fusion module to obtain the final output features of the PET downsampling module of each layer. The features output by the CT downsampling module are processed at the bottleneck layer and then output by the upsampling module at each layer. The PET downsampling and upsampling are connected through jumps to better restore image details.

[0054] After processing these, after obtaining the preliminary prediction results, we compare them with the actual results, obtain the difference areas, and then comprehensively improve the difference area detection to generate click signal data again, combine it with CT data and PET data to generate new input data for retraining.

[0055] After a specified number of trainings, the final output is the lesion segmentation result.

[0056] In an exemplary embodiment, the present invention provides Figure 2 The system click operation diagram shown, Figure 2 The red area in the figure is the simulated click signal. Combining CT data and PET data, the segmentation mask is generated by the model. The segmentation mask is as follows: Figure 2 The green part is shown in the figure. Figure 3 Click to generate a mask preview image for the system provided by the present invention, that is, the mask generated by the first segmentation, the lesion area is as follows Figure 3 The green part is shown in the figure. Figure 4 The system provided by the present invention generates a mask map by clicking on it, that is, the final segmentation mask is generated after multiple clicks. Figure 4 The green part in the figure is the final lesion area provided by the present invention. Figure 5 The model structure diagram provided by the present invention. The segmentation mask can be understood as the lesion area mentioned in the present invention.

[0057] When applying the lesion segmentation method provided by the present invention that integrates PET and CT dual-modality images, it is not necessary to Figure 1 The steps are executed in the order shown. The specific execution order of the steps can be determined according to needs, and the present invention does not limit this.

[0058] The above is a method for lesion segmentation by fusing PET and CT dual-modality images provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding device for lesion segmentation by fusing PET and CT dual-modality images, such as Figure 6 shown.

[0059] Figure 6 A schematic diagram of a lesion segmentation device for fusing PET and CT dual-modality images provided by the present invention, comprising: The segmentation module 601 is used to obtain lesion image data, separate the lesion image data to obtain CT data and PET data, and input the CT data and PET data into a dual-modality medical image segmentation model to obtain a preliminary segmentation result.

[0060] The generation module 602 is used to generate a difference area based on the preliminary segmentation result, perform difference area detection on the difference area, and generate click signal data; the difference area is the area not marked as a lesion in the preliminary segmentation result.

[0061] The training module 603 is used to train the dual-modality medical image segmentation model for a preset number of times using the click signal data, CT data, and PET data to obtain a segmentation result.

[0062] The determination module 604 is configured to determine the segmentation result generated by the last training as the lesion image segmentation result.

[0063] For the specific definition of a method for lesion segmentation that fuses PET and CT dual-modality images, please refer to the definition of a method for lesion segmentation that fuses PET and CT dual-modality images above, which will not be repeated here. Each module in the above-mentioned method for lesion segmentation that fuses PET and CT dual-modality images can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0064] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A lesion segmentation method based on the fusion of PET and CT dual-modality images is provided.

[0065] The present invention also provides Figure 7 A schematic diagram of a computer device for a lesion segmentation method that integrates PET and CT dual-modality images is provided, such as Figure 7As shown in the figure, at the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 This is a flow chart of a lesion segmentation method for fusing PET and CT dual-modality images provided by the present invention.

[0066] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware using a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes in the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0067] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A lesion segmentation method integrating PET and CT dual-modality images, characterized in that: include: Acquiring lesion image data, separating the lesion image data to obtain CT data and PET data, and inputting the CT data and PET data into a dual-modality medical image segmentation model to obtain a preliminary segmentation result; generating a difference region based on the preliminary segmentation result, and performing a difference region detection on the difference region to generate click signal data; The difference area is the area not marked as a lesion in the preliminary segmentation result; Training the dual-modality medical image segmentation model a preset number of times using the click signal data, the CT data, and the PET data to obtain a segmentation result; The segmentation result generated by the last training is determined as the lesion image segmentation result.

2. The method according to claim 1, wherein The dual-modality medical image segmentation model is trained a preset number of times using the click signal data, the CT data, and the PET data to obtain a segmentation result, including: For each of the preset number of trainings, the click signal data is processed by the click encoding module to obtain click features, the CT data is processed by the CT input module to obtain CT features, and the PET data is processed by the PET input module to obtain PET features; In the PET downsampling module of each layer, the PET features and the click features are fused through the multi-scale click attention module to obtain the enhanced PET features of each layer; Processing the CT features through a CT downsampling module to obtain CT features of each layer; The enhanced PET features of each layer and the CT features of each layer are fused through the feature fusion module to obtain the output fusion features of the PET downsampling module of each layer; The output fusion features of the last layer are processed through the bottleneck layer and multi-layer upsampling is performed through the upsampling module to obtain the segmentation result; the upsampling module and the PET downsampling module are connected by jumps.

3. The method according to claim 2, wherein The method further comprises: For each of the preset number of trainings, after the training is completed, new click signal data is generated based on the segmentation results obtained after the training is completed; The generated new click signal data is used as the click signal data for the next round of training.

4. The method according to claim 3, wherein The multi-scale click attention module includes channel attention, spatial attention and click gating mechanism. The multi-scale click attention module is: ; in, is the feature map, is the click feature, is the batch size, which indicates the number of samples processed at a time, is the number of feature channels, is the channel attention, is a click gating mechanism, Expressed as element-wise multiplication, for spatial attention; The CT features are: ； in, Indicates the i layer CT Feature map, Indicates the i layer CT The feature extraction function of the path, for CT A set of network parameters for the path, For the i-1th layer CT Feature map; The PET characteristics are: ； in, Indicates the i layer PET Feature map, Represents the i-th layer PET The feature extraction function of the path, for PET A collection of network parameters for the route.

5. The method according to claim 4, wherein The enhanced PET features of each layer and the CT features of each layer are fused by the feature fusion module to obtain the output fusion features of the PET downsampling module of each layer, specifically including: The CT features of each layer and the enhanced PET features of each layer are normalized to obtain normalized CT features and normalized PET features; Attention enhancement is performed on the normalized CT features and the normalized PET features to obtain attention-enhanced CT features and attention-enhanced PET features; the attention-enhanced CT features are: ； in, Indicates that the attention is enhanced i layer CT feature, Represents the normalized i-th layer CT feature, Indicates the i layer CT Attention map of features, Represents element-wise multiplication; The PET characteristics after attention enhancement are: ； in, Indicates that the attention is enhanced i layer PET feature, Represents the normalized i layer PET feature, Indicates the i layer PET Attention map of features, Represents element-wise multiplication; The attention-enhanced CT features and the attention-enhanced PET features are fused to obtain the output fused features.

6. The method according to claim 5, wherein The output fusion features include shallow fusion features and deep fusion features; When the number of layers is less than the preset threshold, the output fusion feature is a shallow fusion feature, which is: ； in, for CT The initial weights of the features, for sigmoid activation function, It is the feature after shallow fusion; When the number of layers is greater than or equal to the preset threshold, the output fusion feature is deep feature fusion, and the deep fusion feature is: ； in, for PET Enhancement coefficient, is the residual coefficient, for CT Initial weights of features.

7. The method according to claim 6, wherein The adaptive method of the CT feature weight specifically includes: Initialize the CT feature weight to obtain the initialized CT feature weight; the initialized CT feature weight is: ； in, i is the number of layers; The initialized CT feature weight is dynamically adjusted; the formula corresponding to the dynamic adjustment is: ； in, is the overall loss function, is the learning rate, is the number of iterations, For the t The first iteration i The weight of the layer.

8. The method according to claim 1, wherein The difference region detection includes morphological operations and SUV feature weight analysis. The difference region detection is performed on the difference region to generate click signal data, including: Performing morphological operations on the difference region to obtain a boundary region of the difference region; After obtaining the boundary area of ​​the difference area, performing SUV feature weight analysis on the difference area to obtain a weighted difference area; Returning a target difference region according to the weighted difference region and the boundary region; Random selection is performed based on the target difference area to obtain the click signal data.

9. The method according to claim 8, wherein The morphological operation includes a dilation operation, an erosion operation, and a boundary region extraction. The morphological operation is performed on the difference region to obtain the boundary region of the difference region, specifically including: Perform a dilation operation on the difference region to obtain a dilation result; the formula corresponding to the dilation operation is: ； in, Indicates the difference area The expansion result, for The three-dimensional structural element, represents the expansion operation, Represented in three-dimensional structural elements Take the maximum value within; An erosion operation is performed on the difference region to obtain an erosion operation; the formula corresponding to the erosion operation is: ； in, Indicates the difference area The corrosion results, for The three-dimensional structural element, represents the corrosion operation, Represented in three-dimensional structural elements Take the minimum value within; The boundary area is calculated based on the corrosion result and the expansion result; the calculation formula of the boundary area is: ； in, Indicates the difference area The boundary area, represents the image after the dilation operation, Represents the image after the erosion operation.

10. The method according to claim 9, wherein The SUV feature weight analysis includes SUV feature weight calculation and weighted difference map calculation; performing SUV feature weight analysis on the difference area to obtain a weighted difference area specifically includes: The SUV feature weight calculation is performed on the difference area in the PET image to obtain the weight value of each position in the difference area; the formula corresponding to the SUV feature weight calculation is: ； in, Indicates location The weight value at Indicates location The SUV value at Indicates the average SUV value of the difference area, represents the SUV standard deviation of the difference area, is a small positive number; Based on the weight value, a weighted difference map is calculated for the difference area to obtain a weighted difference map, and the weighted difference map is determined as the weighted difference area; the formula corresponding to the weighted difference map calculation is: ； in, represents the weighted difference map, Indicates the difference area, represents the boundary area, Represents the weighting factor for the boundary area.

Images