Intelligent detection and quantitative evaluation method of multiple myeloma lesions based on pet-ct

Abstract

The application relates to the technical field of medical image processing, and discloses a multiple myeloma lesion intelligent detection and quantitative evaluation method based on PET-CT, which comprises the following steps: acquiring and preprocessing PET and CT three-dimensional images; PET functional features and CT anatomical features are extracted through a double-branch network, and the two features are deeply fused by using a cross-modal attention mechanism; a cascaded architecture is used for lesion identification; candidate regions are located by a three-dimensional lesion detection module, and then the candidate regions are finely segmented by a three-dimensional lesion segmentation module to generate a three-dimensional mask; the segmented mask is post-processed to remove false positives and complete false negative lesions; finally, quantitative parameters such as overall metabolic tumor volume are automatically calculated based on the final mask. The application realizes full-automatic and objective lesion identification and segmentation, and improves the sensitivity, specificity and quantitative evaluation consistency of detection.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a method for intelligent detection and quantitative assessment of multiple myeloma lesions based on PET-CT. Background Technology

[0002] Positron emission tomography (PET) combined with computed tomography (CT) is a key imaging technique for the diagnosis, staging, efficacy evaluation, and prognosis prediction of multiple myeloma, as it can simultaneously provide functional metabolic information and precise anatomical localization of lesions. In current clinical practice, the interpretation and analysis of PET / CT images primarily relies on manual or semi-automated operations by radiologists or nuclear medicine physicians.

[0003] In this process, doctors need to review massive amounts of 3D image data, including hundreds of tomographic scan sequences, one by one. This is an extremely demanding and time-consuming task, requiring a high level of energy and focus from the doctor. More importantly, the entire diagnostic process is highly dependent on the doctor's professional experience and subjective judgment. Especially when delineating lesion boundaries for quantitative analysis, there can be significant differences between different doctors, and even among the same doctor at different times, in defining lesions with blurred boundaries or diffuse lesions with irregular shapes. These inter-observer and intra-observer variability directly affects the accuracy and consistency of subsequent calculations of key quantitative indicators, such as total metabolic tumor volume (TMTV) and total lesion glycolysis (TLG) and other systemic tumor burden parameters, thus affecting the accurate assessment of the patient's condition.

[0004] Furthermore, the human brain has limited ability to precisely fuse the functional metabolic information reflected in PET images with the detailed anatomical information carried by CT images at the voxel level. Doctors often rely on experience to make macroscopic correspondences, making it difficult to achieve optimal complementarity between the two modalities at the microscopic level. This leads to inaccurate judgments of lesion extent. Simultaneously, the human body contains many physiological radioactive uptake areas, such as the brain, heart, and urinary system. Their high metabolic activity can be confused with malignant lesions on images, interfering with doctors' identification and easily leading to false positives. Conversely, there is a risk of missing some non-secreting or osteolytic lesions with low metabolic activity, i.e., false negatives.

[0005] Therefore, there is an urgent need for a method that can automatically, objectively, efficiently and accurately identify, segment and quantitatively assess multiple myeloma lesions. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides an intelligent detection and quantitative assessment method for multiple myeloma lesions based on PET-CT, which solves the problems of low efficiency, inconsistent results, and low accuracy in diagnosing multiple myeloma by manually reviewing PET / CT images.

[0007] To achieve the above objectives, the present invention provides a method for intelligent detection and quantitative assessment of multiple myeloma lesions based on PET-CT, comprising the following steps: S1. Acquire the PET image data and CT image data to be processed; In step S1, whole-body PET image data and corresponding CT image data of the same patient are obtained from the medical image archive and communication system or local storage device.

[0008] S2. Perform data preprocessing on the PET image data and CT image data; Step S2 aims to standardize and spatially align the raw image data, providing a data foundation with a uniform format and spatial consistency for subsequent processing steps.

[0009] S3. Perform multimodal fusion processing on the preprocessed PET image data and CT image data to generate fusion feature information; The S3 step aims to combine the functional metabolic information from PET image data with the anatomical structural information from CT image data to generate multidimensional information that can more comprehensively characterize lesion features.

[0010] S4. Input the fused feature information into a pre-trained lesion recognition deep learning model and output a lesion segmentation mask for the multiple myeloma. Step S4 uses a deep learning model to automatically identify and precisely define the spatial extent of the lesion, generating a 3D mask of the same size as the original image. In the mask, voxels with specific values ​​(e.g., a value of 1) represent the lesion area, while voxels with other values ​​(e.g., a value of 0) represent the background area.

[0011] S5. Based on the lesion segmentation mask, perform quantitative evaluation of the lesion region to obtain quantitative evaluation parameters.

[0012] Step S5 automatically calculates a series of clinically significant quantitative indicators based on the lesion region located by the lesion segmentation mask.

[0013] In one specific embodiment, the data preprocessing in step S2 includes: The PET image data were normalized using the Standardized Uptake Value (SUV) method. The CT image data is normalized to Henle unit values ​​and then registered with the PET image data.

[0014] Preferably, the data preprocessing further includes: after the image registration is completed, resampling the PET image data and the CT image data to the same voxel spacing to ensure that the images of the two modalities have completely corresponding voxel grids in three-dimensional space.

[0015] In a specific embodiment, the multimodal fusion processing in step S3 specifically includes: A dual-branch 3D convolutional network is used to extract the functional metabolic features of the PET image data and the anatomical structural features of the CT image data, respectively. The functional metabolic features and anatomical structural features are weighted and fused using an attention mechanism to generate the fused feature information. The attention mechanism adaptively assigns weights to different spatial locations and feature channels, giving greater weight to features that contribute more to lesion identification.

[0016] In a specific embodiment, the lesion recognition deep learning model in step S4 includes a three-dimensional lesion detection module and a three-dimensional lesion segmentation module connected in sequence. Accordingly, the specific process of inputting the fused feature information into the lesion recognition deep learning model includes: The fused feature information is input into the three-dimensional lesion detection module to identify and output a three-dimensional bounding box containing the lesion in the candidate region; The features of the candidate lesion region are cropped from the fused feature information and input into the three-dimensional lesion segmentation module to perform fine segmentation within the candidate region, so as to output the lesion segmentation mask.

[0017] Preferably, the three-dimensional lesion detection module is a detector based on a three-dimensional fast region convolutional neural network (3DFasterR-CNN) model or a three-dimensional YOLO model.

[0018] Preferably, the three-dimensional lesion segmentation module is a segmenter based on a three-dimensional U-shaped network (3DU-Net) model or a three-dimensional vision converter U-shaped network (3DTransUNet) model.

[0019] In one specific embodiment, the quantitative assessment parameters obtained in step S5 include: maximum standardized intake (MPI) ), average standardized intake ( The parameters are: metabolic tumor volume (MTV) and total lecithin glycolysis (TLG). Specifically, these parameters are calculated as follows: Maximum Standardized Intake This parameter reflects the uptake level in the most metabolically active areas of tumor tissue. It is calculated as follows: within the area identified by the lesion segmentation mask... Within this scope, retrieve the SUV values ​​of all voxels and take the maximum value. The formula is: ; in, The set of all lesion regions identified by the lesion segmentation mask. It is a voxel within that region. It is a voxel of value.

[0020] Average Standardized Intake This parameter represents the average metabolic level of the entire lesion area. It is calculated by dividing the lesion area... The SUV values ​​of all voxels within the region are summed, and then divided by the total number of voxels in that region. The formula is: ; in, The lesion area The total number of voxels within.

[0021] Metabolic Tumor Volume (MTV): This parameter quantifies the total volume of metabolically active tumors throughout a patient's body and is a key indicator for assessing tumor burden. It is calculated by statistically analyzing the lesion area. Total number of voxels And multiplied by the physical volume of a single voxel. The formula is: ; in, It is the volume of a single voxel determined according to the image resolution (e.g., 2mm x 2mm x 2mm = 8m). 3 ).

[0022] Total Lesion Glycolysis (TLG): This parameter is a comprehensive indicator that reflects both the tumor's volumetric burden and metabolic activity. It is calculated by taking the average standardized uptake value of the lesion region. Metabolic tumor volume Multiply. The formula is: ; In one specific embodiment, after obtaining the lesion segmentation mask, the method further includes a post-processing step. This step, in conjunction with preset anatomical constraint rules and metabolic thresholds, corrects the lesion segmentation mask to eliminate false positive lesions and complete false negative lesions.

[0023] Preferably, the anatomical constraint rules include: constructing or obtaining a predefined skeletal system mask based on the anatomical information provided by the CT image data; identifying areas outside the skeletal system mask that are identified as lesions by the model as false positive lesions; and removing them from the final lesion segmentation mask.

[0024] This invention provides a method for intelligent detection and quantitative assessment of multiple myeloma lesions based on PET-CT. It has the following beneficial effects: 1. This invention effectively combines the functional metabolic information of PET images with the anatomical structural information of CT images through multimodal fusion processing. In particular, by utilizing a dual-branch network and attention mechanism, the model can simultaneously leverage the advantages of both types of information for judgment. This effectively overcomes the shortcomings of relying solely on PET images, which struggle to distinguish between physiological uptake and lesions, or relying solely on CT images, which struggle to detect lesions without obvious bone destruction. It improves the ability to identify small and atypical lesions, thereby comprehensively enhancing the accuracy of detection.

[0025] 2. This invention employs an end-to-end deep learning model to automatically identify and segment lesions. The entire process is driven by standardized data, avoiding discrepancies in image interpretation results caused by differences in experience and condition among different physicians. This method ensures that for the same image data, it can always output repeatable and consistent detection and segmentation results, providing an objective and reliable basis for clinical diagnosis, efficacy evaluation, and longitudinal comparison.

[0026] 3. This invention utilizes a lesion segmentation mask with voxel-level precision output by a deep learning model, enabling fully automated calculation of key clinical indicators such as maximum standardized uptake, average standardized uptake, metabolic tumor volume, and total glycolysis. Compared to time-consuming manual or semi-automatic delineation methods whose accuracy depends on the operator, this invention significantly reduces the time required for quantitative analysis. Furthermore, the precise lesion boundary segmentation ensures the accuracy of quantitative parameter calculations, providing higher-quality data support for treatment planning and prognostic assessment. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the process of the present invention; Figure 2This is a schematic diagram of the multimodal fusion network structure of the present invention. Detailed Implementation

[0028] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] See attached document Figure 1 This invention provides a method for intelligent detection and quantitative assessment of multiple myeloma lesions based on PET-CT. This method is executed by a computer system and may include the following steps: S1. Acquire the PET and CT image data to be processed. This step reads the whole-body 3D PET image data and the corresponding 3D CT image data of the same patient from the medical image archive and communication system or local storage device.

[0030] S2. Data preprocessing of PET and CT image data. This step performs a series of standardization operations on the raw image data, including: normalizing the normalized acquisition values ​​of PET image data; normalizing the Henle unit values ​​of CT image data; performing image registration between PET and CT image data to align the spatial coordinate systems of the two modalities; and resampling the registered image data to the same voxel spacing to establish a voxel-level spatial correspondence.

[0031] S3. Perform multimodal fusion processing on the preprocessed PET and CT image data to generate fused feature information. This step uses a two-branch 3D convolutional network to extract functional metabolic features and anatomical structural features from the PET and CT image data, respectively. Then, an attention mechanism is used to weightedly combine the two features to generate fused feature information containing both modalities.

[0032] S4. Input the fused feature information into a pre-trained deep learning model for lesion recognition, and output a lesion segmentation mask for multiple myeloma. This step adopts a cascaded model structure. First, a 3D lesion detection module identifies and outputs candidate lesion regions from the fused feature information. Then, the features of the candidate regions are input into a 3D lesion segmentation module to perform voxel-level segmentation of the lesion boundaries, and finally outputs a 3D lesion segmentation mask corresponding to the original image space.

[0033] S5. Based on the lesion segmentation mask, quantitatively assess the lesion region to obtain quantitative assessment parameters. This step automatically calculates a series of quantitative assessment parameters based on the lesion region identified by the lesion segmentation mask output in the previous step, combined with SUV-normalized PET image data. These parameters include, but are not limited to, the maximum normalized uptake value (MNU). ), average standardized intake ( ), metabolic tumor volume ( ) and total glycolysis ( ).

[0034] The method of this invention can be implemented in a computer system, which may be a server, a high-performance computing workstation, or a dedicated medical device equipped with corresponding computing capabilities. In one specific embodiment, the hardware and software operating environment of the computer system used to execute the method of this invention is as follows.

[0035] The hardware environment of a computer system includes: one or more central processing units (CPUs) for executing operating system instructions, controlling data flow, and processing non-parallel computing tasks; one or more graphics processing units (GPUs) for performing large-scale parallel computing during the training and inference of deep learning models, especially high-density computing tasks such as 3D convolution and matrix multiplication; large-capacity main memory (RAM), such as 64GB, 128GB, or higher, for loading complete 3D PET and CT image volume data at runtime, storing network parameters of deep learning models, and intermediate feature maps passed between computing layers; and high-speed storage devices, such as solid-state drives (SSDs), for persistently storing raw DICOM image data, preprocessed data, weight files of trained lesion recognition deep learning models, and the finally generated lesion segmentation masks and quantitative evaluation parameters.

[0036] The software environment of a computer system includes: an operating system, such as a Linux distribution (e.g., Ubuntu Server 20.04) or Windows Server, which provides basic operational support and resource management for upper-level applications. On top of this operating system, a Python programming environment is deployed.

[0037] This Python programming environment includes and configures the following core libraries: a deep learning framework, such as PyTorch or TensorFlow, which provides the fundamental components needed to build, train, and deploy lesion recognition deep learning models, including tensor operation libraries, automatic differentiation engines, and predefined neural network layers (such as 3D convolutional layers, activation function layers, and pooling layers); one or more medical image processing libraries, such as SimpleITK and Pydicom, for parsing, reading, and writing DICOM format PET and CT image data, and performing image registration and resampling operations in the data preprocessing steps; and a numerical computation library, such as NumPy, for supporting efficient multidimensional array operations in the data preprocessing and post-processing stages. Step S1: Data Acquisition In this step, the method is executed via a computer system to acquire the patient's imaging data from one or more data sources. The data source can be a DICOM-compliant Picture Archiving and Communication System (PACS) or a local or network storage device containing DICOM data files.

[0038] The acquisition process specifically includes: for a given patient identifier (e.g., PatientID) and examination identifier (StudyID), locating a folder or dataset containing whole-body PET image data series and CT image data series. An image series consists of multiple independent, sequentially arranged two-dimensional DICOM (.dcm) files, each file representing a tomographic slice.

[0039] Subsequently, the system uses a dedicated DICOM parsing library to read each DICOM file in the series one by one. For each file, the system parses its header metadata to extract the information necessary to construct the 3D volume data. Specifically, the system extracts tags used to determine the 3D spatial location, such as ImagePosition(Patient), Tag0020,0032, and SliceThickness (Tag0018,0050).

[0040] Based on the Z-axis coordinates extracted from the image location (patient) labels, the system precisely sorts all two-dimensional image slices along the Z-axis (usually the patient's head-to-toe direction). After sorting, the system stacks these two-dimensional slice arrays in memory to construct PET three-dimensional image volume data and CT three-dimensional image volume data representing the patient's three-dimensional body space.

[0041] Each generated 3D image volume data is represented as a 3D numerical array with dimensions (depth, height, width), where each element corresponds to the original pixel value of a voxel. These two 3D arrays will serve as input for subsequent data preprocessing steps.

[0042] PET image preprocessing: The purpose of the PET image preprocessing step is to perform a series of purification and standardization processes on the raw PET 3D image volume data to eliminate physical artifacts, reduce noise interference, and convert it into standardized ingestion value (SUV) data with consistency and comparability, ultimately adapting it to the input requirements of deep learning models.

[0043] First, physical correction is performed. The raw PET image data undergoes a series of correction algorithms to compensate for physical interference during the data acquisition process. In one embodiment, this includes: Attenuation correction: Using the registered CT image data acquired in step S1, an attenuation correction map is generated. Based on this correction map, photons in PET data that are attenuated due to absorption by human tissues (especially bones) are corrected.

[0044] Scattering correction: Applying a model (e.g., simulation-based or energy window-based methods) to estimate and subtract scattering coincidences caused by Compton scattering events to correct image contrast.

[0045] Random coincidence correction: Subtracts random coincidence events caused by the simultaneous detection of two uncorrelated photons within the coincidence time window.

[0046] Next, denoising is performed on the physically corrected PET image data. Since PET imaging is based on a Poisson statistical process of radioactive decay, image noise levels are high, especially during low-dose scans. In one embodiment, a three-dimensional Gaussian filter is used to smooth the image volume data to suppress high-frequency statistical noise. In another embodiment, a three-dimensional nonlocal mean filter can be used to better preserve lesion edge details while effectively denoising.

[0047] Subsequently, standardized uptake values ​​(SUVs) are calculated. This process converts voxel values ​​(radioactivity concentrations) in the corrected and denoised PET image volume data into semi-quantitative indicators independent of patient weight and injection dose. This process requires extracting patient weight, radiopharmaceutical injection dose, injection time, drug half-life, and image acquisition start time from DICOM metadata. For each voxel... Its SUV value based on weight ( Calculated using the following formula: ; in, It is a voxel Standardized intake values; It is a voxel Corrected and noise-reduced radioactivity concentration at the location (unit: Bq / mL); This refers to the patient's weight (in grams). This is the total dose of radiopharmaceutical injected into the patient (unit: Bq); It is the decay constant of a radioactive nuclide, and its value is ,in It is the physical half-life of the nuclide; It is the time interval between the injection of radiopharmaceutical and the start of image acquisition.

[0048] Finally, to adapt to the input requirements of subsequent deep learning models, the calculated SUV 3D volume data is normalized. In one embodiment, this process includes clipping and linear scaling. A clinically reasonable upper limit for SUV values ​​(e.g., SUV=20) and a lower limit (e.g., SUV=0) are set. All voxel values ​​in the volume data above the upper limit are forced to be set to the upper limit, and voxel values ​​below the lower limit are set to the lower limit. The purpose of this operation is to suppress the influence of extreme physiological uptake (such as bladder) on the dynamic range of the data.

[0049] After truncation, the value range of the volume data is linearly scaled to the [0,1] interval, for example, using the formula: ; in, Represents the normalized value. Represents the value after clipping, where and These are the upper and lower limits that are set.

[0050] The normalized PET image data obtained by this formula will be used as one of the inputs for the subsequent multimodal fusion step.

[0051] CT image preprocessing: The purpose of CT image preprocessing is to standardize and correct artifacts in the raw CT 3D image volume data to eliminate data acquisition differences and highlight anatomical features associated with the pathology of multiple myeloma, such as bone destruction and soft tissue infiltration.

[0052] First, Hounsfield Unit (HU) value conversion is performed. This is done by reading the rescale slope (RescaleSlope, Tag0028, 1053) and intercept (RescaleIntercept, Tag0028, 1052) from the DICOM metadata, converting the raw pixel values ​​of each voxel to standard HU values. Value. The conversion formula is: ; Next, artifact correction is performed. The artifact correction step aims to reduce the interference of image artifacts on subsequent automatic recognition. In one embodiment, the artifact correction step includes: Metal Artifact Correction (MAR): For patients with orthopedic implants or metal dentures, MAR algorithms based on interpolation or iterative reconstruction are applied to identify and correct strip-shaped or star-shaped high-density artifacts caused by metal objects.

[0053] Motion artifact correction: For image blurring or misalignment in the chest and abdominal regions caused by respiratory motion, respiratory gating technology can be used during data acquisition, or motion correction algorithms based on image registration can be applied in the image post-processing stage to sharpen tissue boundaries.

[0054] Subsequently, the artifact-corrected The 3D volumetric data was standardized to grayscale. To highlight the skeletal structure and soft tissue lesions, a grayscale setting was implemented. The value range is truncated and normalized. For example, a valid range including relevant organizations is defined. Lower limit of value (e.g.) (corresponding to soft tissue) and an effective Upper limit of value (e.g.) (corresponding to cortical bone).

[0055] When applying this range, all data in the 3D volume data below this range will be included. The voxel value is forcibly set to All higher The voxel value is forcibly set to This operation preserves the contrast between the bone destruction area and the soft tissue lesion while compressing the dynamic range of irrelevant information such as air (approximately -1000 HU) and dense metal (>1500 HU). After the truncation operation is completed, the window... value The normalized interval [0,1] is linearly scaled to serve as the input to subsequent deep learning models. This linear scaling can be achieved using the formula: ); in, Represents the normalized value. This represents the value after clipping. Indicates the lower limit of the window width. This indicates the upper limit of the window width.

[0056] This is done using this formula. The resulting normalized CT image data will then serve as one of the inputs for subsequent steps.

[0057] Multimodal image registration and resampling: The purpose of the multimodal image registration and resampling steps is to eliminate spatial positional deviations between preprocessed PET image data and preprocessed CT image data, and to unify their voxel resolution to achieve accurate voxel-level alignment, thus preparing for subsequent multimodal fusion.

[0058] First, multimodal image registration is performed. Since PET and CT scans are typically acquired at different times or on different devices, the patient's position may change slightly, causing the image data from the two modalities to not overlap in the spatial coordinate system. This embodiment uses a rigid registration algorithm to correct this deviation. Rigid registration only involves three-dimensional translation and three-dimensional rotation transformations, ensuring that the morphology of the patient's anatomical structures does not change during the transformation process.

[0059] In one embodiment, the registration process is as follows: CT image volume data with high-resolution anatomical features is set as the stationary image, and PET image volume data is set as the moving image. The system searches for an optimal rigid transformation matrix using an iterative optimization algorithm. The goal of this optimization algorithm is to maximize a similarity metric function that measures the degree of alignment between multimodal images. In this embodiment, mutual information (MI) is used as the similarity metric function because it is robust to the complex, nonlinear intensity relationship between the two images (functional metabolism and anatomical structure).

[0060] In another embodiment, the registration process can be performed based on anatomical landmarks. The operator or algorithm first identifies a set of corresponding anatomical landmarks on both stationary (CT) and moving (PET) images, such as easily identifiable bony structures like specific vertebrae in the spine, the acetabulum in the pelvis, or the sacrum. The system then calculates a rigid transformation matrix that minimizes the sum of squared spatial distances between the two sets of landmarks.

[0061] After registration, the system applies the calculated optimal rigid transformation matrix to the moving image (PET) to generate a transformed PET image volume data that is aligned with the fixed image (CT) in the spatial coordinate system.

[0062] Next, a resampling operation is performed. At this point, although the CT image and the transformed PET image are in the same spatial coordinate system, their voxel spacing (i.e., image resolution) and dimensions are usually still different. In order for the two types of data to be fused into voxels in the subsequent neural network, they need to be resampled onto the same three-dimensional grid.

[0063] This step sets a uniform target voxel spacing, for example, an isotropic 2mm x 2mm x 2mm. Then, the system resamples both the original CT image volume data and the transformed PET image volume data onto this new, uniform 3D mesh using an interpolation algorithm. In one specific embodiment, a trilinear interpolation algorithm is used for both the CT image (anatomical structure) and the PET image (functional information) to maintain image smoothness when calculating voxel values ​​at the new mesh points.

[0064] After this step, the output PET image data and CT image data have exactly the same dimensions (Depth, Height, Width) and voxel spacing, and are anatomically precisely aligned, and can be directly used for subsequent fusion processing.

[0065] Two-branch three-dimensional network structure: See attached document Figure 2 In a specific embodiment, the multimodal fusion processing step S3 of the present invention is implemented through a deep learning model with a dual-branch three-dimensional network structure.

[0066] The dual-branch 3D network structure comprises two parallel encoder branches for independent feature extraction: a PET feature extraction branch and a CT feature extraction branch. Preprocessed and aligned PET 3D image data is input to the PET feature extraction branch; preprocessed and aligned CT 3D image data is input to the CT feature extraction branch.

[0067] The technical purpose of setting up this dual-branch structure is to allow the model to learn specific feature representations for the different characteristics of the two modalities (functional metabolic information from PET and anatomical information from CT), preserving the unique information of each modality before fusion, and avoiding information confusion or loss caused by premature fusion.

[0068] In one embodiment, both the PET feature extraction branch and the CT feature extraction branch consist of multiple stacked 3D convolutional modules. Each 3D convolutional module specifically includes: a 3D convolutional layer (e.g., using a 3x3x3 convolutional kernel) for extracting local features in 3D space; a normalization layer (e.g., 3D batch normalization, 3DBatchNormalization) for accelerating model training and stabilizing feature distribution; and a non-linear activation function (e.g., the ReLU function) for introducing non-linear expressive power.

[0069] In one embodiment, the PET feature extraction branch and the CT feature extraction branch have the same number of network layers and parameter configuration, forming a symmetrical structure. In another preferred embodiment, the two branches employ an asymmetrical structure; for example, the CT feature extraction branch has more convolutional layers or a deeper network depth than the PET feature extraction branch, in order to capture more finely the complex skeletal anatomical boundaries and texture details in CT images.

[0070] Feature extraction process: In the dual-branch 3D network structure, PET image data and CT image data are processed in their respective feature extraction branches through a hierarchical, coarse-to-fine process to generate multi-scale feature representations.

[0071] Specifically, in each feature extraction branch, the input 3D image data sequentially passes through multiple stacked 3D convolutional modules. Within each module, the 3D convolutional layer performs convolution operations on the input feature map (for the first layer, the input is the image data) to extract local features of specific patterns. As data is passed layer by layer in the network, subsequent convolutional layers operate on the feature map output by the previous layer, thereby learning more abstract and complex combined features. For example, shallow networks mainly extract low-level features such as edges, corners, or textures, while deep networks can integrate these low-level features to form high-level semantic features corresponding to specific anatomical structures or metabolic patterns.

[0072] To expand the receptive field of subsequent convolutional layers and reduce computation, a 3D downsampling layer is placed after each or every few 3D convolutional modules. In one embodiment, this downsampling layer is a 3D Max Pooling layer. For example, a 2x2x2 3D Max Pooling layer with a stride of 2 replaces each 2x2x2 voxel block of the input feature map with the maximum value within that block.

[0073] This operation reduces the feature map to half its original size in all three spatial dimensions: depth, height, and width, thus lowering the spatial resolution. Simultaneously, the number of channels in the output feature map is typically increased after downsampling (i.e., using more convolutional kernels), thereby increasing the dimensionality of the features while reducing spatial resolution, allowing for the encoding of richer, higher-level semantic information.

[0074] After the alternating convolution and downsampling operations described above, the PET feature extraction branch ultimately outputs a set of functional metabolic feature maps representing different levels of abstraction; similarly, the CT feature extraction branch also outputs a set of anatomical structure feature maps representing different levels of abstraction. These multi-scale feature maps extracted from the two branches will serve as input to the next attention fusion mechanism.

[0075] Attention fusion mechanism: The purpose of this step is to effectively combine the PET functional metabolic feature maps and CT anatomical feature maps extracted from two independent branches to generate a fused feature information rich in complementary information from the two modalities. In a specific embodiment, this fusion is achieved through a cross-modal attention module.

[0076] The cross-modal attention module performs a bidirectional guidance process: it guides the CT feature map using the PET feature map, and simultaneously guides the PET feature map using the CT feature map.

[0077] In a specific implementation, the bootstrapping process is as follows: First, the PET feature map output by the PET feature extraction branch (denoted as...) is... The input is fed into a 3D convolutional layer (e.g., a kernel size of 1x1x1) and subsequently a sigmoid activation function. This operation generates a PET spatial attention map (denoted as...). The three-dimensional spatial dimension of this figure is... The values ​​are the same, and the value of each voxel in the figure is in the range [0,1]. The values ​​in the table reflect the significance of functional metabolism at the corresponding spatial location.

[0078] Subsequently, this PET spatial attention map The CT feature map output by the CT feature extraction branch (denoted as...) Element-wise multiplication is performed. The result is a PET-guided CT feature map. The formula is: ; in, Element-wise multiplication operation.

[0079] Here In this study, CT anatomical features corresponding to spatial locations showing high metabolic activity on PET images were preserved or enhanced, while CT features of metabolically inactive areas were suppressed.

[0080] Finally, the two feature maps that have undergone bidirectional guidance will be... and Merge the data to generate 3D fused feature information. In one embodiment, the merging operation is a feature concatenation along the feature channel dimension. In another embodiment, the merging operation is an element-wise addition.

[0081] this The fused feature information includes both the location information of the hypermetabolic lesions and the precise anatomical boundary information of the location. This information will be used as the input to the lesion identification and segmentation model in step S4.

[0082] Cascaded detection and segmentation architecture: The goal of step S4 is to accurately identify and segment the three-dimensional spatial extent of all multiple myeloma lesions based on the fused feature information generated in step S3. In a specific embodiment, this goal is achieved through a cascaded deep learning model architecture. This architecture includes a lesion detection module and a lesion segmentation module.

[0083] The technical purpose of this cascaded architecture is to separate the coarse global search task from the fine segmentation task of local regions, in order to solve the technical problem of small lesion volume and scattered distribution in whole-body scan images. First, the lesion detection module quickly locates all potential lesion locations in complete, possibly downsampled, 3D fusion feature information. Then, the lesion segmentation module performs high-resolution, precise boundary segmentation only within these located candidate regions.

[0084] The lesion detection module is a three-dimensional convolutional neural network, whose input is the three-dimensional fused feature information output from step S3. The network's task is to analyze the entire three-dimensional space and output a set of candidate lesion regions. In one embodiment, these candidate regions are represented as the coordinates of a series of 3D bounding boxes and their corresponding confidence scores. The detection module is structured to have a high recall rate to ensure that no potential lesions are missed.

[0085] The lesion segmentation module is also a 3D convolutional neural network, such as a 3D U-Net network with an encoder-decoder structure. The input to this module is not complete whole-body image data, but rather unsampled 3D fusion feature information output from step S3, based on each high-confidence bounding box output by the lesion detection module. The 3D sub-volume is cropped from the input. For each input sub-volume, the lesion segmentation module performs voxel-level binary classification to determine whether each voxel in the region belongs to the lesion or the background.

[0086] The output of this module is a 3D lesion segmentation mask for each input sub-region. Finally, the system remaps and combines all the segmentation masks generated by the lesion segmentation module in their respective local coordinate systems into a blank 3D matrix of the same size as the original input image, based on the position information of their original bounding boxes, thereby generating a final 3D lesion segmentation mask covering the entire body.

[0087] Implementation of the 3D lesion detection module: The lesion detection module's function is to process the fused feature information representing the whole body area output by step S3. The analysis quickly outputs the coordinates and confidence scores of the three-dimensional bounding boxes (3DBoundingBoxes) for all potential lesion areas.

[0088] In one specific embodiment, the 3D lesion detection module is implemented based on a one-stage target detection framework adapted to 3D data, such as a 3D RetinaNet or YOLOv3 architecture. This implementation includes a backbone network for extracting multi-scale features, a Feature Pyramid Network (FPN), and parallel classification and regression sub-networks.

[0089] First, the fusion feature information generated in step S3 The data is fed into the backbone network (which can be a lightweight 3D ResNet structure). This network generates multiple feature maps with different spatial resolutions through a series of 3D convolutions and 3D pooling operations. , , , ,in Representative ratio Feature maps with lower spatial resolution but richer semantic information.

[0090] Next, to address the issue of large variations in lesion size in multiple myeloma, a three-dimensional Feature Pyramid Network (FPN) was used to integrate these multi-scale features. The FPN integrates high-level semantic features (such as...) via a top-down path. ) Perform 3D upsampling and compare it with shallow features of higher resolution from the bottom-up path (such as The fusion is achieved through lateral connections (e.g., a 1x1x1 convolution). This process is repeated layer by layer, ultimately generating a fusion that is effective at all scales (e.g., ...). , , , All of them are multi-scale feature pyramids rich in strong semantic information.

[0091] Subsequently, two independent, weight-shared 3D convolutional subnetworks (i.e., detection heads) are attached to each level of the feature pyramid. )superior: Classification subnetwork: This network is based on the preset... The subnetwork operates on multiple 3D anchor boxes (3DAnchorBoxes) at each spatial location within each layer. Anchor boxes are preset candidate boxes with different dimensions and aspect ratios. The subnetwork outputs a confidence score for each anchor box indicating that it contains a lesion.

[0092] Regression Subnetwork: This network operates on each anchor box in parallel, predicting the 3D spatial offset (e.g., center point coordinate offset) between the anchor box and the actual lesion boundary box. , , and size offset , , ).

[0093] During model inference, the detection module generates a massive number of raw predicted bounding boxes across all pyramid levels. The system first filters out most low-confidence background boxes based on a preset confidence threshold (e.g., 0.5). Then, it applies 3D Non-Maximum Suppression (NMS) to the remaining candidate boxes. This algorithm filters based on the 3D intersection-union ratio (IUU) between candidate boxes, merging multiple highly overlapping predicted boxes into one, ultimately outputting a set of high-confidence, non-overlapping 3D bounding boxes for lesion candidate regions. These bounding boxes serve as input to the lesion segmentation module.

[0094] The function of the lesion segmentation module is to perform fine, voxel-level three-dimensional boundary segmentation on each detected high-confidence lesion candidate region.

[0095] In one specific embodiment, the module is implemented based on a 3D U-Net network with an encoder-decoder structure. This network is specifically designed for segmentation tasks of 3D medical images, effectively combining contextual information and high-resolution local details.

[0096] For each 3D bounding box output by the detection module, the system first uses the bounding box's coordinates to extract the unsampled 3D fused feature information output from step S3. In this process, a corresponding 3D sub-volume is cropped out as the input to the segmentation module.

[0097] The specific structure of this three-dimensional U-Net network includes: Encoder Path: This path captures contextual features of the input sub-region. It consists of multiple consecutive encoder modules. Each encoder module contains two consecutive 3D convolutional layers (e.g., 3x3x3 convolutional kernels followed by a ReLU activation function) and a 3D max-pooling layer for downsampling (e.g., 2x2x2 pooling with a stride of 2). As the data passes through each encoder module, its spatial resolution is halved, while the number of feature channels doubles. Through this path, the network learns multi-level abstract features from local to global perspectives.

[0098] Decoder Path: This path is used to progressively restore the spatial resolution of the feature maps and achieve accurate lesion localization. It consists of multiple decoder modules symmetrical to the encoder path. Each decoder module first doubles the size of the low-resolution feature map from the previous layer through a 3D upsampling layer (e.g., a 3D transposed convolution). Subsequently, this enlarged feature map is concatenated with a corresponding feature map of the same resolution from the encoder path through a skip connection. The concatenated feature map is then processed through two consecutive 3D convolutional layers.

[0099] The skip connection is the key to this structure. It directly passes the high-resolution, shallow features (rich in details such as edges and textures) preserved in the encoder path to the decoder path, and combines them with the deep semantic information contained in the decoder path, thereby greatly improving the accuracy of segmentation boundaries.

[0100] Output layer: In the last layer of the decoder path, its output feature map is passed through a 1x1x1 3D convolutional layer, mapping the number of feature channels to 1 (corresponding to the lesion category). Finally, a sigmoid activation function is applied to generate a probability value in the range [0,1] for each voxel in the input sub-region, which represents the probability that the voxel belongs to the lesion.

[0101] By applying a preset threshold (e.g., 0.5) to the probability map, a binary 3D segmentation mask for the sub-region can be generated. All processed local masks are then recombined to form a complete 3D lesion segmentation result for the whole body.

[0102] Step S5: Quantitative Assessment The purpose of this step is to automatically extract and calculate a series of standardized quantitative biological indicators used clinically for the assessment of multiple myeloma, based on the final whole-body 3D lesion segmentation mask generated in step S4. This step transforms the model's pixel-level segmentation results into objective, quantifiable data with direct diagnostic value for physicians.

[0103] In one specific embodiment, the final segmentation mask output by step S4 is a single three-dimensional binary matrix that identifies all lesion regions within the patient's body (denoted as ). The system is based on this global lesion area. Using the standardized PET image data (i.e., the 3D matrix of SUV values) preprocessed in step S2, the following quantitative evaluation parameters are directly calculated: Maximum Standardized Intake This parameter reflects the uptake level in the most metabolically active areas of tumor tissue. It is calculated as follows: within the area identified by the lesion segmentation mask... Within, retrieve the SUV values ​​of all voxels and take the maximum value.

[0104] ; in, The set of all lesion regions identified by the lesion segmentation mask. It is a voxel within that region. It is a voxel of value.

[0105] Average Standardized Intake This parameter represents the average metabolic level of the entire lesion area. It is calculated by dividing the lesion area... The SUV values ​​of all voxels within the region are summed, and then divided by the total number of voxels in that region. .

[0106] ; in, The lesion area The total number of voxels within.

[0107] Metabolic Tumor Volume (MTV): This parameter quantifies the total volume of metabolically active tumors throughout a patient's body and is a key indicator for assessing tumor burden. It is calculated by statistically analyzing the lesion area. Total number of voxels And multiplied by the physical volume of a single voxel. .

[0108] ; in, It is the volume of a single voxel determined according to the image resolution (e.g., 2mm x 2mm x 2mm = 8m). 3 ).

[0109] Total Lesion Glycolysis (TLG): This parameter is a comprehensive indicator that reflects both the tumor's volumetric burden and metabolic activity. It is calculated by taking the average standardized uptake value of the lesion region. Metabolic tumor volume Multiply.

[0110] ; In a preferred embodiment, in addition to calculating the aforementioned global parameters, the system can also use a three-dimensional connected component labeling algorithm to identify the global lesion region. The analysis is performed to identify and count the number of individual lesions, and optionally, the volume of each individual lesion can be calculated separately. These parameters provide more detailed information on lesion distribution and characteristics.

[0111] After completing this step, the system outputs a structured quantitative assessment report, which includes the above... , , and These core indicators provide comprehensive data support for clinical decision-making.

[0112] False positive elimination: The purpose of this step is to review and filter the lesion set output from step S4 and preliminarily quantified in step S5 to identify and remove false-positive segmentation results caused by physiological uptake, image artifacts, or benign lesions. This step is crucial for improving the clinical usability and reliability of the entire system. In one specific embodiment, this false-positive removal process is implemented through a hybrid strategy combining anatomical prior knowledge and a feature classifier.

[0113] First, filtering based on anatomical atlases is performed. The system is pre-loaded with a standardized three-dimensional digital atlas of human anatomy, which marks three-dimensional regions of organs and tissues that would normally exhibit high radioactive uptake, such as the brain, heart, liver, kidneys, and bladder. By performing deformable image registration between the patient's CT images and this standard atlas, a mapping relationship is established between the patient's specific space and the standard space of the atlas.

[0114] Based on this mapping relationship, the system transforms the predefined physiological uptake organ mask in the atlas into the patient's image space. Subsequently, the system iterates through each individual lesion identified in step S5. If the geometric center of a lesion falls within any of the transformed organ masks, or if the overlap between its volume and the mask exceeds a preset threshold (e.g., 80%), the lesion is preliminarily identified as a false positive due to physiological uptake and removed from the final result. This method effectively eliminates most misjudgments stemming from normal organ metabolism.

[0115] Next, for the remaining candidate lesions that were not filtered out by the anatomical atlas, a fine-tuning process based on a machine learning classifier is performed. This step aims to distinguish true malignant lesions from some benign lesions or inflammations that are atypical in morphology or uptake.

[0116] The specific implementation is as follows: For each candidate lesion, the system extracts a set of multi-dimensional feature vectors from the PET and CT image data. These feature vectors include: Morphological features: calculated from the three-dimensional segmentation mask, such as the volume, surface area, sphericity, and elongation of the lesion; Intensity and statistical characteristics: calculated from PET images, for example , The standard deviation and entropy of SUV values; and the values ​​calculated from CT images. Mean and standard deviation.

[0117] Texture features: Gray-level co-occurrence matrix (GLCM) or gray-level run-length matrix (GLRLM) features, such as contrast, correlation, energy, and homogeneity, are calculated from the lesion regions of PET and CT images to quantify the heterogeneity within the lesions.

[0118] These extracted feature vectors are fed into a pre-trained classification model, such as a support vector machine, random forest, or gradient boosting machine. The classifier is trained by learning from a large number of true positive (real lesions) and false positive (such as degenerative osteoarthritis, inflammation) samples pre-labeled by radiologists.

[0119] During the inference phase, the classifier outputs a confidence score for each input candidate lesion, indicating that it is a true positive. The system sets a confidence threshold and retains only candidate lesions with scores higher than this threshold as the final, validated lesion detection results.

[0120] After this step is completed, the output will be a highly purified 3D lesion segmentation mask with significantly improved specificity, which will serve as the basis for generating the final clinical report.

[0121] False negative completion: The purpose of this step is to identify and recall true lesions (i.e., false negatives) that may have been missed in the main detection process of step S4, thereby improving the overall sensitivity of the system. False negatives may arise from atypical lesion presentation (e.g., extremely low metabolic activity) or lesions visible only on a single modality (e.g., only showing osteolytic destruction on CT without PET uptake). This step completes the process through a multi-strategy candidate region review workflow.

[0122] In one specific embodiment, the completion process includes: First, the system re-examines the low-confidence candidate regions discarded by the lesion detection module. In the main process of S4, to ensure high specificity, a relatively high confidence threshold (e.g., 0.5) is usually set to filter detection boxes. In this step, the system will re-recall those bounding boxes with confidence levels between a lower threshold (e.g., 0.1) and a higher threshold (0.5), and use them as the first candidate set to be completed.

[0123] Secondly, to specifically target osteolytic lesions that are negative on PET but positive on CT, this step performs a parallel, CT-data-based auxiliary detection. This auxiliary detection first uses a HU value thresholding method (e.g., setting...). A full-body skeletal 3D mask is generated (values ​​greater than 200). Then, within the area defined by this skeletal mask, an anomaly detection algorithm (e.g., an algorithm based on morphological operations or local density contrast) is applied to search for anomalies with abnormally low density (e.g., values ​​greater than 200). Suspicious areas (with values ​​between 0 and 100, significantly lower than the surrounding normal bone) were identified. These identified areas were marked as the second candidate set for completion.

[0124] Next, the system merges the first and second candidate regions to be completed. To prevent the introduction of too many false positives, these merged candidate regions are not directly adopted, but need to undergo a fine-grained verification.

[0125] In a preferred embodiment, the verification process reuses a trained feature classifier (e.g., random forest or SVM) for false positive rejection. The system extracts the same multi-dimensional feature vector (morphology, intensity, texture, etc.) from each candidate region to be completed and inputs it into the classifier. Only when the classifier classifies a candidate region as a true positive is the region confirmed as a successfully recalled false negative lesion.

[0126] For each recalled lesion, the system invokes the 3D lesion segmentation module to segment its corresponding 3D sub-region (based on 3D fusion feature information). (Crop) Perform precise voxel-level segmentation to generate its 3D mask.

[0127] Finally, the segmentation masks of all the completed false-negative lesions are merged into the processed (i.e., the master segmentation mask after removing false positives) through a voxel-level OR logical operation. This operation generates the final three-dimensional segmentation result of the whole body lesions, which has both high sensitivity and high specificity.

Claims

1. A method for intelligent detection and quantitative assessment of multiple myeloma lesions based on PET-CT, characterized in that, include: S1. Acquire the PET image data and CT image data to be processed; S2. Perform data preprocessing on the PET image data and CT image data; S3. Perform multimodal fusion processing on the preprocessed PET image data and CT image data to generate fusion feature information; S4. Input the fused feature information into the lesion recognition deep learning model and output a lesion segmentation mask for multiple myeloma; the lesion recognition deep learning model adopts a cascaded architecture, first outputs the bounding box of the lesion candidate region by localization, and then performs local fine segmentation in the candidate region to obtain the lesion segmentation mask. S5. Based on the lesion segmentation mask, perform quantitative evaluation of the lesion region to obtain quantitative evaluation parameters.

2. The intelligent detection and quantitative assessment method for multiple myeloma lesions based on PET-CT according to claim 1, characterized in that, In step S2, data preprocessing specifically includes: The PET image data is subjected to standardized capture value normalization processing; The CT image data is normalized to Henle unit values ​​and then registered with the PET image data.

3. The intelligent detection and quantitative assessment method for multiple myeloma lesions based on PET-CT according to claim 1, characterized in that, In step S3, the multimodal fusion processing specifically includes: A dual-branch 3D convolutional network is used to extract the functional metabolic features of the PET image data and the anatomical structural features of the CT image data, respectively. The functional metabolic features and the anatomical structural features are weighted and fused using an attention mechanism to generate the fused feature information.

4. The intelligent detection and quantitative assessment method for multiple myeloma lesions based on PET-CT according to claim 1, characterized in that, In step S4, the lesion recognition deep learning model includes a three-dimensional lesion detection module and a three-dimensional lesion segmentation module connected in sequence. Accordingly, inputting the fused feature information into the lesion recognition deep learning model specifically includes: The fused feature information is input into the three-dimensional lesion detection module to identify and output candidate lesion regions; The features of the candidate lesion region are input into the three-dimensional lesion segmentation module, and the lesion segmentation mask is output.

5. The intelligent detection and quantitative assessment method for multiple myeloma lesions based on PET-CT according to claim 4, characterized in that, The three-dimensional lesion detection module is a detector based on a three-dimensional fast region convolutional neural network model or a three-dimensional YOLO model.

6. The intelligent detection and quantitative assessment method for multiple myeloma lesions based on PET-CT according to claim 4, characterized in that, The three-dimensional lesion segmentation module is a segmenter based on a three-dimensional U-shaped network model or a three-dimensional visual converter U-shaped network model.

7. The intelligent detection and quantitative assessment method for multiple myeloma lesions based on PET-CT according to claim 1, characterized in that, In step S5, the quantitative assessment parameters include: maximum standardized uptake, average standardized uptake, metabolic tumor volume, and total glycolysis.

8. The intelligent detection and quantitative assessment method for multiple myeloma lesions based on PET-CT according to claim 1, characterized in that, After obtaining the lesion segmentation mask, a post-processing step is also included: By combining preset anatomical constraints and metabolic thresholds, the lesion segmentation mask is corrected to eliminate false positive lesions and complete false negative lesions.

9. The intelligent detection and quantitative assessment method for multiple myeloma lesions based on PET-CT according to claim 2, characterized in that, The data preprocessing also includes: After the image registration is completed, the PET image data and the CT image data are resampled to the same voxel spacing.

10. The intelligent detection and quantitative assessment method for multiple myeloma lesions based on PET-CT according to claim 8, characterized in that, The anatomical constraint rules include: Based on the anatomical information provided by the CT image data, lesions located outside the predefined skeletal system mask are identified as false positive lesions.

Images