A lymphoma identification and chemotherapy prognosis prediction system

Abstract

This invention discloses a lymphoma identification and chemotherapy prognosis prediction system. The data acquisition module acquires clinical feature data, biochemical feature data, PET image data, and CT image data; the preprocessing module preprocesses the data to obtain basic features and standard image data; the lesion identification module identifies lesions to obtain a lesion image set; the prognosis prediction module includes a morphological feature extraction unit, a metabolic feature extraction unit, a semantic feature extraction unit, a fusion unit that integrates multimodal features to obtain fused features, and a prediction unit that predicts the chemotherapy prognosis of each lesion based on the fused features. This invention integrates clinical feature data, biochemical feature data, lesion morphological features, metabolic features, and deep semantic features to accurately identify lymphoma lesions before chemotherapy and to achieve differentiated and individualized chemotherapy prognosis probability prediction for each independent lesion. It overcomes the advantages of existing technologies, such as delayed assessment, reliance on manual intervention, coarse prediction granularity, and one-sided feature utilization.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and artificial intelligence technology, specifically to a lymphoma identification and chemotherapy prognosis prediction system. Background Technology

[0002] Lymphoma, a malignant tumor originating from the lymphatic system, requires early diagnosis and accurate efficacy assessment for clinical treatment decisions. Positron emission tomography (PET) technology, especially PET / CT imaging fused with computed tomography (CT), can simultaneously provide information on the metabolic function and anatomical structure of lesions, and has become a key tool for the diagnosis, staging, and efficacy assessment of lymphoma.

[0003] Currently, the clinical assessment of the efficacy of chemotherapy for lymphoma mainly relies on follow-up PET / CT images after chemotherapy, which are then manually compared with baseline images before chemotherapy using methods such as the Deauville score. This method has a significant time lag, failing to provide prognostic information to physicians before or early in treatment, potentially missing the window of opportunity for optimizing treatment. Furthermore, this process heavily depends on the subjective experience of radiologists for visual assessment and scoring, which is not only time-consuming and labor-intensive but also prone to introducing inter-observer variability, resulting in poor consistency and reproducibility of assessment results.

[0004] To improve the objectivity and automation of assessments, existing technologies have proposed several image-based analysis methods. One type of method uses image segmentation algorithms (such as thresholding and region growing) to delineate tumor regions, then extracts single or a few parameters such as maximum standard uptake (SUVmax), mean standard uptake (SUVmean), or metabolic tumor volume (MTV), and performs statistical analysis with clinical outcomes. However, the accuracy of this type of method heavily depends on the quality of the segmentation results, while automated segmentation algorithms often lack sufficient precision, and high-precision segmentation usually requires a significant amount of time for manual correction and annotation, resulting in low automation and inefficiency. Another type of method attempts to extract a large number of radiomics features from the entire PET / CT image and construct machine learning models to predict the patient's overall treatment response.

[0005] However, existing technical solutions still have the following prominent defects and limitations:

[0006] Lack of lesion-level predictive capability: Most existing methods focus on binary prediction of overall patient "response" or "non-response," or perform mixed analysis of imaging features of all lesions. This completely ignores the crucial clinical fact that lymphoma is often a multifocal disease, and that different lesions in the same patient may respond heterogeneously to the same chemotherapy regimen. The inability to achieve lesion-level, differentiated prognostic prediction greatly limits its application value in guiding personalized, precision treatment (such as local intensification therapy for low-response lesions).

[0007] Feature utilization is one-sided and reliant on manual intervention: Traditional methods often rely on limited quantitative features (such as maximum SUV value and volume) extracted manually or semi-automatically after segmentation, failing to fully and automatically mine and integrate multi-dimensional information. For example, they fail to effectively combine the deep semantic features of lesions, fine morphological features (such as edge tightness), and patient blood biochemical indicators and clinical signs that are closely related to prognosis, resulting in insufficient model representation capabilities and a ceiling on prediction accuracy. Summary of the Invention

[0008] The technical problem to be solved by this invention is to provide a system that can integrate patient clinical and biochemical data, as well as morphological, metabolic, and deep semantic features of lesions, to automatically and accurately identify lymphoma lesions before chemotherapy, and to achieve differentiated and individualized prognostic probability prediction for each independent lesion, so as to overcome the problems of lagging assessment, reliance on manual intervention, coarse prediction granularity, and one-sided use of features in existing lymphoma identification and chemotherapy prognostic prediction systems.

[0009] To address the aforementioned technical problems, the present invention provides a lymphoma identification and chemotherapy prognosis prediction system, comprising:

[0010] The data acquisition module is used to acquire patient clinical characteristic data, biochemical characteristic data, lymphoma PET image data, and corresponding CT image data;

[0011] The preprocessing module is used to preprocess the patient's clinical and biochemical characteristic data to obtain basic features, and to preprocess the lymphoma PET image data based on the CT images to obtain standard image data.

[0012] The lesion identification module adopts the YOLO architecture and is used to identify lesions based on the standard image data to obtain a lesion image set.

[0013] The prognostic prediction module includes at least:

[0014] The morphological feature extraction unit is used to collect the lesion images from each lesion image, match them in the standard image data, obtain the corresponding volume data and edge compactness data, and encode the volume data and edge compactness data to obtain morphological features.

[0015] The metabolic feature extraction unit is used to obtain the corresponding metabolic data based on the matching results of each lesion image in the standard image data, and to encode the metabolic data to obtain metabolic features.

[0016] The semantic feature extraction unit is used to extract deep semantic features from each lesion region through a convolutional neural network model;

[0017] The fusion unit is used to fuse the basic features, metabolic features, morphological features and deep semantic features to obtain fused features;

[0018] The prediction unit is used to predict the prognosis of chemotherapy for each lesion based on the fusion features, and obtain the prediction results.

[0019] In a preferred embodiment, the method further includes a result generation and visualization module, which is used to associate the predicted prognosis of each lesion with the location of the corresponding lesion in the image data and display it visually, while generating a lesion-level prediction report.

[0020] In a preferred embodiment, the preprocessing of the lymphoma PET image data based on the CT images to obtain standard image data specifically includes the following steps:

[0021] The lymphoma PET image data were normalized using standard uptake.

[0022] Using the spatial coordinate system of the CT image as a reference system, the normalized lymphoma PET image data is rigidly registered with the CT image to obtain standard image data.

[0023] In a preferred embodiment, the lesion identification module is trained based on sample lymphoma image data and corresponding annotations, and the lesion identification module includes:

[0024] The backbone network, which adopts the CSPDarkNet architecture, is used to extract feature maps of different scales from the standard image data.

[0025] The feature pyramid network adopts an FPN architecture adapted to the CSPDarkNet architecture, which is used to fuse feature maps of different scales to obtain multiple fused features.

[0026] Multiple detection heads are connected to the output of the feature pyramid network to output lesion data based on different fusion features;

[0027] A filtering module is used to filter the lesion data through non-maximum suppression to obtain the lesion image set.

[0028] In a preferred embodiment, the lesion data includes the location, size, target confidence level, or category probability of the lesion.

[0029] In a preferred embodiment, the lesion images are aggregated and matched within the standard image data to obtain corresponding volume data and edge tightness data. This process specifically includes the following steps:

[0030] The lesion images are concentrated in lesion image B. i Matching is performed on the standard image data to obtain the corresponding three-dimensional voxel regions;

[0031] The number of voxels in the three-dimensional voxel region is counted, and the volume V of the three-dimensional voxel region is calculated from the volume of a single voxel. i ;

[0032] Using a 3D surface reconstruction algorithm, triangular mesh surfaces are extracted from the 3D voxel region, and the total area S of all triangular facets is calculated. i ;

[0033] Through the total area S i and the volume V i The edge compactness C was calculated. i ;

[0034] Repeat the above steps until all lesion images yield the corresponding volume and edge tightness.

[0035] In a preferred embodiment, based on the matching results of each lesion image in the standard image data, the corresponding metabolic data is obtained, specifically including the following steps:

[0036] The standard uptake of all voxels in the three-dimensional voxel region is traversed to obtain the maximum standard uptake SUV. max,i and average standard intake SUV mean,i ;

[0037] Based on the average standard intake SUV mean,i and the volume V of the three-dimensional voxel region i The total glycolysis amount (TLG) of the lesions was obtained. i ;

[0038] Repeat the above steps until the maximum standard uptake, average standard uptake, and total glycolysis of all lesions are obtained from all lesion images.

[0039] In a preferred embodiment, the convolutional neural network model includes a plurality of three-dimensional convolutional layers, three-dimensional pooling layers, and non-linear activation function layers connected in sequence.

[0040] In a preferred embodiment, the fusion unit specifically employs the following method for feature fusion:

[0041] The aforementioned basic features are copied and assigned to all lesions of the same patient;

[0042] Aggregated features are obtained by concatenating the morphological features, metabolic features, deep semantic features, and basic features corresponding to the same lesion.

[0043] Through a learnable attention network, attention weights are dynamically calculated based on the aggregated features and assigned to the metabolic features, morphological features, deep semantic features, and basic features.

[0044] Based on the calculated attention weights, the metabolic features, morphological features, deep semantic features, and basic features are weighted and summed to obtain the fused features.

[0045] In a preferred embodiment, the prediction unit includes a multilayer perceptron, which predicts the prognosis of chemotherapy for each lesion based on the fusion features, thereby obtaining the cure probability of each lesion.

[0046] The lymphoma identification and chemotherapy prognosis prediction system of the present invention has the following advantages compared with the prior art:

[0047] (1) A lymphoma identification and chemotherapy prognosis prediction system of the present invention includes a data acquisition module, a preprocessing module, a lesion identification module and a prognosis prediction module. The data acquisition module is used to acquire patient clinical feature data, biochemical feature data, lymphoma PET image data and corresponding CT image data. The preprocessing module is used to preprocess the patient clinical feature data and biochemical feature data to obtain basic features, and to preprocess the lymphoma PET image data based on CT images to obtain standard image data. On the one hand, it provides a multi-dimensional, high-quality and comparable input basis for subsequent analysis; on the other hand, it eliminates the technical variations caused by different scanning equipment, scanning protocols and patient physiological states, ensuring that the model learns and applies signals related to the essence of the disease rather than technical noise, thereby improving the stability of the model in different centers and on different equipment.

[0048] The lesion identification module adopts the YOLO architecture to identify lesions based on standard image data, obtaining a lesion image set. Using the YOLO architecture as the target detection network, it can automatically, in parallel, and efficiently identify and locate multiple lesions directly from the entire standard image data, generating a lesion image set. This avoids the cumbersome process of traditional manual or semi-automatic segmentation of each lesion, improving the efficiency of lesion identification and avoiding errors caused by the subjectivity of manual delineation, thus providing a precise localization foundation for subsequent lesion-level analysis.

[0049] The prognostic prediction module includes at least a morphological feature extraction unit, a metabolic feature extraction unit, a semantic feature extraction unit, a fusion unit, and a prediction unit. The morphological feature extraction unit aggregates lesion images and matches them against standard image data to obtain corresponding volumetric and edge compactness data. This volumetric and edge compactness data are then encoded to obtain morphological features, quantitatively describing the lesion's load size and invasiveness (irregularity) at the spatial structure level. The metabolic feature extraction unit, based on the matching results of each lesion image against standard image data, obtains corresponding metabolic data and encodes it to obtain metabolic features. Extracting this metabolic data enables an objective and quantitative characterization of the lesion's metabolic activity. The joint quantification of metabolic intensity and spatial morphology for each lesion provides richer feature dimensions and stronger characterization capabilities, offering a more comprehensive basis for prediction and avoiding the problem of existing technologies relying solely on a few simple parameters and having limited feature utilization.

[0050] The semantic feature extraction unit is used to extract deep semantic features from each lesion area through a convolutional neural network model. It can capture complex textures, heterogeneity and spatial relationship information that are difficult for the human eye to recognize or cannot be described by manual features, thereby enhancing the feature expression ability and discrimination potential of the model and improving prediction accuracy. The fusion unit integrates basic features, metabolic features, morphological features, and deep semantic features to obtain fusion features. The prediction unit predicts the prognosis of each lesion based on the fusion features, obtaining prediction results. On the one hand, it integrates the patient's basic features (clinical and biochemical), quantitative imaging features (metabolic and morphological), and deep imaging features (semantic). This multimodal fusion strategy allows the prediction model to not only "see" the images but also "understand" the patient's overall condition, achieving an organic combination of microscopic imaging information and macroscopic clinical background, improving the comprehensiveness and clinical relevance of the prognosis prediction model. On the other hand, it outputs the chemotherapy prognosis prediction results for each lesion individually, realizing differentiated prognosis assessment at the "lesion level." It can identify "weak" lesions in the same patient that may not respond well to treatment, thus providing clinicians with unprecedentedly refined decision support and providing direct and quantitative evidence for subsequent individualized treatment (such as local radiotherapy enhancement for low-response lesions), promoting the realization of precision medicine.

[0051] (2) The lymphoma identification and chemotherapy prognosis prediction system of the present invention also includes a result generation and visualization module, which is used to associate the predicted results of chemotherapy prognosis of each lesion with the position of the corresponding lesion in the image data and to visualize them, while generating a lesion-level prediction report. On the one hand, by associating the predicted results of chemotherapy prognosis of each lesion with the position of the corresponding lesion in the image data, the abstract prediction results are anchored back to the patient's specific images, so that the prediction results are transformed from "reference information" into "executable decision basis", so that doctors can intuitively see "which lesion location may have a good response and which lesion location may have a poor response", truly realizing the closed loop of "individualized lesion-level treatment"; on the other hand, the internal judgment of the model is intuitively visualized in a way that conforms to the clinical doctors' image reading habits, so that doctors can "clearly" understand the model's focus and prediction tendency, and combine their own experience to verify and interpret the visualization results, thereby improving people's trust and control over the machine decision-making process and promoting human-machine collaborative decision-making. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the technical process of an embodiment of the lymphoma identification and chemotherapy prognosis prediction system of the present invention;

[0053] Figure 2 This is a schematic diagram of the overall structure of a prediction system according to an embodiment of the lymphoma identification and chemotherapy prognosis prediction system of the present invention;

[0054] Figure 3 This is a schematic diagram of the lesion identification module structure of an embodiment of a lymphoma identification and chemotherapy prognosis prediction system of the present invention;

[0055] Figure 4 This is a schematic diagram of the backbone network CBS module, C3 module, and SPPF module of an embodiment of a lymphoma identification and chemotherapy prognosis prediction system of the present invention.

[0056] Figure 5 This is a schematic diagram of the convolutional neural network model structure of an embodiment of a lymphoma identification and chemotherapy prognosis prediction system of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0058] In the description of this invention, it should be understood that the terms "upper", "lower", "front", "rear", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0059] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, an integral connection, or a detachable connection; they can refer to the internal connection of two components; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0060] Example

[0061] This embodiment provides a lymphoma identification and chemotherapy prognosis prediction system, such as... Figure 1 and Figure 2 As shown, it includes a data acquisition module, a preprocessing module, a lesion identification module, a prognosis prediction module, and a result generation and visualization module.

[0062] The data acquisition module is used to acquire patient clinical characteristic data, biochemical characteristic data, lymphoma PET image data, and corresponding CT image data. In this embodiment, the biochemical characteristic data includes serum lactate dehydrogenase (LDH) levels and serum... Microglobulin levels ( (MG) Patient clinical characteristics data include at least one of age, sex, performance status (ECOG), B-symptom status, and Ann-Arbor stage. It should be noted that performance status (ECOG) is scored from 0 to 5. B-symptom status is a specific group of systemic symptoms, usually associated with a more active disease state and a worse prognosis, and is an important factor to consider when staging the disease and developing a treatment plan; it is coded with 0 and 1, where 0 indicates absence and 1 indicates presence. Ann-Arbor staging is based on the anatomical distribution of lymphoma cells in the lymphatic system and throughout the body, and is set as stages I-IV. It should be noted that age, serum lactate dehydrogenase (LDH) levels, and serum... Microglobulin levels ( MG is a continuous variable, while gender, performance status (ECOG), B symptom status, and Ann-Arbor stage are categorical variables.

[0063] The preprocessing module is used to preprocess patient clinical and biochemical characteristic data to obtain basic features, and to preprocess lymphoma PET image data based on CT images to obtain standard image data. On the one hand, this provides a multi-dimensional, high-quality, and comparable input foundation for subsequent analysis; on the other hand, it eliminates technical variations caused by different scanning equipment, scanning protocols, and patient physiological states, ensuring that the model learns and applies signals relevant to the essence of the disease, rather than technical noise, thereby improving the stability of the model in different centers and on different devices.

[0064] In this embodiment, Z-Score standardization is performed on continuous variables, specifically using the following calculation method:

[0065] ,

[0066] in, The standardized value. The original values ​​to be standardized. The mean to be standardized. The standard deviation is to be standardized. Standardization maps all continuous variables to the same scale centered at 0 with a standard deviation of 1, avoiding disproportionately high weights assigned to features due to their large original values. Categorical variables are numerically encoded; for example, gender is encoded as 0 for male and 1 for female. Other categorical variables are encoded using the same method, which will not be elaborated here. After encoding the patient-level clinical and biochemical feature data, they are concatenated into a fixed-dimensional vector to obtain the basic features. It should be noted that the basic features are patient-level features. For all lesions identified in the same patient, the basic features are copied and assigned to all lesions to ensure that all lesions in the same patient share the same clinical background information.

[0067] In this implementation, standard image data for lymphoma PET images is obtained by preprocessing CT images, specifically including the following steps:

[0068] Step 1. Normalize the lymphoma PET image data using standard uptake.

[0069] The raw standard uptake (SUV) of each voxel in the PET image data was obtained based on the patient's weight and the dosage of the injected radioactive tracer, specifically calculated using the following method:

[0070] ,

[0071] in, The raw standard uptake (SUV) is given for each voxel, where C is the tissue radioactivity concentration, D is the injection dose, and W is the patient's weight. It should be noted that a voxel is a "pixel" in three-dimensional space for PET and CT images.

[0072] Calculate the mean SUV value of all voxels in the background region. with standard deviation Based on the mean and standard deviation of the calculated SUV values, the raw standard intake (SUV) for each voxel is normalized using the following calculation method:

[0073] ,

[0074] in, It is a very small positive number, used to prevent the denominator from being 0.

[0075] Step 2. Using the spatial coordinate system of the CT image as a reference, perform rigid body registration between the normalized lymphoma PET image data and the CT image to obtain standard image data.

[0076] To eliminate the positional differences between PET and CT image data during scanning, the spatial orientation matrix (including origin, spacing, and direction cosines) of the CT image data is read and used as a fixed reference coordinate system. The normalized PET image data is used as the floating image to be registered, and the CT image is used as the reference image. A set of optimal rigid body transformation parameters is iteratively solved using the gradient descent method. The obtained rigid body transformation parameters are then applied to the normalized PET image data, specifically using the following calculation method:

[0077] ,

[0078] in, For the converted PET image data, For normalized PET image data, rigid body transformation parameters It includes three offset parameters and three rotation angle parameters, which are calculated using the following method:

[0079] ,

[0080] in, Let be a rotation matrix with three rotation angle parameters. This is a translation vector with three offset parameters. This embodiment also employs a trilinear interpolation algorithm to resample the transformed PET image data on a CT coordinate system grid, obtaining standard image data that is perfectly aligned with the spatial position of the CT image data.

[0081] The lesion identification module adopts the YOLO architecture to identify lesions based on standard image data, obtaining a lesion image set. Using the YOLO architecture as the target detection network, it can automatically, in parallel, and efficiently identify and locate multiple lesions directly from the entire standard image data, generating a lesion image set. This avoids the cumbersome process of traditional manual or semi-automatic segmentation of each lesion, improving the efficiency of lesion identification and avoiding errors caused by the subjectivity of manual delineation, thus providing a precise localization foundation for subsequent lesion-level analysis.

[0082] In this embodiment, the lesion identification module adopts the YOLOv5 framework, treating the detection task as a joint regression problem and directly outputting the lesion location and confidence level. For example... Figure 3 As shown, it includes:

[0083] (1) Backbone Network: The backbone network adopts the CSPDarkNet architecture to extract feature maps of different scales from standard image data. In this embodiment, the backbone network includes CBS module 1, CBS module 2, C3 module 1, CBS module 3, C3 module 2, CBS module 4, C3 module 3, CBS module 5, C3 module 4, and SPPF module connected in sequence, progressively downsampling, increasing the number of channels, and reducing the spatial size to extract feature maps of different scales. The SPPF module, as shown... Figure 4 As shown, it includes CBS module eleven, Maxpool layer, Maxpool layer, Maxpool layer, a splicing layer for splicing the outputs of CBS module eleven and the three Maxpool layers, and CBS module twelfth connected to the output of the splicing layer.

[0084] C3 Module 1, C3 Module 2, C3 Module 3, and C3 Module 4 use the same C3 module structure framework, such as Figure 4 As shown, the C3 module sequentially includes CBS module six, two parallel branches connected to the output of CBS module six, a splicing layer that connects the outputs of the two branches, and CBS module ten connected to the output of the splicing layer. The first branch includes CBS module nine; the second branch includes CBS modules seven and eight connected in sequence, and a residual connection that connects the input and output of the second branch. CBS modules one, two, three, four, five, six, seven, eight, nine, ten, eleven, and twelve adopt the same CBS module structure framework, as follows: Figure 4 As shown, the CBS module includes a Conv convolutional layer, a batch normalization layer, and a SiLU activation function layer connected in sequence for feature extraction.

[0085] (2) Feature Pyramid Network: The Feature Pyramid Network adopts an FPN architecture adapted to the CSPDarkNet architecture to fuse feature maps of different scales to obtain multiple fused features. In this embodiment, the feature pyramid includes a 1×1 convolutional layer 1 connected to the output of the SPPF module, a 1×1 convolutional layer 2 connected to the output of the C3 module 3, a 1×1 convolutional layer 3 connected to the output of the C3 module 2, and a 1×1 convolutional layer 4 connected to the output of the C3 module 1. The output features of the 1×1 convolutional layer 1 are passed through the 3×3 convolutional layer 1 to obtain the second feature data, and the second feature data is downsampled to obtain the first feature data. The output features of the 1×1 convolutional layer 1 are upsampled and fused with the output data of the 1×1 convolutional layer 2 to obtain the first fused data, and then passed through the 3×3 convolutional layer 2 to obtain the third feature data. The first fused data is upsampled and fused with the output features of the 1×1 convolutional layer 3 to obtain the second fused data, and then passed through the 3×3 convolutional layer 3 to obtain the fourth feature data. The second fused data is upsampled and fused with the output features of a 1×1 convolutional layer to obtain the third fused data, and then passed through a 3×3 convolutional layer to obtain the fifth feature data.

[0086] (3) Multiple detection heads, connected to the output of the feature pyramid network, are used to output lesion data based on different fusion features. In this embodiment, the detection heads include a first detection head for lymphoma identification based on first feature data, a second detection head for lymphoma identification based on second feature data, a third detection head for lymphoma identification based on third feature data, a fourth detection head for lymphoma identification based on fourth feature data, and a fifth detection head for lymphoma identification based on fifth feature data. The first, second, third, fourth, and fifth detection heads use 1x1 convolutional layers to perform dimensionality upscaling or dimensionality reduction operations on features of different scales and generate lesion data for each lesion. The lesion data includes the location, size, target confidence, or category probability of the lesion.

[0087] (4) Filtering module: The filtering module is used to filter lesion data through non-maximum suppression to obtain a lesion image set, specifically including the following steps:

[0088] Step 1. Data Preparation and Transformation. Merge all candidate bounding boxes of lesion data output from the detection heads into a single list. Convert the coordinate format of all bounding boxes from (center x, center y, width w, height h) to the (top left x1, top left y1, bottom right x2, bottom right y2) format required for calculating the Intersection over Union (IoU). The specific calculation method is as follows:

[0089] .

[0090] Step 2. Confidence Ranking. Based on the target confidence in the lesion data, sort the candidate box list in descending order.

[0091] Step 3. Iteratively suppress overlapping candidate boxes. Initialize an empty list. This is used to store the final filtered boxes. The highest-scoring box M from the sorted list is removed from the candidate list and added to the final output list. In the middle. The calculation box M and each remaining box in the candidate list. The Intersection over Union (IoU) is calculated using the following method:

[0092] ,

[0093] in, This represents the area of ​​the rectangle.

[0094] Delete all bounding boxes in the candidate list whose IoU with M is greater than the preset threshold N. These boxes point to the same lesion as M, but their localization is slightly different or their confidence level is low. Repeat the above steps until the candidate list is empty, at which point the iteration terminates. Output the final list. The image set of lesions consists of each bounding box. This represents a single, independently identified lymphoma lesion area by the system. In this embodiment, the preset threshold N is set to 0.45.

[0095] It should be noted that the lesion identification module is trained based on sample lymphoma image data and corresponding annotations. The sample lymphoma image data is obtained by acquiring PET images of patients before chemotherapy. The PET image data is normalized using standard uptake values ​​(SUVs) to ensure that the metabolic intensity of the lesion area is at a uniform scale. Rigid body registration is then performed between the normalized PET image data and CT image data, aligning the normalized PET image data to the CT image data coordinate system through translation and rotation. The aligned PET image data is then manually annotated at the first level, with each visible lymphoma lesion having a 3D bounding box. Data augmentation techniques, including rotation, translation, Gamma adjustment, and Gaussian noise perturbation, are then used to generate multiple sets of enhanced sample lymphoma image data.

[0096] Prognostic prediction module, such as Figure 2 As shown, it includes at least a morphological feature extraction unit, a metabolic feature extraction unit, a semantic feature extraction unit, a fusion unit, and a prediction unit. The morphological feature extraction unit is used to match the lesion images in the standard image data to obtain the corresponding volume data and edge compactness data, and to encode the volume data and edge compactness data to obtain morphological features, quantitatively describing the lesion's load size and growth invasiveness (irregularity) at the spatial structure level. In this embodiment, morphological feature extraction specifically includes the following steps:

[0097] Step 1. Set up the lesion image set Images of lesions in the middle Matching is performed on a standard image to obtain the corresponding three-dimensional voxel region.

[0098] Step 2. Count the number of voxels in the 3D voxel region, and calculate the volume V of the 3D voxel region using the volume of a single voxel. i The specific calculation method is as follows:

[0099] ,

[0100] in, Let i be the number of voxels in the i-th lesion region. The volume of a single voxel.

[0101] Step 3. Using a 3D surface reconstruction algorithm, extract the triangular mesh surface from the 3D voxel region and calculate the total area S of the triangular facets. i In this embodiment, the Marching Cubes algorithm is used to extract the triangular mesh surface. The mesh, composed of numerous triangular facets, represents the three-dimensional surface of the lesion region. The total area S is obtained by summing the surface areas of all triangular meshes. i .

[0102] Step 4. Through the total area S i and volume V i The edge compactness C was calculated. i The specific calculation method is as follows:

[0103] .

[0104] Step 5. Repeat the above steps until all lesion images yield corresponding volume and edge compactness. Encode the volume data and edge compactness data to obtain morphological features. The morphological features are specifically encoded using the following format:

[0105] ,

[0106] in, Let i be the volume of the i-th lesion. Let represent the tightness of the edge of the i-th lesion.

[0107] The metabolic feature extraction unit is used to obtain corresponding metabolic data based on the matching results of each lesion image in standard image data, and encode the metabolic data to obtain metabolic features. Extracting these metabolic data enables an objective and quantitative characterization of the metabolic activity of the lesions. The joint quantification of metabolic intensity and spatial morphology for each lesion provides richer feature dimensions and stronger characterization capabilities, offering a more comprehensive basis for prediction and avoiding the problem of limited feature utilization and reliance on only a few simple parameters in existing technologies. In this embodiment, the metabolic data includes the maximum standard uptake value (SUV). max,i Average standard intake of SUVs mean,i and total glycolysis (TLG) from lesions i Maximum standard intake value for SUVs max,i Used to characterize the highest metabolic activity of lesion i; mean standard intake (SUV) mean,i Used to characterize the overall metabolic level of lesion i. Total glycolysis (TLG) from lesions. i Used to characterize the total metabolic load of the i-th lesion. In this embodiment, metabolic feature extraction specifically includes the following steps:

[0108] Step 1. Traverse the standard uptake of all voxels in the three-dimensional voxel region to obtain the maximum standard uptake (SUV). max,i and average standard intake SUV mean,i .

[0109] Step 2. Based on average standard intake SUV mean,i and the volume V of the three-dimensional voxel region i The total glycolysis amount (TLG) of the lesions was obtained. i The specific calculation method is as follows:

[0110] .

[0111] Step 3. Repeat the above steps until the maximum standard uptake, average standard uptake, and total glycolysis of all lesions are obtained.

[0112] Step 4. Encode the metabolic data to obtain metabolic features. The metabolic features are encoded using the following format:

[0113] .

[0114] in, This represents the maximum standard uptake for the i-th lesion. The average standard uptake for the i-th lesion. The total glycolysis amount of the i-th lesion.

[0115] The semantic feature extraction unit is used to extract deep semantic features from each lesion region using a convolutional neural network model. It can capture complex textures, heterogeneity, and spatial relationship information that are difficult for the human eye to recognize or cannot be described by handcrafted features, enhancing the model's feature representation ability and discriminative potential, and improving prediction accuracy. The convolutional neural network model includes multiple sequentially connected three-dimensional convolutional layers, three-dimensional pooling layers, and non-linear activation function layers. In this embodiment, the convolutional neural network model, such as... Figure 5 As shown, it includes four sequentially connected convolutional downsampling modules and a global feature aggregation module. Each convolutional downsampling module includes a sequentially connected 3D convolutional layer, a batch normalization layer, a ReLU activation function layer, and a max pooling layer. The global feature aggregation module includes an adaptive average pooling layer, a flattening layer for flattening the 3D data into a 1D vector, and a fully connected layer.

[0116] The fusion unit is used to fuse basic features, metabolic features, morphological features, and deep semantic features to obtain fused features. In this embodiment, the fused features are obtained using the following method:

[0117] The basic features are replicated and assigned to all lesions of the same patient.

[0118] Aggregated features are obtained by splicing together the morphological features, metabolic features, deep semantic features and basic features corresponding to the same lesion.

[0119] A learnable attention network is used to dynamically calculate attention weights assigned to metabolic features, morphological features, deep semantic features, and basic features based on aggregated features. In this embodiment, morphological features, metabolic features, deep semantic features, and basic features are each projected into a common feature space through an independent learnable linear transformation layer (fully connected layer). For each projected feature, a scalar attention score is calculated. The specific calculation method is as follows:

[0120] ,

[0121] in To score attention, and For learnable parameters, The activation function is used. The scalar attention scores obtained from morphological features, metabolic features, deep semantic features, and basic features are normalized using Softmax to obtain the attention weights. .

[0122] Based on the calculated attention weights, metabolic features, morphological features, deep semantic features, and basic features are weighted and summed to obtain fused features. In this embodiment, the fused features... The specific calculation method is as follows:

[0123] .

[0124] The prediction unit is used to predict the prognosis of chemotherapy for each lesion based on fusion features, and obtain the prediction results. On the one hand, it fuses the patient's basic characteristics (clinical and biochemical), quantitative imaging characteristics (metabolic and morphological), and deep imaging characteristics (semantic). This multimodal fusion strategy allows the prediction model to not only "see" the images but also "understand" the patient's overall condition, achieving an organic combination of microscopic imaging information and macroscopic clinical background, improving the comprehensiveness and clinical relevance of the prognosis prediction model. On the other hand, it outputs the chemotherapy prognosis prediction result for each lesion individually, realizing differentiated prognosis assessment at the "lesion level." It can identify "weak" lesions in the same patient that may not respond well to treatment, thus providing clinicians with unprecedentedly refined decision support and providing direct and quantitative evidence for subsequent individualized treatment (such as local radiotherapy enhancement for low-response lesions), promoting the realization of precision medicine. In this embodiment, the prediction unit includes a multilayer perceptron, which predicts the prognosis of chemotherapy for each lesion based on fusion features, and obtains the cure probability of each lesion. In this embodiment, the prognosis prediction module is trained based on sample lymphoma image data and corresponding annotations. The secondary annotation content is the efficacy outcome of the lesion after completing the predetermined chemotherapy regimen, based on follow-up image data.

[0125] The results generation and visualization module is used to associate the predicted prognosis of each lesion after chemotherapy with the corresponding lesion's location in the image data and visualize it, while generating a lesion-level prediction report. On one hand, by associating the predicted prognosis of each lesion after chemotherapy with the corresponding lesion's location in the image data, the abstract prediction results are anchored back to the patient's specific images, transforming the prediction results from "reference information" into "executable decision-making basis." This allows doctors to intuitively see "which lesion location is likely to respond well, and which lesion location is likely to respond poorly," truly achieving a closed loop of "personalized lesion-level treatment." On the other hand, the model's internal judgments are intuitively visualized in a way that conforms to the clinical doctors' image reading habits, enabling doctors to "clearly understand" the model's focus and predictive tendencies. They can then verify and interpret the visualization results based on their own experience, increasing human trust and control over the machine's decision-making process and promoting human-machine collaborative decision-making. In this embodiment, the results generation and visualization specifically includes the following steps:

[0126] Step 1. Convert the coordinates of the lesion bounding box from the normalized coordinates or voxel index coordinates output by the model back to the physical coordinate system of the original image so that it can be accurately overlaid and displayed in the clinical image browsing system.

[0127] Step 2. Load and display the patient's CT images (as an anatomical background) and PET images (as a metabolic overlay). Draw the boundaries of each lesion at its physical coordinates using a rectangle (in 2D view) or a cube outline (in 3D view).

[0128] Step 3. Establish a mapping function from the prediction result to the color space, using a continuous gradient color system. The color depth is proportional to the prediction result, that is, the higher the probability, the more red the color and the deeper it is.

[0129] Step 4. Next to each lesion bounding box, label key information in the form of a text label, including at least the lesion number, prediction result, and detection confidence level.

[0130] Step 5. Generate a structured result, listing all detected lesions in tabular form, with each row corresponding to a lesion, including lesion ID, anatomical location, size, metabolic characteristics, prediction result, and prediction confidence.

[0131] In summary, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A lymphoma identification and chemotherapy prognosis prediction system, characterized in that, include: The data acquisition module is used to acquire patient clinical characteristic data, biochemical characteristic data, lymphoma PET image data, and corresponding CT image data; The preprocessing module is used to preprocess the patient's clinical and biochemical characteristic data to obtain basic features, and to preprocess the lymphoma PET image data based on the CT images to obtain standard image data. The lesion identification module adopts the YOLO architecture and is used to identify lesions based on the standard image data to obtain a lesion image set. The prognostic prediction module includes at least: The morphological feature extraction unit is used to aggregate the lesion images into a set of lesion images, match them in the standard image data, obtain the corresponding three-dimensional voxel regions, and acquire the corresponding volume data and edge compactness data. Specifically, the lesion images are aggregated into lesion image B. i Matching is performed on the standard image data to obtain the corresponding three-dimensional voxel regions; the number of voxels in the three-dimensional voxel regions is counted, and the volume V of the three-dimensional voxel regions is calculated using the volume of a single voxel. i Using a 3D surface reconstruction algorithm, triangular mesh surfaces are extracted from the 3D voxel region, and the total area S of all triangular faces is calculated. i ; through the total area S i and the volume V i The edge compactness C was calculated. i The volume data and edge compactness data are encoded to obtain morphological features; The metabolic feature extraction unit is used to obtain corresponding metabolic data based on the matching results of each lesion image in the standard image data. The metabolic data includes data based on the average standard intake (SUV). mean,i and the volume V of the three-dimensional voxel region i The total glycolysis amount (TLG) of the lesions was obtained. i The metabolic data is then encoded to obtain metabolic features; The semantic feature extraction unit is used to extract deep semantic features from each lesion region through a convolutional neural network model; The fusion unit is used to concatenate morphological features, metabolic features, deep semantic features and basic features corresponding to the same lesion through a learnable attention network to obtain aggregated features. Based on the aggregated features, attention weights are dynamically calculated and assigned to the metabolic features, morphological features, deep semantic features and basic features. Based on the calculated attention weights, the metabolic features, morphological features, deep semantic features and basic features are weighted and summed to obtain the fusion features. The prediction unit is used to predict the prognosis of chemotherapy for each lesion based on the fusion features, and obtain the prediction results.

2. The lymphoma identification and chemotherapy prognosis prediction system according to claim 1, characterized in that: It also includes a result generation and visualization module, which is used to associate the predicted prognosis of each lesion with the location of the corresponding lesion in the image data and display it visually, while generating a lesion-level prediction report.

3. A lymphoma identification and chemotherapy prognosis prediction system according to claim 1 or 2, characterized in that, The process of preprocessing the lymphoma PET image data based on the CT images to obtain standard image data specifically includes the following steps: The lymphoma PET image data were normalized using standard uptake. Using the spatial coordinate system of the CT image as a reference system, the normalized lymphoma PET image data is rigidly registered with the CT image to obtain standard image data.

4. A lymphoma identification and chemotherapy prognosis prediction system according to claim 1 or 2, characterized in that, The lesion identification module is trained based on sample lymphoma image data and corresponding annotations. The lesion identification module includes: The backbone network, which adopts the CSPDarkNet architecture, is used to extract feature maps of different scales from the standard image data. The feature pyramid network adopts an FPN architecture adapted to the CSPDarkNet architecture, which is used to fuse feature maps of different scales to obtain multiple fused features. Multiple detection heads are connected to the output of the feature pyramid network to output lesion data based on different fusion features; A filtering module is used to filter the lesion data through non-maximum suppression to obtain the lesion image set.

5. A lymphoma identification and chemotherapy prognosis prediction system according to claim 4, characterized in that, The lesion data includes the location, size, target confidence level, or category probability of the lesion.

6. A lymphoma identification and chemotherapy prognosis prediction system according to claim 1, characterized in that, Based on the matching results of each lesion image in the standard image data, the corresponding metabolic data is obtained, specifically including the following steps: The standard uptake of all voxels in the three-dimensional voxel region is traversed to obtain the maximum standard uptake SUV. max,i and average standard intake SUV mean,i ; Based on the average standard intake SUV mean,i and the volume V of the three-dimensional voxel region i The total glycolysis amount (TLG) of the lesions was obtained. i ; Repeat the above steps until the maximum standard uptake, average standard uptake, and total glycolysis of all lesions are obtained from all lesion images.

7. A lymphoma identification and chemotherapy prognosis prediction system according to any one of claims 1-2 or 5, characterized in that, The convolutional neural network model includes multiple three-dimensional convolutional layers, three-dimensional pooling layers, and non-linear activation function layers connected in sequence.

8. A lymphoma identification and chemotherapy prognosis prediction system according to any one of claims 1-2 or 5, characterized in that, The fusion unit specifically employs the following method for feature fusion: The aforementioned basic features are copied and assigned to all lesions of the same patient; Aggregated features are obtained by concatenating the morphological features, metabolic features, deep semantic features, and basic features corresponding to the same lesion. Through a learnable attention network, attention weights are dynamically calculated based on the aggregated features and assigned to the metabolic features, morphological features, deep semantic features, and basic features. Based on the calculated attention weights, the metabolic features, morphological features, deep semantic features, and basic features are weighted and summed to obtain the fused features.

9. A lymphoma identification and chemotherapy prognosis prediction system according to any one of claims 1-2 or 5, characterized in that, The prediction unit includes a multilayer perceptron, which predicts the prognosis of chemotherapy for each lesion based on the fusion features, and obtains the cure probability of each lesion.

Images