Medical image grading method fusing image features and symbol rule verification
By fusing 3D PET and CT images and combining them with medical logic rules for verification, the accuracy and interpretability issues of existing medical image grading systems have been resolved, achieving more efficient grading of follicular lymphoma and improving the system's reliability and interpretability.
Patent Information
- Application Number
- CN202511692199.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-18
- Publication Date
- 2026-02-17
AI Technical Summary
Existing medical image grading systems lack logical consistency constraints based on domain knowledge, resulting in low grading accuracy, difficulty in interpretation, and the risk of logical errors.
By fusing 3D PET images and 3D CT images, a 3D fused image is generated. This image is then processed using an image grading model and combined with a medical logic rule base for logical consistency verification, ensuring that the grading results conform to medical logic rules.
It improves the accuracy and interpretability of medical image grading, and the output results are consistent with medical logic rules, enhancing the credibility of medical artificial intelligence systems. It also supports automatic labeling of samples that violate the rules and provides explanations.
Smart Images
Figure CN121544554A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence-assisted medicine, and in particular to a medical image grading method that integrates image features and symbol rule verification. Background Technology
[0002] Medical image grading systems are a core application area of medical artificial intelligence. Automatic medical image classification systems can reduce human misdiagnosis and improve the speed of medical image analysis, especially in scenarios such as follicular lymphoma screening, enabling early lesion identification.
[0003] Existing medical image classification systems mainly rely on deep learning models to automatically classify medical images, such as Residual Networks (ResNet) and Densely Connected Convolutional Networks (DenseNet).
[0004] However, existing medical image grading systems typically lack logical consistency constraints on domain knowledge, are black-box structures, are difficult to interpret, and are prone to logical errors, resulting in low accuracy in grading medical images. Summary of the Invention
[0005] Therefore, it is necessary to provide a medical image grading method that integrates image features and symbol rule verification to address the aforementioned technical problems. This method improves the accuracy of medical image grading.
[0006] The present invention adopts the following technical solution: This invention provides a medical image grading method that integrates image features and symbol rule verification, comprising: Acquire three-dimensional positron emission tomography (PET) images and three-dimensional computed tomography (CT) images, and register and fuse the three-dimensional PET images and three-dimensional CT images to generate a three-dimensional fused image; the three-dimensional fused image includes metabolically active structures from the three-dimensional PET images and structural information from the three-dimensional CT images; The three-dimensional fused image is processed by an image grading model to obtain the grading prediction results of follicular lymphoma; the grading prediction results of follicular lymphoma include pathological grade labels and confidence levels; The metabolic activity and spatial distribution of follicular lymphoma in three-dimensional fused images were quantified to obtain actual structured variables; Logical consistency verification was performed on the follicular lymphoma grading prediction results and the actual structured variables to determine whether the follicular lymphoma grading prediction results conformed to the rules in the medical logic rule base. If the follicular lymphoma grading prediction result conforms to the rules in the medical logic rule base, then the follicular lymphoma grading prediction result is output.
[0007] Preferably, the actual structured variables include total tumor metabolism, tumor volume, maximum lesion spacing, and maximum standardized uptake value. Logical consistency verification is performed on the follicular lymphoma grading prediction results and the actual structured variables to determine whether the follicular lymphoma grading prediction results conform to the rules in the medical logic rule base, specifically including: The follicular lymphoma grading prediction results, along with the total metabolic tumor volume, maximum lesion spacing, and maximum standard uptake value, are input as first-order logic variables into the satisfiability module theory solver. In the satisfiability module theory solver, the first-order logic variables and the follicular lymphoma grading prediction results are judged to determine whether the follicular lymphoma grading prediction results conform to the rules in the medical logic rule base.
[0008] Preferably, the first-order logical variable and the follicular lymphoma grading prediction result are evaluated to determine whether the follicular lymphoma grading prediction result conforms to the rules in the medical logical rule base, specifically including: When the maximum standard uptake value is less than the first preset threshold of the maximum standard uptake value, the number of lesions is the preset number of lesions, and the follicular lymphoma grade prediction result is level one or level two, the rules in the medical logic rule base are satisfied. When the total metabolic tumor volume is greater than the first preset threshold for total metabolic tumor volume and the follicular lymphoma grade prediction result is level one, the rules in the medical logic rule base are not satisfied. When the maximum lesion spacing exceeds the preset threshold, the lesion distribution spans multiple lymph node regions, and the follicular lymphoma grade prediction result is level one, the rules in the medical logic rule base are not satisfied.
[0009] Preferably, the method further includes: If the follicular lymphoma grading prediction results do not conform to the rules in the medical logic rule base, modify the label of the conflict area in the follicular lymphoma grading prediction results to a low confidence mark. The Grad-CAM algorithm is used to generate a heatmap, and conflict areas are highlighted. Low confidence markers and reference explanations are used as conflict description text for conflict areas; reference explanations include medical rules, reasons for conflict, and recommendations.
[0010] Preferably, the construction process of the medical logic rule base specifically includes: Obtain the registered sample fusion 3D image set and the corresponding diagnostic follicular lymphoma grading results; By reviewing clinical guidelines, research literature, and clinical consensus, we obtained diagnostic elements related to the grading of follicular lymphoma, and extracted key discriminant features and corresponding thresholds from these diagnostic elements. Based on key discriminant features and the thresholds corresponding to those key discriminant features, a candidate hierarchical feature set is constructed; The registered sample fused 3D image set was analyzed using image processing tools to extract the actual structured variables related to the lesions, and an indicator dataset corresponding to the graded results of the diagnosed follicular lymphoma was obtained. Based on candidate hierarchical feature sets and indicator datasets, statistical modeling, visualization analysis, or rule-based filtering are used to identify regular combinations between candidate hierarchical feature sets and indicator datasets. Regular combination relationships are identified as a medical logic rule base.
[0011] Preferably, the medical logic rule base is as follows: If the maximum standard uptake value is less than the first preset threshold of the maximum standard uptake value and the number of lesions is the preset number of lesions, then the follicular lymphoma grading result is grade I or grade II. If the total metabolic tumor volume is greater than the first preset threshold for total metabolic tumor volume, the follicular lymphoma grade result will not be grade one. If the maximum distance between lesions is greater than the preset threshold for the maximum distance between lesions and the lesions are distributed across multiple lymphatic regions, the follicular lymphoma grade result will not be grade one. If the lesion is confined to a single lymph node region and the maximum standard uptake value is less than the second preset threshold of the maximum standard uptake value, the follicular lymphoma grade is grade I. If the maximum standard uptake value is greater than or equal to the third preset threshold of the maximum standard uptake value and the total metabolic tumor volume is greater than or equal to the second preset threshold of the total metabolic tumor volume, then the follicular lymphoma grading result is not grade I or grade II.
[0012] Preferably, the three-dimensional PET images and three-dimensional CT images are registered and fused using an image fusion network; the image fusion network adopts 3D-DDcGAN; the image grading model adopts a convolutional neural network or a Bayesian neural network based on the ResNet18 structure.
[0013] Preferably, the training process of the image grading model specifically includes: The registered datasets of sample 3D PET images and sample CT images, along with the corresponding diagnostic follicular lymphoma grading results, were used as the training set. The diagnostic follicular lymphoma grading results included grade I, II, and III. An image fusion network is used to fuse registered sample 3D PET images and sample CT image datasets to generate a sample 3D fused image set. Cross-entropy loss function and variational Bayesian inference loss function based on KL divergence are used as loss functions for the image grading model. Using the grading results of confirmed follicular lymphoma as a supervisory signal, the image grading model is trained using the training set to minimize the loss value calculated by the loss function; The image grading model that minimizes the loss value is determined as the trained image grading model.
[0014] This invention provides a medical image grading device that integrates image features and symbol rule verification, comprising: The acquisition module is used to acquire three-dimensional positron emission tomography (PET) images and three-dimensional computed tomography (CT) images, and to register and fuse the three-dimensional PET images and three-dimensional CT images to generate a three-dimensional fused image; the three-dimensional fused image includes metabolically active structures in the three-dimensional PET images and structural information in the three-dimensional CT images; The image grading module is used to process the 3D fused image through the image grading model to obtain the grading prediction result of follicular lymphoma; the grading prediction result of follicular lymphoma includes pathological grade label and confidence level; The structured index extraction module is used to quantify the metabolic activity and spatial distribution of follicular lymphoma in three-dimensional fused images to obtain actual structured variables. The symbol rule verification module is used to verify the logical consistency between the follicular lymphoma grading prediction results and the actual structured variables, in order to determine whether the follicular lymphoma grading prediction results conform to the rules in the medical logic rule base. The credibility output module is used to output the follicular lymphoma grading prediction result if the prediction result conforms to the rules in the medical logic rule base.
[0015] The present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described medical image grading method that integrates image features and symbol rule verification.
[0016] The present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-mentioned medical image grading method that integrates image features and symbol rule verification.
[0017] The above-mentioned at least one technical solution adopted in this invention can achieve the following beneficial effects: This method registers and fuses 3D CT and 3D PET images to obtain a 3D fused image. By integrating anatomical information from CT and metabolic activity data from PET, this 3D fused image overcomes the limitations of single-modal imaging. The 3D fused image is processed using an image grading model to obtain follicular lymphoma grading prediction results. The metabolic activity and spatial distribution of follicular lymphoma in the 3D fused image are quantified to obtain actual structured variables, providing quantifiable medical indicators for subsequent logical verification. Logical consistency verification is performed between the follicular lymphoma grading prediction results and the actual structured indicators to determine whether the follicular lymphoma grading prediction results conform to the rules in the medical logic rule base. The consistency of the image grading model output with medical logic rules is judged, improving the accuracy of follicular lymphoma grading prediction results. If the follicular lymphoma grading prediction results conform to the rules in the medical logic rule base, the follicular lymphoma grading prediction result is output. This method improves the accuracy of medical image grading. Attached Figure Description
[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:
[0019] Figure 1 A schematic diagram of a medical image grading method that integrates image features and symbol rule verification provided by the present invention; Figure 2 The system structure diagram provided for this invention; Figure 3 The prediction flowchart provided by this invention; Figure 4 This is a schematic diagram illustrating the modules provided by the present invention; Figure 5 This is a schematic diagram of the predicted conflict provided by the present invention; Figure 6 A schematic diagram of a medical image grading device that integrates image features and symbol rule verification provided by the present invention; Figure 7 This is a schematic diagram of a computer device for implementing a medical image grading method that integrates image features and symbol rule verification, as provided by the present invention. Detailed Implementation
[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0021] Devices such as desktop computers, servers, and laptops are capable of executing the solutions of this invention. For ease of explanation, the following description will focus on servers as the executing entity.
[0022] In the actual clinical grading of follicular lymphoma, doctors combine imaging indicators with empirical rules for judgment, such as "if SUVmax < 10 and the lesion is solitary, it may be FL I or II," etc. These rules have strong interpretability and clinical applicability; however, existing deep learning models struggle to naturally embed these medical rules.
[0023] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0024] Figure 1 This is a schematic diagram of a medical image grading method that integrates image features and symbol rule verification according to the present invention, which specifically includes the following steps: S101: Acquire three-dimensional positron emission tomography (PET) images and three-dimensional computed tomography (CT) images, and register and fuse the three-dimensional PET images and three-dimensional CT images to generate a three-dimensional fused image; the three-dimensional fused image includes metabolically active structures in the three-dimensional PET images and structural information in the three-dimensional CT images.
[0025] In an exemplary embodiment, the image fusion network uses 3D-DDcGAN; the image grading model uses a convolutional neural network or a Bayesian neural network based on the ResNet18 structure; and the structured index extraction module uses image processing tools to extract the actual structured variables of the 3D fused image.
[0026] Specifically, this invention performs registration processing on raw 3D PET and 3D CT images from clinical settings and generates a fused image through an image fusion module. The image fusion module uses a 3D dual discriminator conditional generative adversarial network (3D-DDcGAN) to effectively combine metabolic activity (such as high SUV regions) in 3D PET images with structural information (boundaries, organ contours) in 3D CT images, outputting a fused image as the main input for model understanding. The 3D fused image simultaneously retains the structural information from the CT image and the metabolic information from the PET image. The 3D fused image provides raw data for subsequent image fusion and structured feature extraction. The 3D fused image is given by formula (1):
[0027] f =G( PET , CT (1); in, f For 3D fused images, G is The generator subnetwork in the image fusion module, PET For 3D PET images, CT This is a 3D CT image.
[0028] The generator subnetwork in the image fusion module receives registered 3D PET and 3D CT images as input and outputs a fused 3D image through a conditional generation model. f The generator can be constructed using a 3D convolutional structure to simultaneously model the spatial correlation between metabolic and structural features. G This represents the generator network in the image fusion module. The generator is a core component of the 3D-DDcGAN structure.
[0029] S102: The three-dimensional fused image is processed by the image grading model to obtain the grading prediction result of follicular lymphoma; the grading prediction result of follicular lymphoma includes pathological grade label and confidence level.
[0030] In an exemplary embodiment, a 3D PET image and a 3D CT image are registered and fused using an image fusion network; the image fusion network uses 3D-DDcGAN; and the image grading model uses a convolutional neural network or a Bayesian neural network based on the ResNet18 structure.
[0031] In an exemplary embodiment, the training process of the image grading model specifically includes: using a registered dataset of sample 3D PET images and sample CT images, along with the corresponding diagnostic follicular lymphoma grading results, as a training set; the diagnostic follicular lymphoma grading results include Level 1, Level 2, and Level 3; employing an image fusion network to fuse the registered datasets of sample 3D PET images and sample CT images to generate a sample 3D fused image set; using a cross-entropy loss function and a variational Bayesian inference loss function based on KL divergence as the loss function for the image grading model; using the diagnostic follicular lymphoma grading results as a supervision signal, training the image grading model using the training set to minimize the loss value calculated by the loss function; and determining the image grading model corresponding to the minimum loss value as the trained image grading model.
[0032] The fused images are processed using a ResNet18-based convolutional neural network (ResNet18 architecture) or a Bayesian CNN (Bayesian Convolutional Neural Network) to output pathological grade predictions (I / II / III) for follicular lymphoma (FL). The image grading model reads the fused images and performs FL pathological grading (e.g., FL I, FL II, and FL III), outputting classification probabilities or uncertainty indices. The image grading model outputs predicted labels and confidence scores.
[0033] The training phase of the image grading model uses grading labels from clinical pathological diagnoses as supervision signals, and can selectively add rule consistency regularization terms.
[0034] During the training phase of the image grading model, grading labels from clinically diagnosed data serve as supervisory signals. These labels are generated based on the correspondence between historical case image data and pathology reports. The supervisory signals originate from public medical databases or multi-center case databases. Registered images and their corresponding pathological grading labels form training pairs, which guide the image grading model in supervised learning.
[0035] S103: The actual structured variables are obtained by quantifying the metabolic activity and spatial distribution of follicular lymphoma in three-dimensional fused images.
[0036] The actual structured variables include total metabolic tumor volume (TMTV), maximum dissemination distance (DMAX), and maximum standardized uptake value (SUVmax).
[0037] S104: Perform logical consistency verification between the follicular lymphoma grading prediction results and the actual structured variables to determine whether the follicular lymphoma grading prediction results conform to the rules in the medical logic rule base.
[0038] In one exemplary embodiment, the process of constructing a medical logic rule base specifically includes: The process involves acquiring a registered sample fusion 3D image set and the corresponding diagnostic follicular lymphoma (FDL) grading results. By reviewing clinical guidelines, research literature, and clinical consensus, diagnostic elements related to FDL grading are identified, and key discriminant features and corresponding thresholds are extracted from these elements. A candidate grading feature set is constructed based on these key discriminant features and their corresponding thresholds. The registered sample fusion 3D image set is analyzed using image processing tools to extract actual structured variables related to lesions, resulting in an indicator dataset corresponding to the diagnosed FDL grading results. Based on the candidate grading feature set and indicator dataset, statistical modeling, visualization analysis, or rule-based filtering are employed to identify regular combinations between the candidate grading feature set and the indicator dataset. These regular combinations are then defined as a medical logic rule base.
[0039] In an exemplary embodiment, the medical logic rule base is as follows: if the maximum standard uptake value is less than a first preset threshold for the maximum standard uptake value and the number of lesions is a preset number of lesions, then the follicular lymphoma grading result is level one or level two; if the total metabolic tumor volume is greater than a first preset threshold for the total metabolic tumor volume, then the follicular lymphoma grading result is not level one; if the maximum lesion spacing is greater than a preset threshold for the maximum lesion spacing and the lesions are distributed across multiple lymphatic regions, then the follicular lymphoma grading result is not level one; if the lesions are confined to a single lymphatic region and the maximum standard uptake value is less than a second preset threshold for the maximum standard uptake value, then the follicular lymphoma grading result is level one; if the maximum standard uptake value is greater than or equal to a third preset threshold for the maximum standard uptake value and the total metabolic tumor volume is greater than or equal to a second preset threshold for the total metabolic tumor volume, then the follicular lymphoma grading result is not level one or level two.
[0040] Specifically, a rule base containing several medical logic rules is constructed and expressed in symbolic logic form.
[0041] 1. Obtain diagnostic elements from clinical literature and guidelines: First, systematically review authoritative guidelines, research literature and clinical consensus related to the pathological grading of follicular lymphoma (FL), extract key discriminant features such as SUVmax (maximum standard uptake value), TMTV (total metabolic tumor volume), Dmax (maximum lesion spacing), number of lesions and other indicators, and record the widely recognized grading thresholds and empirical judgment rules.
[0042] 2. Extracting structured indicators from images: Based on the above features, combined with labeled data and image processing tools (such as 3DSlicer and LIFEx), structured quantitative indicators are extracted from the patient's fused images to form a dataset of structured variables associated with the labels.
[0043] 3. Constructing the association pattern between labels and structural features: Through statistical modeling, visualization analysis, or expert verification, identify the correspondence between structured indicators and hierarchical labels, and screen out regular combinations with high relevance and high consistency.
[0044] 4. Formal modeling of rules: The above patterns are refined into formal logical rules, specifically manifested as first-order predicate logic (such as "if SUVmax < 10 and the number of lesions = 1, then it should be low-grade follicular lymphoma (FL)") or linear inequality expressions, and a unified rule interface template is constructed.
[0045] Build a rule base: Store all rules as a set of logical expressions, so that the subsequent symbol verification module can directly read and input them into the SMT solver for consistency verification.
[0046] 5. Examples of some of the constructed medical logic rules are as follows: If SUVmax < 10 and the number of lesions = 1, the prediction label should be FL I or FL II; if TMTV > 408.72, the prediction label should not be FL I; if Dmax > 180mm and the lesions are distributed across multiple lymphatic regions, the prediction label should not be FL I; if the lesions are confined to a single region and SUVmax < 8, the prediction is recommended to be FL I; if SUVmax ≥ 15 and TMTV ≥ 600, there may be a risk of conversion to DLBCL, and the prediction should not be FL I or II.
[0047] These rules are all expressed using first-order logic formulas or linear constraints, and have been reviewed by medical experts and verified by data, thus possessing high interpretability and universality.
[0048] In an exemplary embodiment, the actual structured variables include the total metabolic tumor volume, maximum lesion spacing, and maximum standardized uptake value of the tumor. Logical consistency verification is performed on the follicular lymphoma grading prediction results and the actual structured variables to determine whether the follicular lymphoma grading prediction results conform to the rules in the medical logic rule base. Specifically, this includes: inputting the follicular lymphoma grading prediction results, along with the total metabolic tumor volume, maximum lesion spacing, and maximum standardized uptake value of the tumor, as first-order logical variables into a satisfiability modular theory solver; and in the satisfiability modular theory solver, judging the first-order logical variables and the follicular lymphoma grading prediction results to determine whether the follicular lymphoma grading prediction results conform to the rules in the medical logic rule base.
[0049] In an exemplary embodiment, the first-order logical variable and the follicular lymphoma grading prediction result are judged to determine whether the follicular lymphoma grading prediction result conforms to the rules in the medical logical rule base. Specifically, this includes: when the maximum standard uptake value is less than a first preset threshold of the maximum standard uptake value, the number of lesions is a preset number of lesions, and the follicular lymphoma grading prediction result is level one or level two, the rules in the medical logical rule base are met; when the total metabolic tumor volume is greater than a first preset threshold of the total metabolic tumor volume and the follicular lymphoma grading prediction result is level one, the rules in the medical logical rule base are not met; when the maximum lesion spacing is greater than a preset threshold of the maximum lesion spacing, the lesion distribution spans multiple lymphatic regions, and the follicular lymphoma grading prediction result is level one, the rules in the medical logical rule base are not met.
[0050] Specifically, image processing tools such as LIFEx and ITK-SNAP are used to extract parameters such as SUVmax, TMTV, and maximum lesion spacing Dmax, and then standardized modeling is performed using rule templates. For example:
[0051] If SUVmax < 10 and the number of lesions = 1 → label ∈ {FL I, FL II}, FL I is grade I follicular lymphoma, FL II is grade II follicular lymphoma, and FL III is grade III follicular lymphoma.
[0052] If TMTV > 408.72 → it should not be FL. All rules will be input into the Satisfiability Modulo Theories (SMT) solver in first-order logic form.
[0053] All rules will be formally modeled using first-order logic formulas or linear inequality expressions, serving as input for subsequent verification modules.
[0054] Medical logic expression and reasoning mechanism based on satisfiability model theory.
[0055] This invention employs Structured Modeling (SMT) as the formal support framework for logical verification. Structured variables extracted from the image (such as SUVmax, TMTV, and the number of lesions) are encoded together with the grading prediction results as constraints, and the Z3 solver is used for satisfiability assessment. SMT supports various complex expressions such as numerical inequalities, Boolean connections, and conditional rules, enabling high-precision modeling and automatic verification of clinical rules, ensuring the logical acceptability of the output labels. The actual structured indicators of the 3D fused image are provided to the symbolic rule verification module for judgment and do not directly participate in classification prediction.
[0056] Specifically, the symbolic solver Z3 is used to verify the logical consistency of the model's predicted labels and extracted metrics, determining whether the predictions violate medical rules. For example, if SUVmax < 10, it should generally not be FL III; this ensures that the model's behavior conforms to medical common sense and enhances its credibility.
[0057] S105: If the follicular lymphoma grading prediction result conforms to the rules in the medical logic rule base, then output the follicular lymphoma grading prediction result.
[0058] In an exemplary embodiment, the method further includes: if the follicular lymphoma grading prediction result does not conform to the rules in the medical logic rule base, modifying the label of the conflict area in the follicular lymphoma grading prediction result to a low confidence mark; calling the Grad-CAM algorithm to generate a heat map and highlighting the conflict area; using the low confidence mark and reference explanation as the conflict description text of the conflict area; the reference explanation includes medical rules, conflict reasons and suggestions.
[0059] Specifically, if the rule validation passes, the model predicts the label normally; if the rule validation fails, i.e. the consistency check structure is inconsistent, a "low confidence" label is output, along with: suggested label adjustments (e.g., change to FLII); a Grad-CAM heatmap as an interpretive reference; and finally, suggestions on whether to trust the results when providing them to doctors.
[0060] If the follicular lymphoma grading prediction result does not conform to the rules in the medical logic rule base, this invention calls the Gradient-weighted Class Activation Mapping (Grad-CAM) algorithm to generate a heatmap and highlights the areas of interest in the model. Combined with medical rule conflict information, a graphic explanation page is generated for doctors to view.
[0061] The output of this invention includes: follicular lymphoma grading prediction results, logical verification status (whether consistent), confidence score, heatmap interpretation image, and conflict explanation.
[0062] In one exemplary embodiment, the present invention provides as follows Figure 2 The system structure diagram shown is as follows: Figure 2 As shown, the entire system is divided into an image input module, an image fusion module, an image grading module, a structured index extraction module, a symbol rule verification module, and a trusted output module.
[0063] In one exemplary embodiment, the present invention provides as follows Figure 3 The diagram shows the functional layout of the modules.
[0064] In one exemplary embodiment, the present invention provides as follows Figure 4 The prediction flowchart shown is as follows: Figure 3 As shown, the input image module receives CT and PET images; the image fusion module performs multimodal image processing; the image classification module predicts FL levels; the image extraction module extracts indicators such as SUVmax and TMTV; the symbolic logic verification module invokes rule constraints (Z3); the consistency verification module determines whether it is logically acceptable; and there are two output branches for reliable prediction or low-reliability + interpretation.
[0065] In one exemplary embodiment, the present invention provides as follows Figure 5 The diagram illustrating prediction conflicts shows that the background image is a real PET / CT fusion slice image provided by this invention, displaying tumor metabolism and structure; the highlighted red and yellow areas are Grad-CAM heatmaps: the areas of interest of the simulation model (prediction drivers); the yellow-bordered areas are rule conflict areas: where structured indicators (such as TMTV) contradict the prediction labels; the boxes represent the prediction results of this invention, but their base areas differ from the expected locations of medical rules, thus indicating "logical conflict".
[0066] This invention is based on medical images and combines executable medical logic rules with formal verification mechanisms to make FL grading results safer, more controllable, and more interpretable, making them suitable for deployment and application in medical auxiliary diagnostic systems.
[0067] The beneficial effects of this invention are as follows: 1. The output results of the image grading model have consistency with medical logical rules, which improves the credibility of medical artificial intelligence (AI) systems.
[0068] 2. It can automatically label samples that violate medical logic rules, making it easier for doctors to review them further.
[0069] 3. The interpretability enhancement mechanism can help doctors understand the reasons for prediction errors.
[0070] 4. Supports dynamic expansion of knowledge rules, applicable to migration to other cancer types or tasks.
[0071] 5. It can transform some medical logic rules into training constraints, realizing "rule-guided learning".
[0072] When applying the medical image grading method that integrates image features and symbol rule verification provided by this invention, it is not necessary to rely on... Figure 1 The steps shown are executed in sequence. The specific execution order of each step can be determined as needed, and this invention does not impose any restrictions on it.
[0073] The above describes a medical image grading method that integrates image features and symbolic rule verification, provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding medical image grading device that integrates image features and symbolic rule verification, such as... Figure 2 As shown.
[0074] Figure 6 A schematic diagram of a medical image grading device that integrates image features and symbol rule verification provided by the present invention includes: The acquisition module 601 is used to acquire three-dimensional positron emission tomography (PET) images and three-dimensional computed tomography (CT) images, and to register and fuse the three-dimensional PET images and three-dimensional CT images to generate a three-dimensional fused image; the three-dimensional fused image includes metabolically active structures in the three-dimensional PET images and structural information in the three-dimensional CT images.
[0075] The image grading module 602 is used to process the three-dimensional fused image through the image grading model to obtain the grading prediction result of follicular lymphoma; the grading prediction result of follicular lymphoma includes pathological grade label and confidence level.
[0076] The structured index extraction module 603 is used to quantify the metabolic activity and spatial distribution of follicular lymphoma in three-dimensional fused images to obtain actual structured variables.
[0077] The symbol rule verification module 604 is used to verify the logical consistency between the follicular lymphoma grading prediction results and the actual structured variables, in order to determine whether the follicular lymphoma grading prediction results conform to the rules in the medical logic rule base.
[0078] The credibility output module 605 is used to output the follicular lymphoma grading prediction result if the follicular lymphoma grading prediction result conforms to the rules in the medical logic rule base.
[0079] Specific limitations regarding a medical image grading device that integrates image feature and symbol rule verification can be found in the above-described limitations of a medical image grading method that integrates image feature and symbol rule verification, and will not be repeated here. Each module in the aforementioned medical image grading device integrating image feature and symbol rule verification can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.
[0080] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 This paper presents a medical image grading method that integrates image features and symbol rule verification.
[0081] The present invention also provides Figure 7 The schematic diagram of the computer device shown is as follows: Figure 7 As shown, at the hardware level, this computer device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then executes it to achieve the above. Figure 1 This paper presents a medical image grading method that integrates image features and symbol rule verification.
[0082] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.
[0083] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this invention.
Claims
1. A medical image grading method that fuses image features and symbolic rule verification, characterized by, The method comprises the following steps: acquiring three-dimensional positron emission computed tomography (PET) images and three-dimensional computed tomography (CT) images, and registering and fusing the three-dimensional PET images and the three-dimensional CT images to generate three-dimensional fused images; the three-dimensional fused images comprise metabolic activity structures in the three-dimensional PET images and structural information in the three-dimensional CT images; processing the three-dimensional fused images through an image grading model to obtain a follicular lymphoma grading prediction result; the follicular lymphoma grading prediction result comprises a pathological grading label and a confidence level; quantifying metabolic activity and spatial distribution of follicular lymphoma in the three-dimensional fused images to obtain actual structured variables; performing logical consistency verification on the follicular lymphoma grading prediction result and the actual structured variables to determine whether the follicular lymphoma grading prediction result conforms to rules in a medical logical rule base; if the follicular lymphoma grading prediction result conforms to the rules in the medical logical rule base, outputting the follicular lymphoma grading prediction result.
2. The method of claim 1, wherein, The actual structured variables comprise total metabolic tumor volume, maximum lesion spacing, and maximum standard uptake value; the logical consistency verification on the follicular lymphoma grading prediction result and the actual structured variables to determine whether the follicular lymphoma grading prediction result conforms to the rules in the medical logical rule base specifically comprises: inputting the follicular lymphoma grading prediction result and the total metabolic tumor volume, the maximum lesion spacing, and the maximum standard uptake value as first-order logical variables into a satisfiability modulo theories (SMT) solver; in the SMT solver, determining whether the follicular lymphoma grading prediction result conforms to the rules in the medical logical rule base by judging the first-order logical variables and the follicular lymphoma grading prediction result.
3. The method of claim 2, wherein, The determination of whether the follicular lymphoma grading prediction result conforms to the rules in the medical logical rule base by judging the first-order logical variables and the follicular lymphoma grading prediction result specifically comprises: when the maximum standard uptake value is less than a first preset threshold of the maximum standard uptake value, the number of lesions is a preset number of lesions, and the follicular lymphoma grading prediction result is grade 1 or grade 2, the rules in the medical logical rule base are satisfied; when the total metabolic tumor volume is greater than a first preset threshold of the total metabolic tumor volume and the follicular lymphoma grading prediction result is grade 1, the rules in the medical logical rule base are not satisfied; when the maximum lesion spacing is greater than a preset threshold of the maximum lesion spacing, the lesion distribution spans multiple lymphatic regions, and the follicular lymphoma grading prediction result is grade 1, the rules in the medical logical rule base are not satisfied.
4. The method of claim 3, wherein, The method further comprises: if the follicular lymphoma grading prediction result does not conform to the rules in the medical logical rule base, modifying the label of a conflict area in the follicular lymphoma grading prediction result to a low-confidence label; calling a Grad-CAM algorithm to generate a heat map and highlight the conflict area; taking the low-confidence label and a reference explanation as conflict explanation text of the conflict area; the reference explanation comprises medical rules, conflict reasons, and suggestion explanations.
5. The method of claim 1, wherein, The construction process of the medical logical rule base specifically comprises: obtaining a registered sample fused three-dimensional image set and a confirmed follicular lymphoma grading result corresponding to the sample fused three-dimensional image set; obtaining diagnostic elements related to follicular lymphoma grading by combing clinical guidelines, research literature and clinical consensus, and extracting key discriminant features and threshold values corresponding to the key discriminant features from the diagnostic elements; constructing a candidate grading feature set based on the key discriminant features and the threshold values corresponding to the key discriminant features; extracting actual structured variables related to the lesion by analyzing the registered sample fused three-dimensional image set through an image processing tool, and obtaining an index data set corresponding to the confirmed follicular lymphoma grading result; based on the candidate grading feature set and the index data set, using statistical modeling, visual analysis or rule screening to identify the regular combination relationship between the candidate grading feature set and the index data set; determining the regular combination relationship as a medical logic rule base.
6. The method of claim 5, wherein, The medical logic rule base is: if the maximum standard uptake value is less than the maximum standard uptake value first preset threshold value and the number of lesions is a preset number of lesions, then the follicular lymphoma grading result is grade one or grade two; if the total metabolic tumor volume is greater than the total metabolic tumor volume first preset threshold value, then the follicular lymphoma grading result is not grade one; if the maximum lesion spacing is greater than the maximum lesion spacing preset threshold value and the lesion distribution spans multiple lymphatic regions, then the follicular lymphoma grading result is not grade one; if the lesion is limited to a single lymphatic region and the maximum standard uptake value is less than the maximum standard uptake value second preset threshold value, then the follicular lymphoma grading result is grade one; if the maximum standard uptake value is greater than or equal to the maximum standard uptake value third preset threshold value and the total metabolic tumor volume is greater than or equal to the total metabolic tumor volume second preset threshold value, then the follicular lymphoma grading result is not grade one or grade two.
7. The method of claim 1, wherein, The three-dimensional PET image and the three-dimensional CT image are registered and fused through an image fusion network; the image fusion network uses 3D-DDcGAN; and the image grading model uses a convolutional neural network or a Bayesian neural network based on a ResNet18 structure.
8. The method of claim 7, wherein, The training process of the image grading model specifically includes: using the registered sample three-dimensional PET image and sample CT image data set, and the confirmed follicular lymphoma grading result corresponding to the sample three-dimensional PET image and sample CT image set as a training set; the confirmed follicular lymphoma grading result includes grade one, grade two and grade three; using the image fusion network to fuse the registered sample three-dimensional PET image and sample CT image data set to generate a sample three-dimensional fused image set; using a cross-entropy loss function and a variational Bayesian inference loss function based on KL divergence as the loss function of the image grading model; using the confirmed follicular lymphoma grading result as a supervision signal, training the image grading model through the training set to minimize the loss value calculated by the loss function; determining the image grading model corresponding to the minimum loss value as the trained image grading model.