Mammary gland ultrasonic focus explaining and diagnosing system, storage medium and electronic equipment

Abstract

The invention provides a mammary gland ultrasonic focus interpretable diagnosis system, a storage medium and electronic equipment, and relates to the technical field of medical image artificial intelligence auxiliary diagnosis. According to the method, image attribute feature tags are mapped to corresponding nodes of a knowledge graph. Then, the graph neural network model carries out multi-hop reasoning on a graph structure through a message passing mechanism; according to the method, not only can information from related attribute nodes be aggregated, but also the reasoning result of the graph neural network model can be calibrated and explained by utilizing clinical expert rules extracted from the knowledge graph, so that a clear and traceable reasoning path is attached to each prediction of the graph neural network model. The technical problem that image information is separated from the knowledge reasoning process in an existing knowledge graph fusion ultrasonic focus diagnosis method is solved, the visual perception ability of the image and the symbol reasoning ability of the knowledge graph are combined, and therefore an intelligent diagnosis system which is accurate in prediction and clear in explanation is provided.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence-assisted diagnosis technology in medical imaging, specifically to a diagnostic system, storage medium, and electronic device for interpreting breast ultrasound lesions. Background Technology

[0002] In recent years, the rapid development of deep learning technology has brought revolutionary changes to medical image analysis. Models such as convolutional neural networks have demonstrated superior performance in classifying benign and malignant breast ultrasound images, leading to the development of numerous computer-aided diagnostic systems. These systems can automatically extract high-dimensional features from raw images and perform end-to-end predictions, greatly improving diagnostic efficiency. However, their "black box" nature also constitutes a core obstacle to clinical implementation; doctors cannot understand why the model makes a certain diagnosis, i.e., there is a lack of interpretability. In the field of life-or-death medical decision-making, the transparency and reliability of models are equally important. An AI system that cannot explain its decision-making process, no matter how high its accuracy, will struggle to gain the trust of doctors and will be unable to achieve true human-machine collaboration.

[0003] To address these issues, current research primarily follows two core paths. The first is mimicking clinical decision-making processes, and the second is integrating domain knowledge. Taking the second path as an example, human doctors' diagnostic decisions are not only based on the image itself but also rely heavily on the vast medical knowledge stored in their minds. Therefore, integrating structured medical knowledge (such as diseases, symptoms, signs, and their interrelationships) into deep learning models is another important direction for improving their interpretability and generalization ability. Knowledge graphs, as a powerful knowledge representation tool, can clearly express the semantic relationships between medical concepts in the form of "entity-relationship-entity" triples. For example, spiky edge features are strongly correlated with a high probability of malignancy (spiky edge → indicator → high probability of malignancy). Mapping image features onto knowledge graphs and using the graphs for reasoning provides explanations that conform to medical logic.

[0004] However, despite the significant progress made by the above methods, there are still key bottlenecks. In existing knowledge graph fusion ultrasound lesion diagnosis methods, image information and knowledge reasoning processes are separated, failing to achieve true synergy. Summary of the Invention

[0005] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a breast ultrasound lesion interpretable diagnostic system, storage medium, and electronic device, which solves the technical problem of separating image information from the knowledge reasoning process in existing knowledge graph fusion ultrasound lesion diagnostic methods.

[0006] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides an interpretable diagnostic system for breast ultrasound lesions, comprising: a hierarchical feature decoupling module, a knowledge graph construction module, and a feature-knowledge fusion module; in, The hierarchical feature decoupling module is used to process the input breast ultrasound image according to multiple different deep neural network models. The predefined BI-RADS attributes are hierarchically categorized to obtain a list containing A structured vector of specific attribute labels; The knowledge graph construction module is used to construct a knowledge graph for breast ultrasound diagnosis, which includes clinical expert rules. The feature-knowledge fusion module is used to integrate features containing... The structured vectors of specific attribute labels are mapped to the corresponding imaging feature nodes in the knowledge graph to obtain the initial state. The initial state is then inferred through a pre-trained graph neural network model, and the inference results are calibrated and interpreted through the clinical expert rules to generate prediction results.

[0007] Preferably, the hierarchical classification includes coarse-grained classification and fine-grained classification; Among them, coarse particle classification refers to classification according to A predefined BI-RADS attribute classifies features in breast ultrasound images, including boundary, contour, echo, calcification, and blood flow signal. Fine-grained classification classifies attributes based on their specific morphological characteristics, including: The specific morphological manifestations of a boundary include clear boundaries, relatively clear boundaries, unclear boundaries, and blurred boundaries. The specific morphological manifestations of the outline include regular outline, partially regular outline, angular outline edge, lobed outline edge, irregular outline, irregular outline, and spiky outline edge. The specific morphological manifestations of echoes include mixed echoes, mixed solid echoes, mixed cystic-solid echoes, heterogeneous echoes, hypoechoes, hypoechoes with posterior attenuation, and very hypoechoes. The specific morphological manifestations of calcification include no obvious calcification, no calcification, coarse calcification, punctate calcification, and clustered punctate echoes; The specific morphological manifestations of blood flow signals include no obvious blood flow, peripheral linear blood flow, linear blood flow, and abundant blood flow.

[0008] Preferably, the knowledge graph includes a concept layer and a data layer; The conceptual layer defines the ontology framework of the knowledge graph, including expert knowledge, diagnostic reports, case reports, patient cases, and breast ultrasound guidelines. The data layer is an instance-filling layer for the knowledge graph, including a set of rules composed of clinical expert rules and structured knowledge graph instances.

[0009] Preferably, the will include The structured vectors of specific attribute labels are mapped to the corresponding image feature nodes in the knowledge graph to obtain the initial state, including: Structured attribute label vectors Mapped to knowledge graph On the corresponding image feature nodes: After mapping is complete, the activated nodes Obtain the initial state vector: The state value of inactive nodes is set to: .

[0010] Preferably, during the training process, the pre-trained graph neural network model defines a set of strongly correlated rules based on clinical expert rules extracted from the medical knowledge graph. Weak correlation rule set Mutual exclusion rule set A structured constraint loss is applied to samples that perfectly match the rule premises, where the structured constraint loss... The calculation formula is as follows: in, , The weighting coefficients for strong correlations. The weighting coefficients for weak correlations. The penalty coefficient for mutual exclusion is... The graph neural network model has parameters The following feature combination The reasoning result is The probability of; , It is a set of rules.

[0011] Preferably, the graph neural network model includes a two-layer graph convolutional network. In each layer, nodes update their own state by aggregating neighbor information. Layer nodes status The update rules are as follows: in, Represents a node The neighborhood group, It is the first The learnable parameter matrix of the layer, It is the ReLU activation function. Represents a node The neighborhood group, Indicates the first Layer nodes The state.

[0012] Preferably, the loss function during the training process of the graph neural network model includes: in, For the total loss, For cross-entropy loss, These are the balancing parameters.

[0013] Preferably, the step of calibrating and interpreting the inference results using the clinical expert rules to generate prediction results includes: Inspect the input structured attribute label vector Does it match any strongly related rules? If a matching rule exists... Make The result of direct reasoning As a final diagnostic result, this process is formalized as follows: in, Indicates possible diagnostic categories; For structured attribute label vectors that do not match any strong rules Graph neural network models, on the other hand, make comprehensive judgments based on the aggregated information from graph neural networks.

[0014] In a second aspect, the present invention provides a storage medium storing a computer program for interpretable diagnosis of breast ultrasound lesions, wherein the computer program causes a computer to control the breast ultrasound lesion interpretable diagnosis system as described above to perform interpretable diagnosis of breast ultrasound lesions.

[0015] Thirdly, the present invention provides an electronic device, comprising: One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, such that the one or more processors control the breast ultrasound lesion interpretable diagnostic system as described above to perform breast ultrasound lesion interpretable diagnosis.

[0016] (III) Beneficial Effects This invention provides a breast ultrasound lesion interpretation diagnostic system, storage medium, and electronic device, which, compared with existing technologies, have the following advantages: This invention maps image attribute feature labels extracted by a hierarchical feature decoupling module to corresponding nodes in a knowledge graph. Subsequently, the graph neural network model performs multi-hop reasoning on the graph structure through a message passing mechanism. It not only aggregates information from relevant attribute nodes but also utilizes clinical expert rules extracted from the knowledge graph to calibrate and interpret the reasoning results of the graph neural network model, ensuring that each prediction by the model is accompanied by a clear and traceable reasoning path. This invention solves the technical problem of separating image information from the knowledge reasoning process in existing knowledge graph-integrated ultrasound lesion diagnosis methods, combining the visual perception capabilities of images with the symbolic reasoning capabilities of knowledge graphs to provide an intelligent diagnostic system with accurate prediction and clear interpretation. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the overall architecture of the diagnostic system for breast ultrasound lesions in Example 1. Figure 2 This is a schematic diagram of the hierarchical feature decoupling process in the hierarchical feature decoupling module; Figure 3 A schematic diagram illustrating the knowledge graph construction process in the knowledge graph construction module; Figure 4 This is a schematic diagram of instantiating a sample knowledge graph. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This application provides a breast ultrasound lesion interpretable diagnostic system, storage medium, and electronic device, which solves the technical problem of separating image information from knowledge reasoning process in existing knowledge graph fusion ultrasound lesion diagnostic methods. It combines the visual perception capability of images with the symbolic reasoning capability of knowledge graphs, thereby providing an intelligent diagnostic system that can accurately predict and clearly interpret lesions.

[0021] It should be noted that the breast ultrasound lesion interpretation diagnostic system of the present invention can be extended to thyroid ultrasound image diagnostic system, liver ultrasound image diagnostic system, etc.

[0022] The technical solution in this application is to solve the above-mentioned technical problems, and the general idea is as follows: Current research on deep learning technology in medical diagnosis is evolving along two main paths. The first path mimics the clinical decision-making process. Its core idea is to make the model's reasoning process explicit, aligning it with the diagnostic thinking of doctors. Specifically, the model no longer directly predicts the final BI-RADS classification, but first predicts a series of intermediate attributes directly related to clinical guidelines (such as whether the edges are jagged, whether the internal echoes are uneven, etc.). Then, based on the predictions of these attributes, the final diagnosis is derived through an interpretable fusion mechanism (such as a rule engine or evidence-based reasoning). This method breaks down the model's "thinking" process into steps familiar to doctors, thus greatly improving its transparency and credibility. The second path integrates domain knowledge. Human doctors' diagnostic decisions are not only based on the image itself, but also rely on the vast medical knowledge base stored in their minds. Therefore, integrating structured medical knowledge (such as diseases, symptoms, signs, and their interrelationships) into deep learning models is another important direction for improving their interpretability and generalization ability. Knowledge graphs, as a powerful knowledge representation tool, can clearly express the semantic relationships between medical concepts in the form of "entity-relationship-entity" triples. For example, spiky edge features are strongly correlated with a high probability of malignancy (spiky edge → indicator → high probability of malignancy). By mapping image features onto knowledge graphs and using the graphs for reasoning, explanations that conform to medical logic can be provided.

[0023] However, despite the significant progress made by the aforementioned methods, key bottlenecks remain. On the one hand, while attribute-based prediction methods are transparent in process, their attribute fusion often relies on simple linear weighting, failing to fully capture the complex, non-linear conceptual dependencies in clinical knowledge. On the other hand, most existing knowledge graph fusion methods treat image features and knowledge graphs as two independent modules, resulting in the separation of image information from the knowledge reasoning process, thus failing to achieve true synergy.

[0024] To address the aforementioned problems, this invention provides a breast ultrasound lesion interpretable diagnostic system, storage medium, and electronic device based on hierarchical feature decoupling and knowledge graph-enhanced graph neural networks. This system proposes an innovative framework for "ultrasound interpretable diagnosis based on hierarchical decoupling classification and knowledge graph-enhanced graph neural networks." Its main contributions are: 1. This invention proposes a parallel and hierarchical multi-branch network structure for independently modeling five key attributes (boundary, contour, echo, calcification, and blood flow signal) in breast ultrasound (BUS) images that are closely related to BI-RADS category and diagnostic assessment.

[0025] 2. This invention introduces a Graph Neural Network (GNN) to construct a structured medical knowledge graph from clinical reports, diagnostic guidelines, and expert knowledge. The nodes of this graph represent specific BI-RADS attribute descriptions and the final diagnostic category, while the edges represent the medical semantic relationships between them (such as strong correlation, weak correlation, mutual exclusion, etc.).

[0026] 3. This embodiment of the invention designs a feature-knowledge fusion module, which maps the image attribute feature labels extracted by the hierarchical feature decoupling module to the corresponding nodes in the knowledge graph. Subsequently, the GNN performs multi-hop reasoning on the graph structure through a message passing mechanism. It can not only aggregate information from relevant attribute nodes, but also utilize clinical expert rules extracted from the knowledge graph to calibrate and interpret the reasoning results of the graph neural network model, so that each prediction of the graph neural network model is accompanied by a clear and traceable reasoning path.

[0027] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0028] Example 1: This invention provides an interpretable diagnostic system for breast ultrasound lesions, comprising: a hierarchical feature decoupling module, a knowledge graph construction module, and a feature-knowledge fusion module; The hierarchical feature decoupling module is used to process the input breast ultrasound image according to multiple different deep neural network models. The predefined BI-RADS attributes are hierarchically categorized to obtain a list containing A structured vector of specific attribute labels; The knowledge graph construction module is used to build a knowledge graph for breast ultrasound diagnosis and extract clinical expert rules based on the knowledge graph; The feature-knowledge fusion module is used to combine features and knowledge. The structured vectors of specific attribute labels are mapped to the corresponding radiological feature nodes in the knowledge graph to obtain the initial state. The initial state is then inferred through a pre-trained graph neural network model, and the inference results are calibrated and interpreted through clinical expert rules to generate prediction results.

[0029] This invention maps image attribute feature labels extracted by the hierarchical feature decoupling module to corresponding nodes in the knowledge graph. Subsequently, the graph neural network model performs multi-hop reasoning on the graph structure through a message passing mechanism. It not only aggregates information from relevant attribute nodes but also utilizes clinical expert rules extracted from the knowledge graph to calibrate and interpret the reasoning results of the graph neural network model, ensuring that each prediction by the model is accompanied by a clear and traceable reasoning path. This invention solves the technical problem of separating image information from the knowledge reasoning process in existing knowledge graph-integrated ultrasound lesion diagnosis methods, combining the visual perception capabilities of images with the symbolic reasoning capabilities of knowledge graphs to provide an intelligent diagnostic system with accurate prediction and clear interpretation.

[0030] The overall architecture flowchart of the breast ultrasound lesion interpretation and diagnostic system of this invention is as follows: Figure 1 As shown.

[0031] The following is a detailed description of each module in the system: Hierarchical feature decoupling module: This module includes coarse-grained classification units and fine-grained classification units. For example... Figure 2 As shown.

[0032] For the input breast ultrasound image, the system processes it in parallel. (In this embodiment) ) predefined BI-RADS attributes (in Each attribute (boundary, contour, echo, calcification, and blood flow signal) is processed independently. The processing of each attribute follows a hierarchical classification logic from coarse to fine: first, the coarse-grained category of the overall tendency of the attribute is determined (e.g., the calcification attribute is divided into two categories: "no calcification" and "with calcification"); then, based on this tendency, the corresponding fine-grained classification model is called to identify its specific morphological manifestation (e.g., the specific label of the boundary attribute may be "clear boundary" or "blurred boundary"). This process employs a multi-model weighted ensemble strategy. For each hierarchical classification task (coarse-grained and fine-grained), multiple different deep neural network models are trained (in this embodiment, 11 different deep neural network models are trained, specifically including ConvNeXt, DenseNet169, EfficientNetV2, GoogLeNet, MaxViT, MobileNetV3, RegNetY, ResNet-18, NextViT, SwinV2, and VGG-16; it should be noted that the selection of deep network models is based on the characteristics of ultrasound images). Weights are assigned according to the accuracy of each model on the validation set, and the prediction results are weighted and ensembled to generate a structured vector containing five specific attribute labels for the input image. The following is a detailed description: With calcification properties For example, we can treat "no calcification" and "calcification" in this attribute as two major categories in the coarse-grained classification. (in Corresponding to different coarse-grained categories), the specific coarse-grained category of calcification attribute is determined through a multi-model weighted ensemble strategy. The fine-grained category is based on the specific lesion attribute features in the two coarse-grained categories. Then, the corresponding blood flow fine-grained classification model is called, and the specific morphological manifestations are identified through a multi-model weighted ensemble strategy. (For example, the fine-grained categories for the attribute of calcification include no obvious calcification, no calcification, coarse calcification, punctate calcification, clustered punctate echoes, etc.; for the boundary, there are clear boundary, relatively clear boundary, unclear boundary, blurred boundary, etc.; for the contour, there are regular contour, partially regular contour, angular contour edge, lobulated contour edge, irregular contour, irregular contour, spiky contour edge, etc.; for the echo, there are mixed echo, mixed solid echo, mixed cystic solid echo, hypoechoic mass in duct, heterogeneous echo, hypoechoic, hypoechoic with posterior attenuation, extremely hypoechoic, etc.; for the blood flow signal, there is no obvious blood flow, peripheral strip blood flow, strip blood flow, abundant blood flow, etc.). This design is based on the fact that the specific feature categories of each attribute belong to different BI-RADS levels, and also takes into account the overall class distribution of the dataset. This coarse-grained class partitioning method incorporates expert knowledge and alleviates the problem of class imbalance in the dataset, thereby improving the model's prediction accuracy.

[0033] To enhance the robustness and generalization ability of the model, this embodiment of the invention does not employ a single network architecture. Instead, 11 different deep neural network models are trained for each level of classification task (coarse-grained and fine-grained). The selection of these network models is based on the characteristics of five attributes of breast ultrasound images. When analyzing morphological attributes (boundaries, contours), the network is required to possess strong capabilities in local edge sensitivity, multi-scale context modeling, and moderate long-range dependency to effectively suppress inherent speckle noise in ultrasound images and improve the stability and continuity of edge recognition. When analyzing acoustic attributes (echoes, calcifications), the focus is on echo patterns (such as low echoes and mixed echoes) and bright small targets (such as microcalcifications), requiring consideration of local texture modeling, multi-scale statistical characteristics, and high-resolution characterization of minute structures. For blood flow attribute analysis, the spatial distribution and relative positional relationships of the perfusion area are emphasized, placing higher demands on the model's spatial attention mechanism and cross-regional dependency modeling capabilities. To fully consider the characteristics of the above three types of attributes, the aforementioned 11 network models were selected. These models represent classic architectures in the current field of deep learning image classification and have demonstrated their effectiveness and stability in multiple tasks. By using these diverse models, the embodiments of the present invention can ensure complementarity across different network architectures and improve the model's performance on breast ultrasound images.

[0034] The specific process of hierarchical classification is as follows: Coarse-grained classification: Given an input image Each coarse-grained model first predicts the coarse-grained class probability of each attribute of the image. ,in Corresponding to different coarse-grained categories, among which These correspond to 11 different deep neural network models: For images Any attribute For specific coarse-grained categories, this embodiment of the invention employs a weighted integration strategy. First, based on the accuracy of each model on the coarse-grained classification validation set... Assign weights to it : Then, the predicted probabilities of all models for each coarse-grained category are weighted and summed, and normalized using the Softmax function. The category with the highest probability is selected as the coarse-grained classification prediction result for that attribute.

[0035] Fine-grained classification: Based on the results of coarse-grained classification, the system dynamically selects the corresponding fine-grained model for specific shape recognition. Each model then predicts the fine-grained category probability of each attribute of the image. ,in Corresponding to different fine-grained categories: And continue to use a weighted ensemble strategy, based on the accuracy of each model on the fine-grained classification validation set. Assign weights to it : Then, the predicted probabilities of all models for each fine-grained category are weighted and summed, and normalized using the Softmax function. The category with the highest probability is selected as the fine-grained classification prediction result for that attribute.

[0036] in, Represents attributes The For a specific tag, for example, for the boundary attribute, For "clear boundaries", Because the "border is still clear", The boundary is unclear. "Unclear boundaries" The term is "blurred boundary".

[0037] This hierarchical decoupling design effectively alleviates the problems of feature coupling and interference between different attributes, as well as class imbalance, allowing the dedicated feature learning path for each attribute to be optimized independently, thereby improving the model's ability to model specific semantic factors. Through this process, the embodiment of the invention uses a breast ultrasound image as input. A high-quality, interpretable structured input structured attribute label vector was generated. This provides a reliable foundation for knowledge reasoning in the next module.

[0038] Knowledge graph construction module: A knowledge graph for breast ultrasound diagnosis is constructed using ultrasound reports, pathological data, and patient case information. The knowledge graph can be formally defined as: in, Represents a set of nodes. Denotes the set of edges. A collection of node types It is a set of relation types.

[0039] In light of the unique characteristics of breast ultrasound diagnosis, four core node types are defined: (1) Imaging feature nodes It includes all possible values ​​of five major categories of BI-RADS attributes: boundary, shape, echo, calcification, and blood flow, such as "blurred boundary", "irregular shape", and "punctate strong echo".

[0040] (2) Sample nodes Each breast ultrasound case corresponds to a node.

[0041] (3) Diagnostic nodes for benign and malignant diseases : Binary diagnostic results Indicates benign (0), Indicates malignancy (1).

[0042] (4) BI-RADS classification nodes Five-part differential diagnosis results Each corresponds to a different risk level.

[0043] Node relationship types include: (1) Sample-feature association: Sample-boundary feature association Sample-morphological association , Sample-Echo Correlation Sample-calcification association Sample-blood flow correlation .

[0044] (2) Strong correlation between features and diagnosis: Strong correlation between features and benign / malignant diagnosis Features—strong correlation with BI-RADS classification .

[0045] (3) Weak correlation between features and diagnosis: weak correlation between features and benign / malignant diagnosis Features—weak correlation with BI-RADS classification .

[0046] (4) Feature-feature mutual exclusion: .

[0047] In this embodiment of the invention, the knowledge graph includes a concept layer and a data layer, and its construction process is as follows: Figure 3As shown. Among them, the conceptual layer of the knowledge graph defines the ontology framework of the graph. Its design is mainly based on: (1) the American College of Radiology (ACR) BI-RADS® Fifth Edition guidelines, which provides a standardized dictionary for the attribute description of breast ultrasound images and standardized terminology for clinical ultrasound examination reports (it should be noted that the above ARC guidelines are only a specific example, and other guidelines, such as the European EUSOBI breast ultrasound guidelines, can also be used in the specific implementation process); (2) standardized terminology for clinical pathology diagnosis reports; (3) key diagnostic and treatment fields in inpatient cases. Under the guidance of ultrasound physicians, the embodiment of the present invention defines the ontology structure. The key innovation is to model benign and malignant diagnosis results and BI-RADS classification as two independent but interrelated diagnostic dimensions, which is more in line with clinical practice.

[0048] The data layer is the instance-population layer of the knowledge graph, and its data comes from retrospectively collected clinical data, including ultrasound examination reports and corresponding pathology reports. This embodiment of the invention performs the following steps: (1) Knowledge extraction and alignment: To ensure that the terminology of the knowledge graph is consistent with the definition of the concept layer, the original information in the patient case information is extracted in a structured manner to form a valid data table of patient cases, and then standardized and coded and integrated into the patient knowledge graph.

[0049] (2) Knowledge fusion extraction rules: Through the analysis of the knowledge graph, the imaging findings, pathological results and clinical decisions are aligned and associated to extract relevant clinical expert rules: 1. Benign and malignant diagnosis rules: For example, "blurred boundary ∧ punctate strong echo ∧ irregular → malignant". 2. BI-RADS classification rules: "blurred boundary ∧ punctate strong echo ∧ irregular → BI-RADS 4C".

[0050] (3) Knowledge Graph Instantiation: Import all processed triplet data (e.g., <blurred boundaries ∧ strong echoes ∧ irregular, strongly correlated, BI-RADS 4C>) into the Neo4j graph database to construct the final knowledge graph, such as Figure 4 As shown, this knowledge graph provides a structured prior knowledge base for graph neural network model reasoning.

[0051] Feature-Knowledge Fusion Module: Feature-knowledge fusion and reasoning maps the structured vectors output by the hierarchical feature decoupling module to the corresponding attribute nodes in the knowledge graph. This initial state is then input into a pre-trained graph neural network (GNN) model for reasoning. The reasoning results are calibrated and interpreted using clinical expert rules to generate predictions. The specific process is as follows: Feature-knowledge mapping: The structured attribute label vector of the input image is mapped to the corresponding radiographic feature nodes in the knowledge graph. Activated nodes obtain an initial state vector, and the state value of inactive nodes is set to zero, thus obtaining the initial state.

[0052] After processing by the hierarchical feature decoupling module, the input image It is transformed into a structured attribute label vector ,like Map this vector to a knowledge graph. On the corresponding image feature nodes: .

[0053] After mapping is complete, the activated nodes Obtain the initial state vector: The state value of inactive nodes is set to: This mapping process transforms visual perception results into semantic entities in a knowledge graph, laying the foundation for subsequent reasoning.

[0054] The inference phase of a graph neural network model: The initial state is input into a pre-trained graph neural network model to generate inference results.

[0055] The training process of a pre-trained graph neural network model is explained below: The pre-trained graph neural network model in this embodiment of the invention adopts a two-layer graph convolutional network (GCN). In each layer, nodes update their own state by aggregating neighbor information. The key innovation is to integrate clinical expert rules into the training process. Based on the clinical expert rules extracted from the medical knowledge graph, a set of "strong correlation", "weak correlation" and "mutually exclusive" rules is defined, and a structured constraint loss is applied to samples that fully match the premise of the rules. In the training process of the graph neural network model, a breast ultrasound text dataset is used for both training and testing. To enhance the model's reasoning ability, this embodiment of the invention uses a GNN as a knowledge graph-enhanced reasoning engine, which consists of a two-layer graph convolutional network (GCN). In each layer, nodes update their own state by aggregating information from their neighbors. Layer nodes The status update rules are as follows: in, Represents a node The neighborhood group, It is the first The learnable parameter matrix of the layer, Represents a node The neighborhood group, Indicates the first Layer nodes state, The ReLU activation function is used. The key innovation of this invention lies in incorporating clinical expert rules into the training process. This is based on a set of rules extracted from a medical knowledge graph. : This invention defines a set of rules for "strong correlation", "weak correlation", and "mutual exclusion". For satisfying For samples that perfectly match the rule premises, a structured constraint loss is applied: in These are the weighting coefficients for different relation types. The penalty coefficient for mutual exclusion is... The graph neural network model has parameters The following feature combination The reasoning result is The probability of . The overall loss function is: in, For conventional cross-entropy loss, These are the balancing parameters.

[0056] GNN aggregates information through multi-hop message passing to generate inference results for BI-RADS classification and benign / malignant diagnosis tasks.

[0057] The inference results are calibrated and interpreted using clinical expert rules to generate prediction results. Specifically, a rule calibration mechanism is implemented during the inference phase of the graph neural network model, as follows: During the inference phase, the Knowledge Graph Enhanced Graph Neural Network (KGGNN) model aggregates information through multi-hop message passing, ultimately generating inference results for BI-RADS grading and benign / malignant diagnosis tasks. To ensure clinical reliability, a rule calibration mechanism was designed: first, the input structured attribute label vector is checked. Does it match any "strongly related" rules? If a matching rule exists... Make Then output directly. As a final diagnostic result, this process can be formalized as follows: in, Indicates the possible diagnostic categories, depending on the diagnostic task: in the BI-RADS grading task, This represents a specific category within the five-category classification, corresponding to different levels of malignancy risk. In benign / malignant diagnostic tasks, This represents the diagnostic result of a binary classification. This design ensures that when the input features perfectly match clinical expert consensus, the model strictly adheres to clinical expert rules, avoiding predictions that might violate clinical common sense in data-driven methods. For structured attribute label vectors that do not match any strong rules... The model then makes a comprehensive judgment based on the aggregated information from a graph neural network. This framework implements rich interpretability mechanisms, providing doctors with complete visualization of the reasoning path.

[0058] Example 2: This invention provides a storage medium storing a computer program for interpretable diagnosis of breast ultrasound lesions, wherein the computer program causes a computer to control the interpretable diagnosis system for breast ultrasound lesions as described in Embodiment 1 to perform interpretable diagnosis of breast ultrasound lesions.

[0059] Example 3: This invention provides an electronic device, comprising: One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, such that the one or more processors control the breast ultrasound lesion interpretable diagnostic system as described in Example 1 to perform breast ultrasound lesion interpretable diagnosis.

[0060] It is understood that the storage medium and electronic device provided in the embodiments of the present invention correspond to the breast ultrasound lesion interpretable diagnostic system provided in the embodiments of the present invention. The explanations, examples and beneficial effects of the relevant contents can be referred to the corresponding parts of the breast ultrasound lesion interpretable diagnostic system, and will not be repeated here.

[0061] In summary, compared with existing technologies, it has the following beneficial effects: 1. In this embodiment of the invention, the image attribute feature labels extracted by the hierarchical feature decoupling module are mapped to the corresponding nodes in the knowledge graph. Subsequently, the graph neural network model performs multi-hop reasoning on the graph structure through a message passing mechanism. It not only aggregates information from relevant attribute nodes but also utilizes clinical expert rules extracted from the knowledge graph to calibrate and interpret the reasoning results of the graph neural network model, ensuring that each prediction of the graph neural network model is accompanied by a clear and traceable reasoning path. This embodiment of the invention solves the technical problem of separating image information from the knowledge reasoning process in existing knowledge graph fusion ultrasound lesion diagnosis methods, achieving a combination of the visual perception capability of images and the symbolic reasoning capability of knowledge graphs, thereby providing an intelligent diagnostic system with accurate prediction and clear interpretation.

[0062] 2. The hierarchical decoupled classification architecture proposed in this embodiment decomposes the complex attribute recognition task into two stages: coarse-grained and fine-grained. It avoids gradient conflicts and representation confusion caused by multiple tasks sharing the backbone network through independent feature learning paths.

[0063] 3. This invention innovatively integrates structured medical knowledge into the training process of graph neural networks. Through the "strong correlation rule-guided" training strategy, the model can strictly follow medical consensus when faced with clear clinical indications.

[0064] 4. The reasoning paths generated in this embodiment of the invention (such as "blurred boundaries ∧ punctate strong echoes ∧ irregularities → high indication BI-RADS 4C") provide doctors with intuitive decision-making basis, significantly improving clinical credibility. This dual-drive model of "data-driven + knowledge-guided" not only improves diagnostic accuracy but also provides explanations that conform to clinical logic, providing a reliable technical path for the clinical implementation of AI-assisted diagnostic systems.

[0065] 5. In this embodiment of the invention, 11 different deep neural network models are trained for each hierarchical classification task, and weighted ensemble is performed based on the performance of the validation set. This effectively improves the model's adaptability to different ultrasound equipment and imaging parameters, and enhances the stability and reliability of the system in real clinical environments.

[0066] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0067] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A diagnostic system for interpreting breast ultrasound lesions, characterized in that, include: The module consists of a hierarchical feature decoupling module, a knowledge graph construction module, and a feature-knowledge fusion module. in, The hierarchical feature decoupling module is used to process the input breast ultrasound image according to multiple different deep neural network models. The predefined BI-RADS attributes are hierarchically categorized to obtain a list containing A structured vector of specific attribute labels; The knowledge graph construction module is used to construct a knowledge graph for breast ultrasound diagnosis, which includes clinical expert rules. The feature-knowledge fusion module is used to integrate features containing... The structured vectors of specific attribute labels are mapped to the corresponding imaging feature nodes in the knowledge graph to obtain the initial state. The initial state is then inferred through a pre-trained graph neural network model, and the inference results are calibrated and interpreted through the clinical expert rules to generate prediction results.

2. The breast ultrasound lesion interpretation diagnostic system as described in claim 1, characterized in that, The hierarchical classification includes coarse-grained classification and fine-grained classification; Among them, coarse particle classification refers to classification according to A predefined BI-RADS attribute classifies features in breast ultrasound images, including boundary, contour, echo, calcification, and blood flow signal. Fine-grained classification classifies attributes based on their specific morphological characteristics, including: The specific morphological manifestations of a boundary include clear boundaries, relatively clear boundaries, unclear boundaries, and blurred boundaries. The specific morphological manifestations of the outline include regular outline, partially regular outline, angular outline edge, lobed outline edge, irregular outline, irregular outline, and spiky outline edge. The specific morphological manifestations of echoes include mixed echoes, mixed solid echoes, mixed cystic-solid echoes, heterogeneous echoes, hypoechoes, hypoechoes with posterior attenuation, and very hypoechoes. The specific morphological manifestations of calcification include no obvious calcification, no calcification, coarse calcification, punctate calcification, and clustered punctate echoes; The specific morphological manifestations of blood flow signals include no obvious blood flow, peripheral linear blood flow, linear blood flow, and abundant blood flow.

3. The breast ultrasound lesion interpretation diagnostic system as described in claim 1, characterized in that, The knowledge graph includes a concept layer and a data layer; The conceptual layer defines the ontology framework of the knowledge graph, including expert knowledge, diagnostic reports, case reports, patient cases, and breast ultrasound guidelines. The data layer is an instance-filling layer for the knowledge graph, including a set of rules composed of clinical expert rules and structured knowledge graph instances.

4. The breast ultrasound lesion interpretation diagnostic system as described in claim 1, characterized in that, The term will include The structured vectors of specific attribute labels are mapped to the corresponding image feature nodes in the knowledge graph to obtain the initial state, including: Structured attribute label vectors Mapped to knowledge graph On the corresponding image feature nodes: After mapping is complete, the activated nodes Obtain the initial state vector: The state value of inactive nodes is set to: .

5. The breast ultrasound lesion interpretation diagnostic system as described in any one of claims 1 to 4, characterized in that, The pre-trained graph neural network model defines a set of strongly correlated rules during training based on clinical expert rules extracted from the medical knowledge graph. Weak correlation rule set Mutual exclusion rule set A structured constraint loss is applied to samples that perfectly match the rule premises, where the structured constraint loss... The calculation formula is as follows: in, , The weighting coefficients for strong correlations. The weighting coefficients for weak correlations. The penalty coefficient for mutual exclusion is... The graph neural network model has parameters The following feature combination The reasoning result is The probability of; , It is a set of rules.

6. The breast ultrasound lesion interpretation diagnostic system as described in claim 5, characterized in that, The graph neural network model comprises two layers of graph convolutional networks. In each layer, nodes update their own state by aggregating information from their neighbors. Layer nodes status The update rules are as follows: in, Represents a node The neighborhood group, It is the first The learnable parameter matrix of the layer, It is the ReLU activation function. Represents a node The neighborhood group, Indicates the first Layer nodes The state.

7. The breast ultrasound lesion interpretation diagnostic system as described in claim 5, characterized in that, The loss function during the training process of the graph neural network model includes: in, For the total loss, For cross-entropy loss, These are the balancing parameters.

8. The breast ultrasound lesion interpretation diagnostic system as described in claim 5, characterized in that, The process of calibrating and interpreting the inference results using the clinical expert rules to generate prediction results includes: Inspect the input structured attribute label vector Does it match any strongly related rules? If a matching rule exists... Make The result of direct reasoning As a final diagnostic result, this process is formalized as follows: in, Indicates possible diagnostic categories; For structured attribute label vectors that do not match any strong rules Graph neural network models, on the other hand, make comprehensive judgments based on the aggregated information from graph neural networks.

9. A storage medium, characterized in that, It stores a computer program for interpretable diagnosis of breast ultrasound lesions, wherein the computer program causes a computer to control the interpretable diagnosis system for breast ultrasound lesions as described in any one of claims 1 to 8 to perform interpretable diagnosis of breast ultrasound lesions.

10. An electronic device, characterized in that, include: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, such that the one or more processors control the breast ultrasound lesion interpretable diagnostic system as described in any one of claims 1 to 8 to perform breast ultrasound lesion interpretable diagnosis.

Images