A method and system for generating a structured report of a medical image

Abstract

The application provides a medical image structured report generation method and system, the method comprising: extracting the base feature map of standardized image data by using a multi-scale feature extraction network, and performing multi-level down-sampling to obtain a multi-scale feature map sequence, and calculating the corresponding adaptive fusion weight based on the semantic richness of each scale feature map, and performing weighted fusion on the multi-scale feature map sequence based on the adaptive fusion weight to obtain a target semantic feature vector; performing semantic deduction on the target semantic feature vector based on a medical knowledge graph to generate image semantic description information, matching a target structured report template based on the image semantic description information, and generating a medical image structured report based on an adaptive template filling strategy. The application can fully capture the multi-scale semantic information of the lesion area in the medical image, and simultaneously realizes the dynamic adaptation of the report template and the image semantic content, thereby improving the consistency and clinical practicability of the report and the image semantic content.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method and system for generating structured medical image reports. Background Technology

[0002] In the field of medical image processing technology, with the widespread application of medical imaging equipment (such as CT, MRI, DR, etc.), the amount of medical image data is growing explosively. Traditional medical image reports mainly rely on radiologists' manual interpretation of images and text descriptions, which not only consumes a lot of manpower and time, but also makes the quality of reports susceptible to the influence of physicians' experience level and subjective judgment, with risks such as inconsistent descriptions and omission of key information.

[0003] To address this, the industry has gradually introduced automated technologies to assist in report generation. However, existing technical solutions still have significant shortcomings: on the one hand, conventional image feature extraction methods are unable to fully capture the multi-scale semantic information of lesion areas in medical images, resulting in insufficient feature representation capabilities for minor lesions or complex structures; on the other hand, report generation methods based on fixed templates lack a deep understanding of the semantic content of images and adaptive matching capabilities, and cannot dynamically adjust the report structure according to actual image features, resulting in low consistency between the generated structured report and the semantic content of images, and poor clinical applicability. Summary of the Invention

[0004] This invention provides a method and system for generating structured medical image reports, which fully captures multi-scale semantic information of lesion areas in medical images, while dynamically adapting the report template to the semantic content of the images, thereby improving the consistency between the report and the semantic content of the images and their clinical applicability.

[0005] To solve the above problems, the present invention adopts the following technical solution: This invention provides a method for generating structured medical image reports, comprising: Acquire raw image data output from medical imaging equipment, preprocess the raw image data to obtain standardized image data; The basic feature map of the standardized image data is extracted using a multi-scale feature extraction network. The basic feature map is downsampled at multiple levels to obtain a multi-scale feature map sequence. The corresponding adaptive fusion weight is calculated based on the semantic richness of each scale feature map. The multi-scale feature map sequence is then weighted and fused based on the adaptive fusion weights to obtain the target semantic feature vector. Based on a medical knowledge graph, semantic inference is performed on the target semantic feature vector to generate image semantic description information; Based on the image semantic description information, a target structured report template is matched, and a medical image structured report is generated based on an adaptive template filling strategy.

[0006] Furthermore, before calculating the adaptive fusion weights corresponding to the semantic richness based on feature maps at each scale, the process further includes: Calculate the probability distribution of feature maps at each scale in the corresponding channel dimension, calculate the information entropy value of each spatial location based on the probability distribution, and perform global statistics on the information entropy values ​​of all spatial locations to obtain the cumulative information entropy value. Calculate the spatial gradient of the feature map at each scale in the corresponding horizontal and vertical directions, calculate the gradient magnitude based on each spatial gradient, and perform global statistics on all gradient magnitudes to obtain the edge gradient intensity. The semantic richness is calculated based on the cumulative information entropy value and edge gradient strength of the feature maps at each scale.

[0007] Preferably, the step of semantically deriving the target semantic feature vector based on a medical knowledge graph to generate image semantic description information includes: The target semantic feature vector is mapped to the initial node activation state in a medical knowledge graph, which includes anatomical structure nodes, pathological nodes, and clinical association edges. The pathological nodes refer to abnormal nodes that are observable in medical images and have pathological significance. The clinical association edges refer to clinically significant relationship edges that connect different nodes. Perform graph convolution propagation operation to propagate the node activation state along the clinical association edge and update the semantic confidence of each node; The attention mechanism is used to aggregate neighbor node information and generate image semantic description information containing contextual relationships.

[0008] Preferably, the graph convolution propagation operation includes: Calculate the edge weights of the clinically associated edges, which are determined based on the physiological connection strength and pathological co-occurrence frequency between anatomical structure nodes; Perform multi-layer graph convolution operations. Each layer of graph convolution operations includes a message passing phase and a state update phase. In the message passing phase, neighbor node features are propagated along the edge weights. In the state update phase, a gated recurrent unit is used to fuse the current node features with the received neighbor node information. After the final graph convolution operation, the semantic confidence of each pathological node is output.

[0009] Preferably, the step of matching the target structured report template based on the image semantic description information includes: Calculate the semantic similarity between the image semantic description information and each standard report template in the structured report template library. The structured report template library includes standard report templates for various image examination types. Each standard report template defines required fields, optional fields, and logical dependencies between fields. The standard report template with the highest semantic similarity is selected as the target structured report template.

[0010] Preferably, the generation of structured medical image reports based on the adaptive template filling strategy includes: Identify the set of fields to be populated in the target structured report template; For each field to be filled, retrieve the corresponding image descriptive sub-map based on the image semantic description information; When the confidence level of the image descriptor is higher than a preset first threshold, the field content is directly generated using the image descriptor. When the confidence level of the image descriptor is lower than the preset first threshold but higher than the preset second threshold, a field containing a confidence level identifier is generated. Based on the logical dependencies between fields, the logical consistency of the content of each field is verified, and a structured medical image report is generated.

[0011] Preferably, the step of verifying the logical consistency of the content of each field based on the logical dependency relationship between fields includes: Extract the field constraint rules defined in the target structured report template, including mutual exclusion, coexistence, and causal relationships; Construct a field content logic graph, where nodes represent field content and edges represent logical relationships between field contents; Detect whether there are any logical conflict paths in the field content logic graph that violate the constraints between fields; When a logical conflict path is detected, the field content with a confidence level higher than a preset value is retained, and the field content with a confidence level lower than the preset value is corrected based on the medical knowledge graph until all logical conflict paths are eliminated.

[0012] Furthermore, after generating the structured medical image report based on the adaptive template filling strategy, the process also includes: Receive feedback from physicians regarding corrections to the structured medical image report; The correction feedback information, along with the original image data, the target semantic feature vector, and the image semantic description information, are stored in the case database. The convolution parameters of the multi-scale feature extraction network, the node features of the medical knowledge graph, and the template configuration of the structured report template library are updated periodically based on the case database.

[0013] This invention also provides a system for generating structured medical image reports, comprising: The acquisition module is used to acquire the raw image data output by the medical imaging equipment, and to preprocess the raw image data to obtain standardized image data. The calculation module is used to extract the basic feature map of the standardized image data using a multi-scale feature extraction network, perform multi-level downsampling on the basic feature map to obtain a multi-scale feature map sequence, calculate the corresponding adaptive fusion weight based on the semantic richness of each scale feature map, and perform weighted fusion on the multi-scale feature map sequence based on each adaptive fusion weight to obtain the target semantic feature vector. The derivation module is used to perform semantic derivation on the target semantic feature vector based on the medical knowledge graph to generate image semantic description information; The generation module is used to match the target structured report template based on the image semantic description information, and generate a medical image structured report based on an adaptive template filling strategy.

[0014] Compared with the prior art, the technical solution of the present invention has at least the following advantages: This invention provides a method for generating structured medical image reports. It employs a multi-scale feature extraction network to extract deep features from standardized image data and introduces an adaptive fusion weighting mechanism based on semantic richness. This achieves weighted fusion of multi-scale feature map sequences, effectively enhancing the ability of the target semantic feature vector to represent minute lesions or complex structures in the lesion region and fully capturing multi-scale semantic information of the lesion region in medical images. Furthermore, by fusing a medical knowledge graph to perform semantic inference on the target semantic feature vector, the underlying visual features are transformed into image semantic description information conforming to clinical standards, improving the accuracy and interpretability of image content understanding. Further, based on the image semantic description information, a target structured report template is matched, and an adaptive template filling strategy is used to generate the final report. This achieves dynamic adaptation between the report template and the image semantic content, thereby improving the consistency between the report and the image semantic content and its clinical applicability. Attached Figure Description

[0015] Figure 1 This is a flowchart of an embodiment of a method for generating structured medical image reports according to the present invention; Figure 2 This is a flowchart of another embodiment of the method for generating structured medical image reports according to the present invention; Figure 3 This is a structural block diagram of an embodiment of the medical image structured report generation system of the present invention. Detailed Implementation

[0016] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0017] Please refer to Figure 1As shown, the present invention provides a method for generating structured medical image reports, comprising the following steps: S11. Obtain the raw image data output by the medical imaging equipment, and preprocess the raw image data to obtain standardized image data. S12. Use a multi-scale feature extraction network to extract the basic feature map of the standardized image data, perform multi-level downsampling on the basic feature map to obtain a multi-scale feature map sequence, calculate the corresponding adaptive fusion weight based on the semantic richness of each scale feature map, and perform weighted fusion on the multi-scale feature map sequence based on each adaptive fusion weight to obtain the target semantic feature vector. S13. Based on the medical knowledge graph, perform semantic inference on the target semantic feature vector to generate image semantic description information; S14. Match the target structured report template based on the image semantic description information, and generate a medical image structured report based on the adaptive template filling strategy.

[0018] First, raw image data is acquired from medical imaging equipment via a standard interface. This raw image data typically has equipment-specific data formats and grayscale distribution characteristics. Preprocessing operations include standardization processes such as grayscale normalization, noise suppression, and spatial resolution unification. These aim to eliminate data differences caused by different equipment and scanning parameters, ensuring that the image data meets the input requirements of deep learning networks while preserving the complete morphological information and contrast characteristics of the lesion area.

[0019] Secondly, a multi-scale feature extraction network is employed to perform deep feature learning on standardized image data. First, a convolutional neural network extracts basic feature maps containing low-level visual information such as edges and textures. Then, multi-level downsampling operations are performed on these basic feature maps to construct a multi-scale feature map sequence with different spatial resolutions. Shallow feature maps retain fine spatial localization information, while deep feature maps capture abstract semantic context information. Subsequently, an adaptive fusion mechanism is introduced: by quantifying the semantic richness of feature maps at each scale (such as information entropy and feature activation density), the adaptive fusion weights corresponding to each scale are dynamically calculated. This weighted fusion of the multi-scale feature map sequence allows the neural network to automatically adjust the contribution of features at different scales based on the image content, ultimately obtaining a target semantic feature vector that balances detail accuracy and semantic integrity.

[0020] Then, the target semantic feature vector is input into the medical knowledge graph for semantic derivation. The medical knowledge graph pre-integrates clinical professional knowledge in the field of medical imaging, such as anatomical structure knowledge, pathological sign definitions, and disease association rules. A graph neural network can be used to map the low-dimensional feature vector into a high-dimensional semantic description that conforms to medical logic, obtaining image semantic description information, including lesion localization description (anatomical location), sign feature description (morphology, density, edge), and disease association description (benign / malignant tendency, differential diagnosis suggestions).

[0021] Finally, structured reports are automatically generated based on image semantic description information. First, a structured report template library is established, with each template corresponding to different image examination types, organ locations, or disease categories. Using semantic similarity calculations or rule-based matching algorithms, the target structured report template with the highest fit to the current image semantic description information is selected from the library. Then, an adaptive template filling strategy is adopted: the fields to be filled in the template (such as lesion location, size, density, and diagnostic opinions) are parsed, and the corresponding entities in the image semantic description information are automatically mapped to the corresponding fields in the report template. For missing information or uncertain descriptions, reasonable inferences are made or the status to be supplemented is marked based on a medical knowledge graph, ultimately generating a complete and formatted medical image structured report.

[0022] This invention provides a method for generating structured medical image reports. It employs a multi-scale feature extraction network to extract deep features from standardized image data and introduces an adaptive fusion weighting mechanism based on semantic richness. This achieves weighted fusion of multi-scale feature map sequences, effectively enhancing the ability of the target semantic feature vector to represent minute lesions or complex structures in the lesion region and fully capturing multi-scale semantic information of the lesion region in medical images. Furthermore, by fusing a medical knowledge graph to perform semantic inference on the target semantic feature vector, the underlying visual features are transformed into image semantic description information conforming to clinical standards, improving the accuracy and interpretability of image content understanding. Further, based on the image semantic description information, a target structured report template is matched, and an adaptive template filling strategy is used to generate the final report. This achieves dynamic adaptation between the report template and the image semantic content, thereby improving the consistency between the report and the image semantic content and its clinical applicability.

[0023] In one embodiment, before calculating the adaptive fusion weights based on the semantic richness of feature maps at each scale, the method further includes: Calculate the probability distribution of feature maps at each scale in the corresponding channel dimension, calculate the information entropy value of each spatial location based on the probability distribution, and perform global statistics on the information entropy values ​​of all spatial locations to obtain the cumulative information entropy value. Calculate the spatial gradient of the feature map at each scale in the corresponding horizontal and vertical directions, calculate the gradient magnitude based on each spatial gradient, and perform global statistics on all gradient magnitudes to obtain the edge gradient intensity. The semantic richness is calculated based on the cumulative information entropy value and edge gradient strength of the feature maps at each scale.

[0024] Specifically, for each spatial location, the feature response values ​​of all channels at that location are extracted along the channel dimension. A normalization operation is then performed to transform these feature response values ​​into a probability distribution, representing the energy distribution state of that spatial location across different feature channels. Based on the obtained probability distribution, the information entropy value for each spatial location is calculated. The information entropy value reflects the dispersion and uncertainty of the feature response at that location. A higher information entropy value indicates that the location contains richer and more distinctive feature information. Subsequently, a global statistical operation is performed on the information entropy values ​​of all spatial locations, and the cumulative information entropy value is calculated through an aggregation function. This cumulative information entropy value quantifies the overall content complexity and feature diversity level of the feature map at the corresponding scale.

[0025] The gradient operator scans the feature map horizontally to capture the rate of change of the feature response in the horizontal dimension; similarly, it scans vertically to capture the rate of change in the vertical dimension. Based on the horizontal and vertical gradients, a synthesis operation is performed to obtain the gradient magnitude at each spatial location. The gradient magnitude characterizes the degree of abrupt change in the feature response at that location, corresponding to structural information such as edges, contours, or texture boundaries in the image. A global statistical operation is performed on the gradient magnitudes at all spatial locations to obtain the edge gradient intensity. This edge gradient intensity quantifies the richness and saliency level of boundary information in the corresponding scale feature map from a structural geometry perspective.

[0026] Cumulative information entropy reflects the inherent information diversity and uncertainty of feature maps, while edge gradient intensity reflects the structural boundary clarity and morphological discernibility of feature maps. Both characterize the semantic carrying capacity of feature maps from different dimensions. Through weighted fusion or normalized synthesis, cumulative information entropy and edge gradient intensity are integrated into a unified semantic richness index. This index can objectively evaluate the effectiveness and completeness of feature maps at various scales in representing the semantic content of images, providing a quantitative basis for the calculation of adaptive fusion weights.

[0027] In this embodiment, semantic richness is comprehensively evaluated from two complementary dimensions: information content complexity and structural boundary saliency. This avoids the noise sensitivity problem that may result from relying solely on information entropy, and the underestimation of texture regions that may result from relying solely on gradient strength, making the semantic richness index more comprehensive and robust. Furthermore, accurate semantic richness evaluation ensures that the fused target semantic feature vector can simultaneously retain the fine morphological features of the lesion region and the semantic association with the context. This ensures that the generated image semantic description information meets clinical diagnostic requirements in terms of lesion localization accuracy and the completeness of sign description, ultimately improving the accuracy and reliability of structured medical image reports. In addition, through the collaborative calculation of information entropy and gradient strength, the dominant representation regions of feature maps at each scale can be accurately identified. Shallow feature maps excel in edge gradient strength, while deep feature maps are more advantageous in cumulative information entropy values, thus enabling the adaptive fusion weights to be rationally allocated according to scale characteristics.

[0028] In one embodiment, please refer to Figure 2 As shown, the step of semantically deriving the target semantic feature vector based on a medical knowledge graph to generate image semantic description information may specifically include: S131. Map the target semantic feature vector to the initial node activation state in the medical knowledge graph. The medical knowledge graph includes anatomical structure nodes, pathological nodes, and clinical association edges. The pathological nodes refer to abnormal nodes that can be observed in medical images and have pathological significance. The clinical association edges refer to clinically significant relationship edges that connect different nodes. S132. Perform graph convolution propagation operation to propagate the node activation state along the clinical association edge and update the semantic confidence of each node; S133. Aggregate neighbor node information based on the attention mechanism to generate image semantic description information containing contextual relationships.

[0029] The target semantic feature vector is input into the medical knowledge graph, and a feature mapping network transforms it into the initial activation state of each node in the medical knowledge graph. Specifically, the medical knowledge graph pre-constructs a hierarchical node system and relationship network: anatomical structure nodes cover entity representations of various systems, organs, and subdivided anatomical regions of the human body; pathology nodes represent observable abnormal entities with pathological significance in medical images, including various lesions, signs, and density changes; clinical association edges connect different nodes, including location association edges and disease association edges. Location association edges represent the location relationship between anatomical sites and pathological manifestations, while disease association edges represent the suggestive relationship between pathological signs and diseases, as well as the differential diagnostic relationship between diseases. The feature mapping process calculates the similarity between the target semantic feature vector and the semantic embeddings of various nodes to generate the initial activation value of the corresponding node, representing the initial node activation state. A higher activation value indicates a higher degree of fit between the image features and the medical concept represented by the node.

[0030] A graph convolutional propagation operation is performed on a medical knowledge graph to iteratively propagate and update the node activation state along clinically related edges. Specifically, each node aggregates the activation information of its neighboring nodes connected by clinically related edges, performs weighted fusion based on the relation type weights of the edges, and updates the semantic confidence of the current node through a nonlinear transformation. This propagation process simulates the knowledge reasoning chain in clinical diagnosis, where the activation of anatomical structure nodes can be propagated to the corresponding pathological nodes along location-related edges, and the activation between pathological nodes can be propagated to differential diagnosis nodes along disease-related edges. After multiple rounds of iterative propagation, the semantic confidence of each node tends to stabilize, forming a globally consistent node activation distribution, thus expanding the initial local feature responses into a global semantic state containing knowledge associations.

[0031] The attention mechanism dynamically calculates the association strength weights between each node and its neighboring nodes, prioritizing the aggregation of neighboring information highly relevant to the current image features and suppressing interference from irrelevant or weakly associated nodes. Through hierarchical attention aggregation, lesion localization descriptions can be extracted from the anatomical structure dimension, sign feature descriptions from the pathological node dimension, and disease association and differential diagnosis prompts from the clinical association edge dimension. These are ultimately fused to generate image semantic description information containing complete contextual relationships. This description information is organized in a structured form, covering clinical report elements such as anatomical localization, pathological signs, disease tendencies, and recommendations. Furthermore, each element forms a logically self-consistent descriptive chain through association paths in a knowledge graph.

[0032] In this embodiment, a semantic inference method driven by a medical knowledge graph organically integrates the feature representation capabilities of deep learning with the structured reasoning capabilities of clinical knowledge, thereby significantly improving the accuracy, completeness, and clinical interpretability of image semantic description information. Specifically, by mapping and activating target semantic feature vectors to nodes in the medical knowledge graph, a semantic leap from low-level visual features to high-level medical concepts is achieved. Secondly, through iterative propagation along clinically relevant edges via graph convolution, the thought process of radiologists making diagnostic inferences based on anatomical knowledge and pathological associations is simulated, making the node confidence updates reasonable and guided by knowledge. Furthermore, by aggregating neighbor node information based on an attention mechanism, the focus on key areas of the image and the filtering out of irrelevant information are achieved, so that the generated image semantic description information not only includes accurate lesion localization and sign characterization, but also has the contextual association capabilities for disease identification and clinical prompts.

[0033] In one embodiment, performing the graph convolution propagation operation includes: Calculate the edge weights of the clinically associated edges, which are determined based on the physiological connection strength and pathological co-occurrence frequency between anatomical structure nodes; Perform multi-layer graph convolution operations. Each layer of graph convolution operations includes a message passing phase and a state update phase. In the message passing phase, neighbor node features are propagated along the edge weights. In the state update phase, a gated recurrent unit is used to fuse the current node features with the received neighbor node information. After the final graph convolution operation, the semantic confidence of each pathological node is output.

[0034] The determination of edge weights comprehensively considers two dimensions: the strength of physiological connections between anatomical structural nodes and the frequency of pathological co-occurrence. The strength of physiological connections reflects the spatial proximity and functional correlation of anatomical structural nodes in the human physiological system. The closer the spatial location of the anatomical structural nodes and the more coordinated their physiological functions, the higher the weight of their connecting edges. The frequency of pathological co-occurrence is based on large-scale clinical case statistics, reflecting the probability that the pathological entities represented by different nodes co-occur in real diagnosis and treatment scenarios. The higher the co-occurrence frequency of pathological node pairs, the higher the weight of their connecting edges.

[0035] Based on the calculated edge weights, multi-layer graph convolution operations are performed to achieve deep propagation and refined updating of node activation states. Specifically, each layer of graph convolution operation includes a message passing phase and a state update phase: In the message passing phase, each node sends its own feature information to neighboring nodes along clinical association edges. The sending process is weighted according to the edge weights, with neighboring nodes receiving stronger messages due to higher weights, thus achieving priority propagation of important medical associations; In the state update phase, a gated recurrent unit is used to fuse the original features of the current node with the received information from neighboring nodes, dynamically controlling the retention ratio of historical states and new input information. This allows nodes to selectively absorb association information from neighboring nodes while inheriting their own semantic features, avoiding semantic dilution caused by excessive smoothing. The multi-layered stacked graph convolution structure enables information to expand its propagation range layer by layer in the knowledge graph, capturing directly related local semantics at shallow layers and indirectly related global context at deeper layers.

[0036] After completing the final graph convolution operation, the semantic confidence score of each pathological node is output based on the final updated node state. Specifically, the semantic confidence score transforms the node state into a probability distribution through normalization, representing the degree of credibility of the pathological entity corresponding to each pathological node under the current image features. Pathological nodes with higher confidence scores indicate a higher degree of matching between the image features and the pathological entity, and should be highlighted in subsequent image semantic description information; pathological nodes with lower confidence scores serve as reference information for differential diagnosis or as a basis for exclusion. This output format transforms the abstract graph convolution operation results into clinically interpretable quantitative indicators.

[0037] In this embodiment, the introduction of the gated loop unit enables the node state update to have temporal memory and selective forgetting capabilities, effectively alleviating the excessive smoothing problem in deep graph convolution and ensuring the discriminative power of node semantic features after multi-layer propagation. Secondly, the staged separation design of message passing and state update enables the aggregation of neighbor information and the retention of its own features to achieve a dynamic balance, which fully utilizes the associative reasoning ability of the knowledge graph and maintains the semantic specificity of each node.

[0038] In one embodiment, matching the target structured report template based on the image semantic description information includes: Calculate the semantic similarity between the image semantic description information and each standard report template in the structured report template library. The structured report template library includes standard report templates for various image examination types. Each standard report template defines required fields, optional fields, and logical dependencies between fields. The standard report template with the highest semantic similarity is selected as the target structured report template.

[0039] Specifically, the structured report template library pre-constructs standard report templates covering various imaging examination types. Each template is designed according to clinical practice guidelines and radiology report writing guidelines, including imaging examination type identifiers, anatomical location description frameworks, pathological sign description paradigms, and diagnostic conclusion expression structures. Each standard report template defines required fields, optional fields, and logical dependencies between fields: required fields represent the core clinical elements that must be presented under this examination type; optional fields represent supplementary information selectively added based on actual imaging findings; and logical dependencies specify the conditional triggering rules and content association constraints between fields, ensuring the medical rationality and clinical completeness of the template structure. Semantic similarity calculation compares the entity concepts and relationship paths in the semantic description information of the images with the semantic embedding representations of the template fields, comprehensively evaluating the similarity between the two in dimensions such as examination type fit, anatomical location matching, and sign description coverage, generating a quantitative semantic similarity score.

[0040] The selection process for the target structured report template follows the optimal matching principle, comparing the image semantic description information with all candidate templates in the template library to identify the template structure that best matches the current image content in terms of examination type, anatomical location, and pathological features. The determination of the target structured report template means that subsequent report generation will use the field framework and logical structure defined by this template as the content organization benchmark. Required fields will be the mandatory filling targets for report generation, while optional fields will be dynamically determined based on the actual coverage of the image semantic description information. The logical dependencies between fields will guide the order and conditional association of the filled content, thereby ensuring that the generated medical image structured report not only meets clinical standards but also closely reflects the personalized characteristics of the specific image.

[0041] In this embodiment, by constructing a template intelligent matching mode based on semantic similarity calculation, the precise adaptation of image semantic content and standardized report structure is achieved, significantly improving the standardization and flexibility of medical image structured report generation. The pre-defined field hierarchy and logical dependencies in the structured report template library ensure that the template system possesses both standard constraints and dynamic adaptability. Required fields ensure the complete presentation of core clinical elements, while optional fields support flexible supplementation of personalized information. Logical dependencies maintain the medical rationality of the report content. Furthermore, the establishment of the optimal selection strategy enables the system to automatically identify the report paradigm most suitable for the current image characteristics, providing clear structural guidance and field constraints for the subsequent adaptive template filling process. This ensures that the generated medical image structured report strictly adheres to clinical diagnosis and treatment guidelines and radiology report writing standards while fully reflecting the individualized characteristics of specific images.

[0042] In one embodiment, generating a structured medical image report based on an adaptive template filling strategy includes: Identify the set of fields to be populated in the target structured report template; For each field to be filled, retrieve the corresponding image descriptive sub-map based on the image semantic description information; When the confidence level of the image descriptor is higher than a preset first threshold, the field content is directly generated using the image descriptor. When the confidence level of the image descriptor is lower than the preset first threshold but higher than the preset second threshold, a field containing a confidence level identifier is generated. Based on the logical dependencies between fields, the logical consistency of the content of each field is verified, and a structured medical image report is generated.

[0043] Specifically, the fields to be filled include required and optional fields predefined in the template. Required fields correspond to the core components of the clinical report, while optional fields correspond to auxiliary descriptions or extended information. The field identification process simultaneously extracts the semantic type identifier, content format specifications, and associated constraints of each field, establishing a mapping index between fields and image semantic description information, providing a structured list of filling targets for content retrieval and generation.

[0044] Image descriptive subgraphs are local knowledge structures extracted from the semantic derivation process of the aforementioned medical knowledge graph and associated with the semantic type of a specific field. They contain the specific entity concepts, attribute features, and relational paths required for that field. The retrieval process locates the matching subgraph in the image semantic description information based on the semantic type identifier of the field, and simultaneously obtains the confidence level accumulated by the subgraph during the semantic derivation process, which characterizes the reliability and strength of evidence of the descriptive content.

[0045] The first and second thresholds can be customized. When the confidence level of the subgraph is higher than the preset first threshold, it indicates that the description has high evidence support and reliability. The field content is directly generated using the entity concepts and relationship paths in the subgraph to form a definitive description statement. When the confidence level of the subgraph is lower than the preset first threshold but higher than the preset second threshold, it indicates that the description has a certain degree of uncertainty or insufficient evidence. A confidence level indicator is added while generating the field content, presented in the form of a prompt or a pending confirmation label, taking into account both information completeness and risk warning. When the confidence level of the subgraph is lower than the preset second threshold, the field content is marked as missing information or in a state of needing to be supplemented, triggering a manual review process or a knowledge graph completion process.

[0046] Logical dependencies include conditional triggering rules, mutual exclusion constraints, and hierarchical association rules. The verification process traverses the combined states of all field contents to detect any conflicts that violate preset logic. When inconsistencies are detected, automatic corrections are performed based on priority strategies or confidence comparisons, prioritizing the retention of high-confidence field contents and adjusting or removing low-confidence conflicting content to ensure that the final generated field set is logically consistent in medical terms and coherent in clinical expression.

[0047] After integrating and validating the content of each field, the final structured medical image report is generated according to the format specifications and layout of the target structured report template. The report generation process follows clinical report writing standards, organizing the field content into paragraph structures that conform to physicians' reading habits, while preserving the semantic tags and confidence metadata of each field.

[0048] In this embodiment, the image descriptive subgraph retrieval mechanism establishes a precise mapping between template fields and knowledge graph semantics, enabling the filled content to have a traceable knowledge source. The dual-threshold hierarchical strategy automatically distinguishes between deterministic descriptions and uncertain prompts, avoiding misleading presentations of low-confidence information while retaining potentially valuable clues. Furthermore, the validation of logical dependencies ensures the overall consistency of multi-field combinations, effectively avoiding potential medical logical contradictions or clinical expression conflicts that may result from the generation of isolated fields.

[0049] In one embodiment, the verification of the logical consistency of the content of each field based on the logical dependency relationship between fields includes: Extract the field constraint rules defined in the target structured report template, including mutual exclusion, coexistence, and causal relationships; Construct a field content logic graph, where nodes represent field content and edges represent logical relationships between field contents; Detect whether there are any logical conflict paths in the field content logic graph that violate the constraints between fields; When a logical conflict path is detected, the field content with a confidence level higher than a preset value is retained, and the field content with a confidence level lower than the preset value is corrected based on the medical knowledge graph until all logical conflict paths are eliminated.

[0050] Inter-field constraint rules characterize the medical logical relationships that must be satisfied between different field contents, including three basic types: mutual exclusion, coexistence, and causation. Mutual exclusion stipulates that specific field contents cannot appear simultaneously, reflecting the opposition or exclusivity between medical concepts; coexistence stipulates that specific field contents must exist simultaneously, reflecting the accompaniment or dependence between medical concepts; causation stipulates the conditional triggering chain between field contents, reflecting the deductive association between pathological signs and diagnostic conclusions.

[0051] Based on the content of each field to be verified, a logical graph of the field content is constructed. Nodes in the logical graph represent entities of each field content, and node attributes include the semantic type, confidence level, and source subgraph identifier of the field content. Edges represent the logical relationships between field content items. Edges are established based on the original dependencies of fields in the template and the semantic similarity between content entities, forming a directed graph network representing the global relational structure of the field content. This logical graph transforms discrete field content into a visualized relational topology, providing a structured analytical framework for conflict detection.

[0052] The detection process is performed according to the extracted constraint rule types: for mutual exclusion relationships, it checks whether there are related paths in the graph connecting the contents of mutually exclusive fields; for coexistence relationships, it checks whether there are isolated nodes in the graph that lack the contents of coexisting fields; for causal relationships, it checks whether there are path structures in the graph where the causal chain is broken or the causal direction is reversed. When any related path that violates a constraint rule is found, the path is marked as a logical conflict path, and the conflict type, the fields involved, and the association strength are recorded to form a list of conflicts to be resolved.

[0053] When a logical conflict path is detected, a conflict resolution strategy based on confidence comparison is executed. For each logical conflict path, the confidence level of each field content on the path is compared. Field content with a confidence level higher than a preset value is retained as a reliable benchmark, while field content with a confidence level lower than a preset value is marked as an object to be corrected. The field content to be corrected is re-derived and corrected based on the medical knowledge graph: other candidate image description subgraphs related to the semantic type of the field are retrieved from the knowledge graph, the confidence level of each candidate subgraph and its logical consistency with the retained field content are evaluated, and the candidate subgraph that satisfies the constraint rules and has the best confidence level is selected to replace the original field content. The conflict detection and correction process is iteratively executed until there are no more logical conflict paths that violate the constraint rules in the field content logic graph, achieving convergence of global logical consistency. Thus, through logical consistency verification based on inter-field constraint rules and graph structure analysis, global optimization and conflict resolution of the structured medical image report content are achieved, significantly improving the medical rigor of the generated report. Furthermore, the confidence-driven conflict resolution strategy ensures the priority retention of highly reliable information while leveraging the reasoning capabilities of medical knowledge graphs to provide corrective alternatives for low-confidence content. This avoids information loss caused by simple deletion and achieves a balance between report completeness and accuracy.

[0054] In one embodiment, after generating the structured medical image report based on the adaptive template filling strategy, the method further includes: Receive feedback from physicians regarding corrections to the structured medical image report; The correction feedback information, along with the original image data, the target semantic feature vector, and the image semantic description information, are stored in the case database. The convolution parameters of the multi-scale feature extraction network, the node features of the medical knowledge graph, and the template configuration of the structured report template library are updated periodically based on the case database.

[0055] The feedback information includes physicians' records of modifications to field content, supplementary information, deletion markers, and annotations of the reasons for corrections, covering multiple feedback types such as semantic description deviation correction, diagnostic conclusion adjustment, and expression optimization. Feedback information can be received through a human-computer interaction interface, supporting both structured form entry and natural language annotation, ensuring that physicians' clinical experience and judgment can be effectively transmitted back to the system.

[0056] The case database is organized in case units, each of which fully records the entire data chain from raw image input to final report output. This includes raw and standardized data at the image level, multi-scale feature maps and target semantic feature vectors at the feature level, atlas derivation results and descriptive information at the semantic level, and generated reports and physician corrections at the application level. This end-to-end data storage architecture allows subsequent model updates to trace the source of deviations at specific stages, supporting targeted parameter optimization and knowledge expansion.

[0057] Based on the amended examples accumulated in the case database, the convolutional parameters of the multi-scale feature extraction network are updated periodically. The update process filters out cases with semantic description biases, analyzes the mapping deviation between the target semantic feature vector and the physician's corrected semantic description, and adjusts the layer parameters of the convolutional network through backpropagation. This enhances the network's ability to extract features from error-prone lesion areas and difficult pathological signs, optimizes the sensitivity of the multi-scale feature fusion weights to key clinical information, and enables the updated network to generate feature representations that better meet clinical diagnostic needs.

[0058] Similarly, based on feedback from the case database, the node features of the medical knowledge graph are updated periodically. The update process involves statistically analyzing newly added pathological entities, corrected anatomical associations, and adjusted diagnostic relationships involved in physician corrections. The clinical rationality and frequency of these corrections are assessed, and high-frequency, reasonable corrections are transformed into node expansions or edge relationship adjustments in the knowledge graph. The semantic embedding representation of pathological nodes and the weight configuration of clinically related edges are updated, ensuring that the reasoning capabilities of the knowledge graph keep pace with clinical consensus.

[0059] Furthermore, based on the correction pattern analysis in the case database, the template configuration of the structured report template library is updated regularly. The update process involves cluster analysis to identify common characteristics of physician corrections, pinpointing deficiencies in existing templates regarding field settings, constraint rules, or logical structure. New template types are added to adapt to new examination techniques or disease changes. Logical dependencies between fields are optimized to match the latest standards for clinical report writing. The strategy for classifying required and optional fields is adjusted to balance report completeness and conciseness, ensuring the template library continuously adapts to the evolving needs of clinical practice.

[0060] In this embodiment, by constructing a physician feedback-driven continuous learning model, the dynamic evolution and performance improvement of the medical image structured report generation system were realized, significantly enhancing the system's clinical adaptability and long-term availability.

[0061] Please refer to Figure 3 As shown, embodiments of the present invention also provide a system for generating structured medical image reports, comprising: The acquisition module 31 is used to acquire the raw image data output by the medical imaging equipment, and to preprocess the raw image data to obtain standardized image data. The calculation module 32 is used to extract the basic feature map of the standardized image data using a multi-scale feature extraction network, perform multi-level downsampling on the basic feature map to obtain a multi-scale feature map sequence, calculate the corresponding adaptive fusion weight based on the semantic richness of each scale feature map, and perform weighted fusion on the multi-scale feature map sequence based on each adaptive fusion weight to obtain the target semantic feature vector. The derivation module 33 is used to perform semantic derivation on the target semantic feature vector based on the medical knowledge graph to generate image semantic description information; The generation module 34 is used to match the target structured report template based on the image semantic description information and generate a medical image structured report based on an adaptive template filling strategy.

[0062] Regarding the system in the above embodiments, the specific manner in which each module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0063] In one embodiment, the present invention also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the aforementioned method for generating a structured medical image report. The storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0064] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0065] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0066] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for generating structured reports of medical images, characterized in that, include: Acquire raw image data output from medical imaging equipment, preprocess the raw image data to obtain standardized image data; The basic feature map of the standardized image data is extracted using a multi-scale feature extraction network. The basic feature map is downsampled at multiple levels to obtain a multi-scale feature map sequence. The corresponding adaptive fusion weight is calculated based on the semantic richness of each scale feature map. The multi-scale feature map sequence is then weighted and fused based on the adaptive fusion weights to obtain the target semantic feature vector. Based on a medical knowledge graph, semantic inference is performed on the target semantic feature vector to generate image semantic description information; Based on the image semantic description information, a target structured report template is matched, and a medical image structured report is generated based on an adaptive template filling strategy.

2. The method according to claim 1, characterized in that, Before calculating the adaptive fusion weights corresponding to the semantic richness based on feature maps at each scale, the following steps are also included: Calculate the probability distribution of feature maps at each scale in the corresponding channel dimension, calculate the information entropy value of each spatial location based on the probability distribution, and perform global statistics on the information entropy values ​​of all spatial locations to obtain the cumulative information entropy value. Calculate the spatial gradient of the feature map at each scale in the corresponding horizontal and vertical directions, calculate the gradient magnitude based on each spatial gradient, and perform global statistics on all gradient magnitudes to obtain the edge gradient intensity. The semantic richness is calculated based on the cumulative information entropy value and edge gradient strength of the feature maps at each scale.

3. The method according to claim 1, characterized in that, The step of semantically deriving the target semantic feature vector based on a medical knowledge graph to generate image semantic description information includes: The target semantic feature vector is mapped to the initial node activation state in a medical knowledge graph, which includes anatomical structure nodes, pathological nodes, and clinical association edges. The pathological nodes refer to abnormal nodes that are observable in medical images and have pathological significance. The clinical association edges refer to clinically significant relationship edges that connect different nodes. Perform graph convolution propagation operation to propagate the node activation state along the clinical association edge and update the semantic confidence of each node; The attention mechanism is used to aggregate neighbor node information and generate image semantic description information containing contextual relationships.

4. The method according to claim 3, characterized in that, The execution of the graph convolution propagation operation includes: Calculate the edge weights of the clinically associated edges, which are determined based on the physiological connection strength and pathological co-occurrence frequency between anatomical structure nodes; Perform multi-layer graph convolution operations. Each layer of graph convolution operations includes a message passing phase and a state update phase. In the message passing phase, neighbor node features are propagated along the edge weights. In the state update phase, a gated recurrent unit is used to fuse the current node features with the received neighbor node information. After the final graph convolution operation, the semantic confidence of each pathological node is output.

5. The method according to claim 1, characterized in that, The target structured report template matched based on the image semantic description information includes: Calculate the semantic similarity between the image semantic description information and each standard report template in the structured report template library. The structured report template library includes standard report templates for various image examination types. Each standard report template defines required fields, optional fields, and logical dependencies between fields. The standard report template with the highest semantic similarity is selected as the target structured report template.

6. The method according to claim 1, characterized in that, The generation of structured medical image reports based on the adaptive template filling strategy includes: Identify the set of fields to be populated in the target structured report template; For each field to be filled, retrieve the corresponding image descriptive sub-map based on the image semantic description information; When the confidence level of the image descriptor is higher than a preset first threshold, the field content is directly generated using the image descriptor. When the confidence level of the image descriptor is lower than the preset first threshold but higher than the preset second threshold, a field containing a confidence level identifier is generated. Based on the logical dependencies between fields, the logical consistency of the content of each field is verified, and a structured medical image report is generated.

7. The method according to claim 6, characterized in that, The logical consistency check of the content of each field based on the logical dependency relationship between fields includes: Extract the field constraint rules defined in the target structured report template, including mutual exclusion, coexistence, and causal relationships; Construct a field content logic graph, where nodes represent field content and edges represent logical relationships between field contents; Detect whether there are any logical conflict paths in the field content logic graph that violate the constraints between fields; When a logical conflict path is detected, the field content with a confidence level higher than a preset value is retained, and the field content with a confidence level lower than the preset value is corrected based on the medical knowledge graph until all logical conflict paths are eliminated.

8. The method according to claim 1, characterized in that, After generating the structured medical image report based on the adaptive template filling strategy, the following is also included: Receive feedback from physicians regarding corrections to the structured medical image report; The correction feedback information, along with the original image data, the target semantic feature vector, and the image semantic description information, are stored in the case database. The convolution parameters of the multi-scale feature extraction network, the node features of the medical knowledge graph, and the template configuration of the structured report template library are updated periodically based on the case database.

9. A system for generating structured medical image reports, characterized in that, include: The acquisition module is used to acquire the raw image data output by the medical imaging equipment, and to preprocess the raw image data to obtain standardized image data. The calculation module is used to extract the basic feature map of the standardized image data using a multi-scale feature extraction network, perform multi-level downsampling on the basic feature map to obtain a multi-scale feature map sequence, calculate the corresponding adaptive fusion weight based on the semantic richness of each scale feature map, and perform weighted fusion on the multi-scale feature map sequence based on each adaptive fusion weight to obtain the target semantic feature vector. The derivation module is used to perform semantic derivation on the target semantic feature vector based on the medical knowledge graph to generate image semantic description information; The generation module is used to match the target structured report template based on the image semantic description information, and generate a medical image structured report based on an adaptive template filling strategy.

Images