Medical report automatic analysis and structured generation method based on multi-modal vision-language model

Abstract

The invention relates to the technical field of medical natural language processing, in particular to a medical report automatic analysis and structured generation method based on a multi-modal vision-language model. The method comprises the following steps: S1, acquiring a target medical image and an initial text report corresponding to the target medical image, and performing feature extraction and preprocessing to obtain a multi-level visual feature sequence and a text feature sequence; s2, obtaining at least one focus-description cluster corresponding to the independent clinical semantic unit according to the multi-level visual feature sequence and the text feature sequence; s3, for each focus-description cluster, the fuzzy medical description in the focus-description cluster is converted into structured attribute information with a deterministic state label; and S4, based on all the focus-description clusters and the corresponding structured attribute information, carrying out content assembly and generation under logic constraints, and outputting a structured medical report with a confidence identifier. According to the invention, deep understanding and high-reliability structured output of the medical report are realized.

Description

Technical Field

[0001] This application relates to the field of medical natural language processing technology, and in particular to a method for automatic parsing and structured generation of medical reports based on a multimodal visual-language model. Background Technology

[0002] Existing methods for generating medical reports have the following limitations:

[0003] The ambiguity of the "visual-language" semantic gap: Conventional methods attempt to hard-align image features with textual terms. However, medical descriptions contain a large number of ambiguous and context-dependent words (such as "possibly," "not excluded," "similar to before"). Simply matching "ground-glass opacity" to an image feature while ignoring preceding modifiers like "newly developed" or "larger than before" can lead to serious clinical misdiagnosis. Existing models struggle to capture these subtle semantic differences.

[0004] The inability to handle complex, multi-lesion, and dynamic images: When a report describes multiple lesions (such as multiple metastases in the liver) or involves the different manifestations of the same lesion in different sequences (such as T1 and T2 phases of MRI), existing models struggle to establish cross-regional associations and consistent understanding across sequences, easily resulting in structured outputs that are confusing with information.

[0005] Ignoring the inherent logical structure and clinical priors of reports: Medical reports are not simply a collection of words, but follow strict clinical logic (e.g., "imaging findings -> imaging diagnosis -> recommendations"). Existing end-to-end generation models often ignore this inherent structure, resulting in reports that may be logically disorganized or omit key parts (such as follow-up recommendations).

[0006] Poor adaptability to rare diseases or "normal" scenarios: The model performs poorly on rare diseases where training data is scarce. At the same time, a normal report with "no clear abnormalities" has a text-image association pattern (i.e., "how to describe normal") that is completely different from that of an abnormal report. In this scenario, conventional models are prone to "illusions" and generate non-existent lesions out of thin air.

[0007] Therefore, there is an urgent need for an intelligent approach that can deeply understand the complexities of medicine, handle special scenarios, and output highly reliable structured reports. Summary of the Invention

[0008] This application provides a method for automatic parsing and structured generation of medical reports based on a multimodal vision-language model to solve the above-mentioned problems.

[0009] This application provides a method for automatic parsing and structured generation of medical reports based on a multimodal vision-language model, the method comprising:

[0010] S1. Obtain the target medical image and its corresponding initial text report, and perform feature extraction and preprocessing on the target medical image and the initial text report respectively to obtain a multi-level visual feature sequence and a text feature sequence containing uncertain semantic tags;

[0011] S2. Based on the multi-level visual feature sequence and the text feature sequence, a cross-modal semantic association network is constructed. Through this network, the potential lesion regions in the image and the medical term entities in the text are associated and decoupled to obtain at least one lesion-description cluster corresponding to an independent clinical semantic unit.

[0012] S3. For each lesion-description cluster, perform contextual semantic analysis and uncertainty disambiguation processing. By fusing visual evidence and textual context, the ambiguous medical descriptions are transformed into structured attribute information with deterministic state labels.

[0013] S4. Based on all the lesion-description clusters and their corresponding structured attribute information, and following the predefined clinical report logic template, perform content assembly and generation under logical constraints, and at the same time evaluate the confidence of each part of the generated content, and output a structured medical report with confidence labels.

[0014] By employing the aforementioned technical solutions and introducing uncertainty disambiguation processing (S3) and logical constraint mechanisms, a deep understanding of medical reports and highly reliable structured output are achieved, significantly outperforming existing technologies. Performance improvements are reflected in: precise decoupling of multi-lesion information through the cross-modal semantic association network (S2) avoids misattribution errors and improves structured accuracy. Uncertainty disambiguation processing (S3) transforms ambiguous clinical descriptions into deterministic state labels, greatly enhancing the clinical usability and security of the output results. Increased efficiency is reflected in: adhering to clinical report logic templates and constraints, the generated reports have a rigorous structure, conforming to doctors' reading habits and reducing the time doctors spend on secondary review of automatically generated reports. High reliability and security are reflected in: the introduction of confidence assessment enables the system to recognize its uncertainties and proactively identify low confidence levels, ensuring safety in clinical applications.

[0015] Optionally, in step S1, feature extraction and preprocessing are performed on the initial text report, including:

[0016] The initial text report is segmented into sentences and medical entity recognition is performed to obtain multiple sentence units and corresponding medical terminology entities;

[0017] Based on a predefined uncertainty lexicon, the sentence units are scanned and semantically analyzed to identify and label the vague descriptive words contained therein and their grammatical modification range;

[0018] The labeled sentence units are co-encoded with their corresponding medical term entities to generate the text feature sequence containing uncertain semantic tags;

[0019] Feature extraction and preprocessing of the target medical image include:

[0020] The target medical image is sampled and segmented, and the spatial visual features of each frame or segment are extracted; a temporal coding model is used to model the serialized spatial visual features and capture the change pattern of the target medical image in the time dimension.

[0021] The spatial visual features are fused with the extracted temporal dimension change features to form the multi-level visual feature sequence used for subsequent cross-modal association.

[0022] The above technical solution significantly improves the accuracy of subsequent cross-modal association and disambiguation processing S3 through refined processing of text and image features. By explicitly labeling uncertain semantic markers in text features, the model can perform targeted disambiguation processing S3, rather than simply ignoring or misunderstanding ambiguous descriptions. By fusing temporal variation features into the visual feature sequence, the model can handle complex dynamic images, such as contrast-enhanced CT or MRI, thereby improving the accuracy of judging lesion progression or regression. This overcomes the limitations of existing models in processing dynamic images and ambiguous semantics, laying a data foundation for achieving highly reliable structured reports.

[0023] Optionally, in step S2, constructing a cross-modal semantic association network specifically involves constructing a visual-semantic heterogeneous graph;

[0024] The nodes of the visual-semantic heterogeneous graph include potential lesion region nodes identified by the multi-level visual feature sequence, and medical term entity nodes extracted by the text feature sequence.

[0025] The edges of the visual-semantic heterogeneous graph include: cross-modal edges representing the semantic correlation between potential lesion region nodes and medical term entity nodes, semantic edges representing the medical knowledge association between medical term entity nodes, and visual relation edges representing the spatial or feature similarity between potential lesion region nodes.

[0026] The above technical solution achieves deep associations within and between modalities by constructing a visual-semantic heterogeneous graph. The heterogeneous graph structure effectively decouples information from multiple lesions, ensuring that specific textual descriptions are associated only with the corresponding image lesions, thus overcoming the information confusion problem in existing technologies. By introducing semantic edges and visual relational edges, the model can utilize prior medical knowledge and spatial information between lesions to further enhance the accuracy and robustness of cross-modal associations.

[0027] Optionally, in step S2, the correlation and decoupling analysis through the network to obtain the lesion-description cluster includes:

[0028] Cross-modal message propagation and node feature updating are performed on the visual-semantic heterogeneous graph, enabling visual nodes and text nodes to interact and merge in the feature space based on their connection relationships.

[0029] Based on the updated node features, calculate the overall association strength between nodes in the graph;

[0030] Based on the comprehensive association strength, community discovery or cluster analysis is performed on all nodes in the graph, and the closely connected potential lesion area node group and medical term entity node group are divided into an independent lesion-description cluster, thereby achieving decoupling and alignment of different clinical semantic units in visual and textual modalities.

[0031] The above technical solution achieves deep associations within and between modalities by constructing a visual-semantic heterogeneous graph. The heterogeneous graph structure effectively decouples information from multiple lesions, ensuring that specific textual descriptions are associated only with the corresponding image lesions, thus overcoming the information confusion problem in existing technologies. By introducing semantic edges and visual relational edges, the model can utilize prior medical knowledge and spatial information between lesions to further enhance the accuracy and robustness of cross-modal associations.

[0032] Optionally, in step S3, the contextual semantic analysis and uncertainty disambiguation processing includes:

[0033] For the text description within the lesion-description cluster, locate the medical term entities and attributes affected by the uncertain semantic tags;

[0034] Combining the visual features corresponding to the lesion-description cluster, we analyze the contextual reasons for the generation of the uncertain semantic markers. The contextual reasons include dynamic changes in description time / atypical description signs / description based on indirect evidence.

[0035] Based on the contextual reasons and the strength of visual evidence, a discrete deterministic state label is assigned to the medical attributes associated with the medical term entity, and the deterministic state label is used to replace the original ambiguous text description.

[0036] The above technical solution, through contextual causal analysis and visual evidence fusion of uncertain semantic tags, achieves the transformation from ambiguous text to deterministic structured attribute information. This results in structured reports with higher clinical reliability and accuracy, solving the problem of existing technologies being unable to handle the ambiguity of medical descriptions.

[0037] Optionally, when the contextual cause describes dynamic changes over time, the specific process of the uncertainty disambiguation process includes:

[0038] Retrieve historical medical images associated with the target medical image;

[0039] Visual features of the corresponding anatomical regions in the historical medical images are extracted and compared with the visual features of the current target medical images in a temporal difference analysis to generate evidence of visual changes.

[0040] Based on the significance of the visual change evidence, fuzzy words describing temporal dynamics in the text are quantitatively interpreted and disambiguated, and the deterministic state labels of the medical attributes associated with the medical term entities are corrected accordingly.

[0041] The above technical solution, by introducing temporal differential comparison of historical images, transforms ambiguous temporal dynamic descriptions into quantifiable evidence of visual changes, thereby achieving precise semantic disambiguation. This significantly improves the structured accuracy and clinical value of follow-up reports, overcoming the limitations of existing technologies in processing dynamic images and temporal comparative descriptions.

[0042] Optionally, in step S4, the assembly and generation of content under logical constraints, following a predefined clinical report logic template, includes:

[0043] Based on the examination type of the target medical image, the corresponding hierarchical report template is invoked. The report template contains three logical parts: image manifestation, image diagnosis, and recommendations.

[0044] When generating the image representation portion, the lesions are sorted and organized according to their clinical importance corresponding to the lesion-description clusters;

[0045] When generating the image diagnosis section, logical guidance is provided based on the deterministic status labels in the structured attribute information. If there are attributes with a deterministic status of uncertain or suspicious, a suggested diagnosis or prompting language is generated in the diagnostic conclusion, and corresponding follow-up or further examination suggestions are triggered in the suggestion section.

[0046] By employing the aforementioned technical solution and adhering to clinical report logic templates and guided by deterministic status labels, the generated structured medical reports are ensured to be not only accurate but also structurally rigorous, aligned with clinical reasoning, and capable of automatically generating key follow-up or examination recommendations. This endows the system with primary clinical decision support capabilities, significantly enhancing the clinical usability of the reports.

[0047] Optionally, during the process of assembling and generating content under logical constraints, mandatory constraints are applied to the logical consistency of the report content, specifically including:

[0048] During the content assembly and generation process, it is ensured that the content of the image diagnosis section must be inferred from the lesions and signs already described in the image presentation section;

[0049] Ensure that the content of the recommendations section is directly related to any doubts or unclear diagnoses found in the imaging diagnosis section;

[0050] The above constraints are achieved by limiting the contextual information and vocabulary range that the model can access when generating different parts during the decoding process.

[0051] The above technical solution ensures the logical consistency of the generated report through mandatory constraints, fundamentally eliminating the generation of unfounded diagnostic conclusions or irrelevant suggestions, and significantly improving the clinical reliability and safety of structured medical reports.

[0052] Optionally, in step S4, the confidence assessment of each part of the generated content includes:

[0053] For each generated structured field, a comprehensive confidence score is calculated by integrating the strength of visual evidence from its source, the textual certainty after disambiguation, and the degree of logical consistency between the field and other parts of the report.

[0054] The overall confidence score is output together with the structured medical report and serves as the confidence indicator.

[0055] The above technical solution, through the integration of confidence assessments from multiple sources, provides a quantified reliability indicator for each structured field. This enables physicians to quickly identify potential risks in reports, improving the transparency and clinical safety of the AI ​​system.

[0056] Optionally, a pre-review process may be performed before the structured medicine report is output, including:

[0057] Before outputting the structured medical report with confidence indicators, set a confidence threshold to check whether the overall confidence score of key medical fields in the report is lower than the confidence threshold, or whether there are conflicting pieces of evidence from different sources.

[0058] If any of the above situations are detected, the corresponding content will be marked with significant risk in the final output report, and a prompt message will be generated to transfer the report to manual review.

[0059] The aforementioned technical solution, through pre-screening, endows the system with the capabilities of "self-protection" and "proactive assistance," ensuring that decision-making power can be safely transferred to human experts when the AI ​​system is uncertain or has internal conflicts. This is crucial for building doctors' trust in the AI ​​system and ensuring clinical safety. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart illustrating a method for automatic parsing and structured generation of medical reports based on a multimodal visual-language model, provided as an embodiment of this application. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0063] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0064] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0065] Existing medical report generation methods suffer from a series of problems, including the ambiguity of the "visual-language" semantic gap, a lack of ability to handle complex, multi-lesion, and dynamic images, neglect of the inherent logical structure and clinical priors of reports, and poor adaptability to rare diseases or "normal" scenarios.

[0066] Based on this, this application provides a method for automatic parsing and structured generation of medical reports based on a multimodal visual-language model. By introducing uncertainty disambiguation processing S3 and a logical constraint mechanism, it achieves a deep understanding of medical reports and highly reliable structured output, significantly outperforming existing technologies. Performance improvements are reflected in: precise decoupling of multi-lesion information through a cross-modal semantic association network S2 avoids misattribution errors and improves structured accuracy. Uncertainty disambiguation processing S3 transforms ambiguous clinical descriptions into deterministic state labels, greatly enhancing the clinical usability and safety of the output results. Increased efficiency is reflected in: following clinical report logic templates and constraints, the generated report structure is rigorous, conforms to doctors' reading habits, and reduces the time doctors spend on secondary review of automatically generated reports. High reliability and safety are reflected in: the introduction of confidence assessment enables the system to recognize its uncertainties and proactively identify low confidence levels, ensuring safety in clinical applications.

[0067] For specific implementation details, please refer to the following examples.

[0068] Figure 1 This is a flowchart illustrating a method for automatic parsing and structured generation of medical reports based on a multimodal visual-language model, as provided in one embodiment of this application. Figure 1 As shown, the method includes:

[0069] S1. Obtain the target medical image and its corresponding initial text report. Perform feature extraction and preprocessing on the target medical image and the initial text report respectively to obtain a multi-level visual feature sequence and a text feature sequence containing uncertain semantic tags.

[0070] S2. Based on the multi-level visual feature sequence and text feature sequence, construct a cross-modal semantic association network. Through this network, perform association and decoupling analysis on potential lesion regions in images and medical term entities in text to obtain at least one lesion-description cluster corresponding to an independent clinical semantic unit.

[0071] S3. For each lesion-description cluster, perform contextual semantic analysis and uncertainty disambiguation processing. By fusing visual evidence and textual context, the ambiguous medical descriptions are transformed into structured attribute information with deterministic state labels.

[0072] S4. Based on all lesion-description clusters and their corresponding structured attribute information, and following a predefined clinical report logic template, content is assembled and generated under logical constraints. At the same time, the confidence level of each part of the generated content is evaluated, and a structured medical report with confidence level labels is output.

[0073] This invention aims to address the problems existing in current automatic medical report generation methods, such as the "visual-linguistic" semantic gap, the ambiguity of medical descriptions, the inadequacy of processing multi-lesion or dynamic images, and the lack of logical structure in reports. Existing technologies struggle to accurately disambiguate semantics when dealing with ambiguous terms such as "possibly" and "cannot be ruled out" prevalent in medical reports, easily leading to clinical misjudgments. Furthermore, existing models often fail to effectively decouple and associate multi-lesion information and ignore the inherent hierarchical logical template of "image presentation-image diagnosis-recommendation" in medical reports. This method, based on real-time signal processing and cross-modal semantic association theory, achieves deep understanding and highly reliable structured generation of medical reports by introducing uncertain semantic tags and logical constraints. The working principle is as follows: First, through feature extraction and preprocessing (S1), the original image and text information are transformed into feature sequences that can be used for cross-modal interaction, where the text feature sequences are specifically marked with uncertain semantic tags. Second, a cross-modal semantic association network (S2) is constructed to precisely align and decouple potential lesion regions in the images with medical entities in the text, forming lesion-description clusters of independent clinical semantic units. Next, for each lesion-description cluster, in the uncertainty disambiguation processing S3 module, visual evidence and textual context are integrated to transform ambiguous descriptions into structured attribute information with deterministic state labels. Finally, in the structured content assembly and generation S4 stage, the clinical report logic template and constraints are strictly followed to ensure the clinical rigor of the generated report, and the reliability of the output is guaranteed through a confidence assessment mechanism.

[0074] The core components of this method include: a feature extraction and preprocessing module (S1), a cross-modal semantic association network (S2), an uncertainty disambiguation processing module (S3), and a structured content assembly and generation module (S4). In the feature extraction and preprocessing module (S1), the target medical image undergoes frame sampling and temporal coding to extract multi-level visual feature sequences; the initial text report undergoes sentence segmentation, entity recognition, and uncertainty lexical tagging to generate a text feature sequence containing uncertainty semantic tags. The cross-modal semantic association network (S2) interactively fuses visual and text features, obtaining lesion-description clusters through graph clustering and other methods, achieving decoupling of multi-lesion information. The uncertainty disambiguation processing module (S3) is a key improvement of this invention. By analyzing the contextual reasons for uncertainty semantic tags and combining the strength of visual evidence, it assigns deterministic state labels to medical attributes, overcoming the shortcomings of semantic hard alignment in existing technologies. The structured content assembly and generation module (S4) strictly follows the clinical report logic template, ensuring logical consistency in the generated image presentation, image diagnosis, and recommendation sections, and quantifies the reliability of each generated structured field through a confidence assessment mechanism.

[0075] This invention, by introducing uncertainty disambiguation processing S3 and a logical constraint mechanism, achieves a deep understanding of medical reports and highly reliable structured output, significantly outperforming existing technologies. Performance improvements are reflected in: precise decoupling of multi-lesion information through a cross-modal semantic association network S2, avoiding misattribution errors and improving structured accuracy; and the transformation of ambiguous clinical descriptions into deterministic state labels through uncertainty disambiguation processing S3, greatly enhancing the clinical usability and safety of the output results. Increased efficiency is reflected in: adhering to clinical report logic templates and constraints, generating reports with a rigorous structure that conforms to doctors' reading habits, reducing the time doctors spend on secondary review of automatically generated reports. High reliability and safety are reflected in: the introduction of confidence assessment, enabling the system to recognize its uncertainties and proactively identify low confidence levels, ensuring safety in clinical applications.

[0076] In some embodiments, step S1 involves feature extraction and preprocessing of the initial text report, including: segmenting the initial text report into sentences and recognizing medical entities to obtain multiple sentence units and corresponding medical term entities; scanning and semantically analyzing the sentence units based on a predefined uncertainty vocabulary library to identify and label the fuzzy descriptive words and their grammatical modification range contained therein; jointly encoding the labeled sentence units and their corresponding medical term entities to generate a text feature sequence containing uncertainty semantic tags; and feature extraction and preprocessing of the target medical image, including: performing frame sampling and segmentation on the target medical image to extract the spatial visual features of each frame or segment; using a temporal coding model to model the serialized spatial visual features to capture the change patterns of the target medical image in the time dimension; and fusing the spatial visual features with the extracted temporal dimension change features to jointly constitute a multi-level visual feature sequence for subsequent cross-modal association.

[0077] This implementation focuses on the details of step S1, aiming to address the shortcomings of existing technologies in handling ambiguous semantics in medical reports and in capturing the temporal information of dynamic images (such as multi-phase enhanced CT and dynamic MRI). Existing methods, in text processing, typically only perform simple entity recognition, failing to capture uncertain semantic markers in medical reports such as "possibly," "not excluded," and "similar to previous ones." These markers are often cautious descriptions made by clinicians based on uncertain evidence or dynamic changes. In image processing, existing methods mostly focus only on single-frame or static spatial features, ignoring the changing patterns of lesions over time or contrast agent perfusion in dynamic image sequences, thus lacking temporal evidence for subsequent cross-modal association. The feature extraction and preprocessing module S1 of this invention scans the text using an uncertainty lexicon, accurately identifying and marking ambiguous descriptive words and their modification ranges, generating a text feature sequence containing uncertain semantic markers, laying the foundation for subsequent disambiguation processing S3. For images, a temporal coding model is used to model the serialized spatial visual features, capture the change patterns in the temporal dimension (such as the regression, progression or enhancement of lesions), and fuse spatial features with temporal change features to generate a multi-level visual feature sequence, thereby providing comprehensive visual evidence for the analysis of dynamic images.

[0078] In terms of text feature extraction, this method first performs natural language processing on the initial text report, including sentence segmentation and medical entity recognition. The key is the introduction of a predefined uncertainty lexicon, containing words such as "tend to," "consider," "not rule out," "possibly," and "seems." The system scans and semantically analyzes each sentence unit, identifying these ambiguous descriptive words and determining their grammatical modification scope—that is, which medical term entity and its attributes they apply to. Subsequently, these labeled sentence units and medical term entities are co-encoded to form a text feature sequence containing uncertainty semantic tags. In terms of image feature extraction, for the target medical image (e.g., chest CT sequence), frame sampling or segmentation is first performed to extract the spatial visual features of each frame or segment (e.g., using ResNet or ViT). Then, a temporal coding model (e.g., LSTM, Transformer, or TCN) is used to model these sequential spatial visual features to capture the changing patterns of lesions at different time points or between different sequences. Finally, spatial visual features are fused with temporal change features to form a multi-level visual feature sequence. These features not only include spatial information such as the morphology and density of lesions, but also their dynamic change information, providing rich input for the cross-modal association in step S2.

[0079] This implementation significantly improves the accuracy of subsequent cross-modal association and disambiguation processing S3 through refined processing of text and image features. By explicitly labeling uncertain semantic markers in text features, the model can perform targeted disambiguation processing S3, rather than simply ignoring or misunderstanding ambiguous descriptions. By fusing temporal variation features into the visual feature sequence, the model can handle complex dynamic images, such as contrast-enhanced CT or MRI, thereby improving the accuracy of judging lesion progression or regression. This overcomes the limitations of existing models in processing dynamic images and ambiguous semantics, laying a data foundation for achieving highly reliable structured reports.

[0080] In some embodiments, in step S2, constructing a cross-modal semantic association network specifically involves constructing a visual-semantic heterogeneous graph. The nodes of the visual-semantic heterogeneous graph include potential lesion region nodes identified by multi-level visual feature sequences and medical term entity nodes extracted by text feature sequences. The edges of the visual-semantic heterogeneous graph include: cross-modal edges representing the semantic correlation between potential lesion region nodes and medical term entity nodes, semantic edges representing the medical knowledge correlation between medical term entity nodes, and visual relationship edges representing the spatial or feature similarity between potential lesion region nodes.

[0081] This implementation details how to construct a cross-modal semantic association network S2 by building a visual-semantic heterogeneous graph to solve the problems of confusion and misattribution of information among multiple lesions in existing technologies. In complex medical reports, multiple lesions may exist, and text descriptions are often scattered, making it difficult to accurately associate specific descriptions with specific lesion regions in images. Existing methods typically employ simple attention mechanisms or linear alignment, which easily leads to association errors. This method constructs a visual-semantic heterogeneous graph, transforming visual and textual information into a unified graph structure. Through the connection of nodes and different types of edges, multi-level semantic associations are achieved. Potential lesion region nodes represent local regional features in the image, while medical term entity nodes represent key medical concepts in the text. Cross-modal edges directly connect visual and text nodes, quantifying their semantic relevance. Semantic edges connect text nodes, utilizing prior knowledge such as medical knowledge graphs to reflect hierarchical or co-occurrence relationships, such as between "lung nodules" and "lung cancer." Visual relationship edges connect visual nodes, reflecting spatial relationships (e.g., adjacent, distant) or feature similarities between lesions. This heterogeneous graph structure can comprehensively capture the complex relationships within and between modes, providing a strong structural foundation for subsequent accurate decoupling and alignment.

[0082] The construction process of the visual-semantic heterogeneous graph is as follows: First, potential lesion regions are identified from multi-level visual feature sequences using a region proposal network or attention mechanism, and their features are used as potential lesion region nodes. Second, medical term entities are extracted from text feature sequences containing uncertain semantic labels, and their features are used as medical term entity nodes. Next, different types of edges are defined: cross-modal edges are constructed by calculating the similarity or relevance of visual and text node features, with stronger connections between nodes with high similarity. Semantic edges connect text entity nodes with medical relevance based on pre-trained medical knowledge graphs or co-occurrence statistics. Visual relationship edges are constructed by calculating the spatial distance or feature similarity between potential lesion region nodes; for example, connections are established between lesions that are close to each other, or between lesions with similar morphology and density.

[0083] This implementation achieves deep associations within and between modalities by constructing a visual-semantic heterogeneous graph. The heterogeneous graph structure effectively decouples information from multiple lesions, ensuring that specific text descriptions are associated only with the corresponding image lesions, thus overcoming the information confusion problem in existing technologies. By introducing semantic edges and visual relational edges, the model can utilize prior medical knowledge and spatial information between lesions to further enhance the accuracy and robustness of cross-modal associations.

[0084] In some embodiments, step S2 involves performing association and decoupling analysis through the network to obtain lesion-description clusters, including: performing cross-modal message propagation and node feature updates on the visual-semantic heterogeneous graph, enabling visual nodes and text nodes to interact and merge in the feature space based on their connection relationships; calculating the comprehensive association strength between nodes in the graph based on the updated node features; and performing community detection or clustering analysis on all nodes in the graph based on the comprehensive association strength, dividing the tightly connected potential lesion region node group and medical term entity node group into an independent lesion-description cluster, thereby achieving decoupling and alignment of different clinical semantic units in the visual and text modalities.

[0085] This implementation details how to perform association and decoupling analysis on a visual-semantic heterogeneous graph to ultimately obtain lesion-description clusters, a crucial step in achieving accurate alignment of multiple lesions. After constructing the heterogeneous graph, information interaction is required through graph neural networks (GNNs) to fully integrate intramodal and intermodal semantic information. Existing methods typically lack such a deep interaction mechanism, making it difficult to distinguish and align lesions in multi-lesion scenarios. This method, through cross-modal message propagation, enables the features of visual and text nodes to influence and update each other, enhancing their semantic relevance in the feature space. Subsequently, clustering analysis is performed based on this comprehensive association strength, tightly grouping visual and text nodes belonging to the same clinical semantic unit (i.e., describing the same lesion or clinical phenomenon) together to form independent lesion-description clusters.

[0086] The specific process of association and decoupling analysis includes: First, applying graph neural networks, such as Heterogeneous Graph Attention Networks (HAN), to the visual-semantic heterogeneous graph. During the message propagation phase, each node (whether visual or text node) receives information from its neighboring nodes, which is weighted and aggregated through different types of edges (cross-modal edges, semantic edges, and visual relation edges). For example, a visual node aggregates information from its associated text nodes, and a text node aggregates information from its associated visual nodes. Through multiple rounds of message propagation, the features of visual and text nodes are updated, making them more semantically consistent in the feature space. Second, based on the updated node features, the comprehensive association strength between any two nodes in the graph is calculated. This strength comprehensively considers the similarity of node features as well as their connection paths and weights in the graph. Finally, based on the comprehensive association strength, community detection algorithms (such as the Louvain algorithm) or clustering algorithms (such as DBSCAN) are used to divide the groups of visual nodes and text nodes with high association strength into independent lesion-description clusters. For example, a lesion-description cluster may contain nodes representing the lesion region in the right upper lobe of the lung in an image and entity nodes representing medical terms such as "ground-glass nodule" or "well-defined borders" in the text that describe the lesion.

[0087] This implementation achieves precise decoupling and alignment of different clinical semantic units across visual and textual modalities through cross-modal message propagation and clustering based on comprehensive association strength. This method effectively solves the information confusion problem in multi-lesion scenarios, ensuring the accuracy and independence of the input data for subsequent disambiguation processing S3 and structured generation S4.

[0088] In some embodiments, step S3 involves performing contextual semantic analysis and uncertainty disambiguation, including: locating the medical term entities and attributes affected by the uncertain semantic markers for the text description within the lesion-description cluster; analyzing the contextual reasons for the generation of the uncertain semantic markers by combining the visual features corresponding to the lesion-description cluster, including dynamic changes in description time / atypical description symptoms / description based on indirect evidence; and assigning a discrete deterministic state label to the medical attributes associated with the medical term entities based on the strength of the contextual reasons and visual evidence, with the deterministic state label replacing the original ambiguous text description.

[0089] This embodiment details the specific process of uncertainty disambiguation processing S3, which is the core innovation of this invention in overcoming the ambiguity problem in medical reports. Medical reports contain numerous ambiguous descriptions (such as "possibly inflammation," "malignancy cannot be ruled out"), reflecting the cautious judgment of clinicians in the absence of definitive evidence. Existing models often fail to understand the clinical intent behind this ambiguity. This method analyzes the specific reasons for the generation of uncertain semantic labels within the context of the lesion-description cluster, combined with the strength of visual evidence. For example, it considers whether the lesion is in a dynamic state (e.g., post-treatment follow-up), whether the signs are atypical on imaging, or whether it is based on indirect clinical evidence (e.g., lymph node enlargement). Based on the analysis of the causes and the quantification of visual evidence, the ambiguous textual descriptions are transformed into discrete definitive state labels (e.g., "clear progression," "potentially benign," "uncertain"), thereby achieving semantic disambiguation.

[0090] The specific process of contextual semantic analysis and uncertainty disambiguation processing S3 is as follows: First, after locating uncertain semantic markers (e.g., "possible") and their corresponding medical term entities (e.g., "pulmonary nodule") and attributes (e.g., "nature") from the lesion-description cluster, the system calls upon the multi-level visual features (including spatial features and temporal change features) corresponding to that lesion-description cluster. Second, the contextual cause of the uncertain semantic markers is analyzed through a classification or inference model. For example, if the visual features contain significant temporal change information, the contextual cause is determined to be "description of dynamic changes over time." If the lesion boundaries in the visual features are blurred, the density is uneven, and the differences from typical lesion features are large, the contextual cause is determined to be "description of atypical signs." If the text description involves indirect signs (e.g., "pleural effusion"), but the lesion itself is not obvious, the contextual cause is determined to be "description based on indirect evidence." Finally, based on the analyzed contextual cause and the quantification strength of the visual evidence, a discrete deterministic state label is assigned. For example, if the contextual reason is "atypical symptoms" and visual evidence (such as the malignancy probability prediction of the AI ​​model) shows moderate intensity, the status label might be assigned as "suspected malignancy." These definitive status labels (such as "clearly benign," "clearly progressing," and "uncertain") will replace the original vague textual description as structured attribute information output.

[0091] This implementation method achieves the transformation from ambiguous text to deterministic structured attribute information by performing contextual causal analysis and visual evidence fusion on uncertain semantic tags. This results in structured reports with higher clinical reliability and accuracy, solving the problem that existing technologies cannot handle the ambiguity of medical descriptions.

[0092] In some embodiments, when the contextual cause describes temporal dynamic changes, the specific process of uncertainty disambiguation includes: retrieving historical medical images associated with the target medical image; extracting visual features of the corresponding anatomical region in the historical medical images and performing temporal difference comparison with the visual features of the current target medical image to generate visual change evidence; and, based on the significance of the visual change evidence, quantifying and disambiguating the ambiguous terms in the text that describe temporal dynamics, and thereby correcting the deterministic state labels of the medical attributes associated with the medical term entities.

[0093] This implementation provides a specific solution for disambiguation of fuzzy descriptions of temporal dynamic changes. In clinical follow-up, doctors frequently use terms such as "similar to before," "slightly enlarged," and "potential for progression" to describe changes in lesions over time. These descriptions are based on comparisons with historical images. Existing models often fail to acquire or utilize historical image information for precise comparison when processing these temporal dynamic descriptions, making it difficult to accurately determine the true trend of lesion changes. This method actively retrieves historical medical images, performs temporal difference comparisons, and generates quantified visual evidence of changes, thereby providing precise quantitative interpretation and disambiguation of fuzzy terms in the text.

[0094] The specific process of uncertainty disambiguation processing S3 is as follows: When the system identifies that the contextual cause of the uncertain semantic label belongs to "describing temporal dynamic changes", it first retrieves historical medical images (such as a CT scan from three months ago) associated with the target medical image (such as the current CT scan) from the patient's PACS system. Secondly, through registration technology, the visual features of the corresponding anatomical region in the historical images (such as lesion volume, density, and morphology) are precisely aligned and temporally differentially compared with the visual features of the current target medical image. For example, the percentage change in lesion volume and the change in average density are calculated to generate visual change evidence. Subsequently, based on the significance of the visual change evidence, the ambiguous terms describing temporal dynamics in the text are quantitatively interpreted. For example, if the text describes "slightly increased", and the temporal difference comparison shows that the lesion volume has increased by 5% (within a preset threshold range), the system confirms the authenticity of "slightly increased" and corrects it to a deterministic status label, such as "minor progression". If the text describes "similar to before", and the comparison result shows that the lesion change is less than 3%, it is corrected to "stable status".

[0095] This implementation transforms ambiguous temporal dynamic descriptions into quantifiable evidence of visual change by introducing temporal differential comparison of historical images, thereby achieving precise semantic disambiguation. This significantly improves the structured accuracy and clinical value of follow-up reports, overcoming the limitations of existing technologies in processing dynamic images and temporal comparative descriptions.

[0096] In some embodiments, a corresponding hierarchical report template is invoked based on the examination type of the target medical image. The report template includes three logical parts: image manifestation, image diagnosis, and recommendations. When generating the image manifestation part, the lesions are sorted and organized according to their clinical importance corresponding to the lesion-description cluster. When generating the image diagnosis part, logical guidance is provided based on the deterministic status labels in the structured attribute information. If there are attributes with a deterministic status of uncertain or suspicious, a suggested diagnosis or prompting language is generated in the diagnostic conclusion, and corresponding follow-up or further examination recommendations are triggered in the recommendations part.

[0097] This implementation details the process of assembling structured content and generating S4, focusing on adhering to clinical report logic templates and introducing a logical guidance mechanism to ensure the clinical rigor and completeness of the generated report. Medical reports are not simply a list of information, but follow strict clinical reasoning logic (e.g., from objective description to subjective judgment, and then to subsequent recommendations). Existing end-to-end generation models often overlook this inherent structure, resulting in reports that may be logically disorganized or lack crucial recommendations. This method ensures the structure and clinical relevance of the report by invoking a hierarchical report template and using logical guidance based on disambiguated deterministic status labels.

[0098] The specific process of content assembly and generation of S4 is as follows: First, the system calls the corresponding hierarchical report template based on the examination type of the target medical image (e.g., CT, MRI, ultrasound). This template pre-sets the structure of three logical parts: image manifestation, image diagnosis, and recommendations. Second, when generating the image manifestation section, the system sorts and organizes all lesions corresponding to the lesion-description cluster according to their clinical importance (e.g., lesions with high malignancy probability, large size, or significant changes), ensuring that the most important information is described first. When generating the image diagnosis section, the system uses the deterministic status labels in the structured attribute information for logical guidance. For example, if the deterministic status label of the lesion is "definitely malignant," a definitive diagnosis is generated in the diagnostic conclusion; if there is an attribute with a deterministic status label of "uncertain" or "suspicious" (e.g., a lung nodule is "suspicious for malignancy"), the system generates a suggested diagnosis (e.g., "prone to malignancy, further examination required") or suggestive language in the diagnostic conclusion, and triggers the automatic generation of corresponding follow-up (e.g., "re-examination recommended in 3 months") or further examination recommendations (e.g., "PET-CT evaluation recommended") in the recommendations section.

[0099] This implementation method, by following a clinical report logic template and guided by deterministic status labels, ensures that the generated structured medical reports are not only accurate but also structurally rigorous, consistent with clinical reasoning, and can automatically generate key follow-up or examination recommendations. This endows the system with primary clinical decision support capabilities, greatly enhancing the clinical usability of the reports.

[0100] In some embodiments, during the content assembly and generation process under logical constraints, mandatory constraints are applied to the logical consistency of the report content. Specifically, during the content assembly and generation process, it is ensured that the content of the image diagnosis section must be inferred from the lesions and signs described in the image presentation section; and that the content of the suggestion section is directly related to the doubts or unclear diagnoses in the image diagnosis section. The above constraints are achieved by limiting the contextual information accessible to the model and the range of selectable vocabulary when generating different parts during the decoding process.

[0101] This implementation details the mandatory constraint mechanism in the structured generation S4, aiming to address potential logical inconsistencies or "illusions" in reports generated by existing models. Logical consistency is crucial in medical reports; for example, diagnostic conclusions must be supported by objective signs in imaging findings, and recommendations must address any doubts or ambiguities in the diagnosis. Existing end-to-end generation models lack such mandatory logical constraints, potentially generating unfounded diagnoses or irrelevant recommendations. This method, by imposing mandatory constraints during the decoding process, ensures causal relationships and consistency among the three logical components of the report: imaging findings, imaging diagnosis, and recommendations.

[0102] The specific implementation of logical constraints: A restricted decoding mechanism is employed during the content assembly and generation of S4. First, when generating the image diagnosis section, the model is forced to reason and reference only from the lesions and signs already described in the image presentation section (i.e., information already structured into deterministic state labels). For example, if the image presentation does not describe "pulmonary nodules," the image diagnosis section cannot generate a diagnosis of "pulmonary nodules." Second, when generating the suggestion section, the model is forced to suggest suggestions only for suspicious points corresponding to the deterministic state labels of "uncertain" or "suspicious" present in the image diagnosis section. For example, the model is only allowed to generate the term "suggest PET-CT" if the image diagnosis indicates "suspicious malignancy." This constraint is achieved by limiting the contextual information accessible to the model when generating different sections during the decoding process (e.g., only accessing the already generated content of the image presentation when generating a diagnosis) and the selectable vocabulary range (e.g., dynamically adjusting the suggestion vocabulary based on the diagnostic results).

[0103] This implementation method ensures the logical consistency of the generated report through mandatory constraints, fundamentally eliminating the generation of unfounded diagnostic conclusions or irrelevant recommendations, and significantly improving the clinical reliability and safety of structured medical reports.

[0104] In some embodiments, step S4, which involves evaluating the confidence of each generated part of the content, includes: for each generated structured field, integrating the strength of visual evidence from its source, the textual certainty after disambiguation, and the degree of logical consistency between the field and other parts of the report, to calculate a comprehensive confidence score; the comprehensive confidence score is output together with the structured medical report as the confidence identifier.

[0105] This embodiment details the confidence assessment mechanism, a crucial step in ensuring high output reliability. In clinical applications, AI systems must be able to "aware of their own uncertainty," meaning they can proactively identify risks when information is incomplete or evidence is insufficient. Existing models typically output only a single result, lacking a quantitative assessment of the result's reliability, which is dangerous in clinical decision-making. This method calculates a comprehensive confidence score by fusing multi-source evidence (visual evidence, textual certainty, and logical consistency) and outputs it as a confidence indicator, providing doctors with decision-making references.

[0106] The specific process of confidence assessment: The system calculates a comprehensive confidence score for each generated structured field in the structured medical report (e.g., lesion size, nature, state of change, etc.). This score is the result of fusing multi-source information: First, the strength of visual evidence (e.g., confidence in lesion segmentation, accuracy of feature extraction) reflects the degree of support for the field on the image. Second, the textual certainty status after disambiguation (e.g., "clear progression" or "uncertain") reflects the semantic certainty of the textual information. Finally, the degree of logical consistency between the field and other parts of the report (image presentation, image diagnosis, recommendations) (quantified by the degree of satisfaction of logical constraints) reflects the reasonableness of the field in the overall structure of the report. These three parts of evidence are used to calculate the comprehensive confidence score through a weighted fusion model (e.g., based on deep learning or Bayesian networks). For example, if a lesion has high visual evidence strength, a textual certainty status of "clearly benign," and high logical consistency with the diagnosis and recommendations, the comprehensive confidence score is close to 1. Conversely, if the visual evidence strength is low and the textual certainty status is "uncertain," the score is low. The overall confidence score is output as a confidence indicator along with the structured medical report.

[0107] This implementation provides a quantified reliability indicator for each structured field by fusing confidence assessments from multiple sources of evidence. This enables physicians to quickly identify potential risks in reports, improving the transparency and clinical safety of the AI ​​system.

[0108] In some embodiments, a pre-review process is performed before the structured medical report is output, including: setting a confidence threshold, checking whether the overall confidence score of key medical fields in the report is lower than the confidence threshold, or whether there are conflicting pieces of evidence from different sources, before outputting the structured medical report with a confidence index; if the above situations are detected, the corresponding content is marked with significant risk in the final output report, and a prompt message for transfer to manual review is generated.

[0109] This implementation details the "pre-review processing" mechanism before the output of structured medical reports, which is the final line of defense for achieving a secure closed loop in clinical applications. Due to the special nature of medical reports, even high-confidence AI results require additional security safeguards. This method introduces pre-review processing, which proactively identifies high-risk content in reports by setting confidence thresholds and detecting evidence conflicts. It also forcibly generates prompts to transfer the report to human review, ensuring that human experts intervene when the system is uncertain or contradictory, thus preventing potential clinical errors.

[0110] The specific process of pre-review is as follows: First, the system sets a predefined confidence threshold (e.g., 0.85) to determine the reliability of key medical fields (such as lesion nature, stage, etc.). Before the final structured medical report is output, the system checks the overall confidence score of all key medical fields. If the score of any key field is lower than the threshold, a risk alert is triggered. Second, the system checks for conflicting evidence from different sources. For example, visual evidence shows a significant reduction in lesion volume, but the textual definitive status is marked as "probable progression." This conflict also triggers a risk alert. If any of the above situations are detected, the system will significantly mark the corresponding content as risky in the final output report (e.g., using red highlighting or special symbols) and generate a prompt message for manual review, such as "The system detected a low confidence score (0.75) for the [lesion nature] field, which is recommended for radiologists to conduct a thorough review."

[0111] This implementation method, through pre-approval processing, endows the system with the capabilities of "self-protection" and "proactive assistance," ensuring that decision-making power can be safely transferred to human experts when the AI ​​system is uncertain or has internal contradictions. This is crucial for building doctors' trust in the AI ​​system and ensuring clinical safety.

Claims

1. A method for medical report automatic parsing and structured generation based on a multi-modal vision-language model, characterized in that, The method comprises the following steps: S1, obtaining a target medical image and an initial text report corresponding to the target medical image, and performing feature extraction and preprocessing on the target medical image and the initial text report respectively to obtain a multi-level visual feature sequence and a text feature sequence containing uncertainty semantic labels; S2, constructing a cross-modal semantic association network according to the multi-level visual feature sequence and the text feature sequence, and performing association and decoupling analysis on potential lesion regions in the image and medical term entities in the text through the network to obtain at least one lesion-description cluster corresponding to an independent clinical semantic unit; S3, for each lesion-description cluster, performing context semantic analysis and uncertainty disambiguation processing, and converting ambiguous medical descriptions into structured attribute information with a certainty status label by fusing visual evidence and text context; S4, based on all the lesion-description clusters and their corresponding structured attribute information, following a pre-defined clinical report logic template, performing content assembly and generation under logical constraints, and simultaneously performing confidence evaluation on each part of the generated content, and outputting a structured medical report with a confidence identifier.

2. The method of claim 1, wherein, In step S1, the feature extraction and preprocessing of the initial text report comprises: performing sentence segmentation and medical entity recognition on the initial text report to obtain multiple sentence units and corresponding medical term entities; based on a pre-defined uncertainty vocabulary library, scanning and performing semantic analysis on the sentence units to identify and label the ambiguous description words and their grammatical modification range contained therein; encoding the labeled sentence units and the corresponding medical term entities together to generate the text feature sequence containing uncertainty semantic labels; the feature extraction and preprocessing of the target medical image comprises: frame sampling and segmentation of the target medical image to extract spatial visual features of each frame or segment; using a time encoding model to model the serialized spatial visual features to capture the change pattern of the target medical image in the time dimension; fuse the spatial visual features and the extracted time dimension change features to form the multi-level visual feature sequence used for subsequent cross-modal association.

3. The method of claim 2, wherein, In step S2, the construction of the cross-modal semantic association network is specifically constructing a visual-semantic heterogeneous graph; the nodes of the visual-semantic heterogeneous graph include potential lesion region nodes identified by the multi-level visual feature sequence and medical term entity nodes extracted by the text feature sequence; the edges of the visual-semantic heterogeneous graph include: cross-modal edges representing the semantic correlation between potential lesion region nodes and medical term entity nodes, semantic edges representing the medical knowledge association between medical term entity nodes, and visual relationship edges representing the spatial or feature similarity between potential lesion region nodes.

4. The method of claim 3, wherein, In step S2, the association and decoupling analysis through the network to obtain the lesion-description cluster comprises: performing cross-modal message propagation and node feature updating on the visual-semantic heterogeneous graph, so that the visual nodes and the text nodes interact and fuse in the feature space based on their connection relationship; Based on the updated node features, the comprehensive correlation strength between nodes in the graph is calculated; Based on the comprehensive correlation strength, community discovery or clustering analysis is performed on all nodes in the graph, and closely connected potential lesion area node groups and medical term entity node groups are divided into an independent lesion-description cluster, thereby realizing decoupling and alignment of different clinical semantic units in visual and text modalities.

5. The method of claim 3, wherein, In step S3, the context semantic analysis and uncertainty disambiguation processing includes: For the text description in the lesion-description cluster, the medical term entity and attribute affected by the uncertainty semantic label are located; Combined with the visual features corresponding to the lesion-description cluster, the context reason for the uncertainty semantic label is analyzed, and the context reason includes description time dynamic change / description atypical sign / description based on indirect evidence; According to the strength of the context reason and visual evidence, a discrete certainty state label is assigned to the medical attribute associated with the medical term entity, which is used to replace the original ambiguous text description.

6. The method of claim 5, wherein, When the context reason is description time dynamic change, the specific process of the uncertainty disambiguation processing includes: Call the historical medical image associated with the target medical image; Extract the visual features of the corresponding anatomical region in the historical medical image, and perform time difference comparison with the visual features of the current target medical image to generate visual change evidence; According to the significance of the visual change evidence, the ambiguous words in the text describing the time dynamic are quantitatively interpreted and disambiguated, and the certainty state label of the medical attribute associated with the medical term entity is corrected accordingly.

7. The method of claim 6, wherein, In step S4, the content assembly and generation under logical constraints are performed according to the pre-defined clinical report logic template, including: According to the examination type of the target medical image, the corresponding hierarchical report template is called, which includes three logical parts of image manifestation, image diagnosis and suggestion; When generating the image manifestation part, the clinical importance of the lesion corresponding to the lesion-description cluster is sorted and organized; When generating the image diagnosis part, the certainty state label in the structured attribute information is used for logical guidance. If there is an attribute with an uncertain or suspicious certainty state, a suggestive diagnosis or a suggestive language is generated in the diagnosis conclusion, and a corresponding follow-up or further examination suggestion is generated in the suggestion part.

8. The method of claim 7, wherein, In the process of content assembly and generation under logical constraints, the logical consistency of the report content is forcedly constrained, specifically including: During the content assembly and generation process, it is ensured that the content of the image diagnosis part must be inferred from the lesions and signs described in the image manifestation part; It is ensured that the content of the suggestion part is directly related to the doubts or unclear diagnoses in the image diagnosis part; The above constraints are realized by limiting the context information and the range of selectable words that the model can access when generating different parts during the decoding process.

9. The method of claim 8, wherein, In step S4, the confidence evaluation of the generated content of each part includes: For each generated structured field, the strength of visual evidence from its source, the certainty status of the disambiguated text, and the degree of logical consistency of the field with other parts of the report are fused to compute an overall confidence score; The overall confidence score is output together with the structured medical report as the confidence identifier.

10. The method of claim 9, wherein, Before the structured medical report is output, a pre-audit process is performed, including: A confidence threshold is set, and before the structured medical report with the confidence identifier is output, it is detected whether there is a situation that the overall confidence score of a key medical field in the report is lower than the confidence threshold, or whether there is a situation that different source evidences point to conflicts; If the above situations are detected, the corresponding content in the final output report is marked with a prominent risk, and prompt information for handing over to manual audit is generated.

Images