A knowledge-enhanced report generation method and device for cervical lesion MRI images

Abstract

The application relates to the technical field of medical image analysis processing, in particular to a cervical lesion MRI image knowledge enhancement type report generation method and device, a fine-tuned vision-language model is used for cross-modal semantic understanding of a cervical lesion MRI image, information such as an anatomical structure, an image sign and a suspected abnormal region is automatically extracted, and a structured draft is generated; standardized processing is performed through a negative description filtering, region-sign alignment verification and field completion rule filtering mechanism; a mixed retrieval strategy is used to acquire standard expressions and diagnostic rules from a medical knowledge base for knowledge enhancement; a multi-source problem set is formed through parallel quality evaluation of a rule reflection module and a large language model reflection module for automatic correction; and finally, an optimal output is selected from multiple candidate results through a multi-candidate scoring selection mechanism. The application can realize structured expression and closed-loop optimization of a medical image report, and effectively improve the accuracy, consistency and clinical practicability of report generation.

Description

Technical Field

[0001] This invention relates to the field of medical image analysis technology, specifically to a method and apparatus for generating knowledge-enhanced reports from MRI images of cervical lesions. Background Technology

[0002] In the clinical diagnosis and treatment of cervical cancer, MRI (magnetic resonance imaging) is a core examination tool for staging, treatment planning, and efficacy evaluation of cervical cancer because of its high soft tissue resolution and excellent imaging contrast, which can clearly show the extent of cervical lesion infiltration, anatomical structure involvement, and lymph node metastasis.

[0003] Current clinical cervical MRI reports rely on manual interpretation and subjective writing by physicians. Report quality is affected by physician experience and workload, and commonly suffers from problems such as missed signs, inconsistent descriptions, mixed positive and negative information, and non-standard structure, failing to meet the needs of precise diagnosis and treatment. Existing intelligent report generation technologies mainly include four categories: traditional machine learning, single-model deep learning, general visual language models, and simplified intelligent agent processes. However, all have significant shortcomings. They mostly adopt a single-stage, single-model processing architecture, lacking a task collaborative decomposition mechanism. The generated results often fail to accurately match the anatomical region, the reasoning process is opaque and lacks correctability, and it is difficult to simultaneously consider global semantic reasoning and local lesion localization, resulting in insufficient accuracy and standardization in report generation. Summary of the Invention

[0004] To address the aforementioned technical problems, the present invention aims to propose a knowledge-enhanced report generation method and device for cervical lesion MRI images, which optimizes report quality based on knowledge constraints and self-correction mechanisms to meet the needs of precise clinical diagnosis and treatment.

[0005] This invention provides a method for generating knowledge-enhanced reports from MRI images of cervical lesions, comprising the following steps:

[0006] S1. Preprocess the input cervical lesion MRI images to obtain standardized image data; input the standardized image data into a visual-language model, and extract multi-scale visual feature representations through an image encoder; map the visual features to the semantic space of the language model based on a cross-modal projection matrix to obtain visual-language alignment features; use an autoregressive text decoder to serialize and decode the visual-language alignment features to generate a structured draft containing anatomical structure fields, image sign fields, suspected abnormal area fields, and uncertainty fields; S2. Standardize the structured draft, including removing negative descriptions and invalid information, establishing the correspondence between image features and anatomical regions, completing necessary structural fields, unifying the expression form and semantic granularity, and outputting the standardized structured intermediate representation. S3. Retrieve standardized expressions and diagnostic rules related to the current image signs, anatomical structures and descriptive content from the pre-built medical knowledge base, and introduce the retrieved knowledge information as external constraints into the generation process to output the knowledge-enhanced structured representation; S4. The structured representation enhanced by the knowledge is evaluated in parallel by the rule reflection module and the large language model reflection module to form a multi-source problem set, and the structured representation is automatically corrected and the corrected structured representation is output. S5. Generate multiple candidate structured results, score and screen them based on structural integrity, semantic consistency and medical rationality indicators, and select the result with the best score as the final output cervical lesion MRI structured report; wherein, steps S1 to S5 constitute a closed-loop report generation process, and there is bidirectional information transmission between each step, including forward transmission flow and backward correction flow.

[0007] Preferably, the preprocessing of the input cervical lesion MRI image in S1 specifically includes the following steps: First, the original three-dimensional MRI volume data is decomposed into a two-dimensional slice sequence along the axial direction; then, the pixel intensity of each two-dimensional slice is normalized, and the original pixel values ​​are mapped to the [0, 255] interval through linear transformation; then, the single-channel grayscale image is copied and expanded into a three-channel format to adapt to the input requirements of the pre-trained visual-language model; finally, all two-dimensional slices are adjusted to a preset fixed resolution to generate a standardized image data tensor.

[0008] Preferably, the visual-language model in S1 is a dedicated model obtained by performing domain adaptation on a cervical lesion MRI image dataset using LLaVA or Qwen-VL. The adaptation involves adjusting the parameters of the model's attention layer and projection layer using a cervical MRI image-report paired dataset and low-rank adaptation technology, so that the model can master the professional visual representation of cervical anatomy and the language expression pattern of medical imaging terminology. The image encoder adopts a VisionTransformer or convolutional neural network architecture.

[0009] Preferably, the image encoder in S1 extracts multi-scale visual feature representations:

[0010] in, The total number of image patches. Indicates the first Flattened pixel vectors of an image patch For image patch embedding matrix, For the first The location encoding vector of each image patch For the embedded first Image patch feature vectors; visual feature representations output by the image encoder. ,in This represents the total number of layers in the visual encoder. For visual feature dimensions, Given a real matrix space, through a learnable linear projection matrix Mapped to the feature space of the language model:

[0011] in, For visual-language alignment features, To define the feature dimension of the language model, a projection operation is used to place the visual features and text word embeddings in the same vector space, facilitating subsequent autoregressive text generation. The autoregressive text decoder employs a causal language model integrated with the visual-language model, using visual-language alignment features as prefix cues, and generates structured text token by token through an autoregressive approach. At each generation time, the model calculates the conditional probability distribution for each candidate word in the vocabulary:

[0012] in, Let be the probability normalization function. For the first The word units generated at each time point For the generated word sequence, For the decoder in the The hidden state vector at time step 1. The output layer projection matrix. The temperature parameter is used to control the randomness of the generated text. The structured draft is structured text organized according to a predefined JSON Schema format. The generation process of the structured draft adopts a constraint decoding strategy, that is, JSON Schema syntax constraints are introduced during the decoding process to ensure that the generated text always conforms to the predefined structured format.

[0013] Preferably, the removal of negative descriptions and invalid information in S2 includes two levels of filtering: the first level is keyword rule filtering, which performs precise matching based on a pre-built negative description keyword dictionary; the negative description keyword dictionary includes: negation words, weakened modifiers, and negative sentence templates. For the text content of each field in the structured draft, it is checked one by one whether it contains keywords or matching sentence templates from the dictionary. If a match is successful, the statement is marked as a candidate negative description; the second level is semantic classifier filtering, which uses a lightweight binary classifier based on a pre-trained language model to perform fine-grained semantic-level judgment on the candidate negative descriptions. The input of the classifier is the candidate statement and its context fragment, and the output is the probability value of the statement belonging to a negative description. ,when When that happens, remove the statement from the structured draft, where This is a preset threshold.

[0014] Preferably, the semantic classifier filters the judgment probability. This can be expressed by the following formula:

[0015] in, Candidate statements, For pre-training Encoder, For pooling operations, The classifier weight matrix is... For bias terms, This is the Sigmoid activation function.

[0016] Preferably, in step S2, the establishment of the correspondence between imaging features and anatomical regions is based on a pre-constructed anatomical structure-feature compatibility matrix, and the predefined set of anatomical regions is... The predefined set of symptom types is Then the compatibility matrix Defined as if signs Appearing in the area If the condition is met, the value is 1; otherwise, the value is 0. Represents a set of predefined anatomical regions The number of regions in This represents the number of task types in the predefined set of feature types S; for each region-feature pair in the structured draft. Query the compatibility matrix to determine Whether it is true; if If the signs are incompatible, they are marked as incompatible pairs, triggering a correction operation. This correction operation first attempts to remap the incompatible signs to compatible anatomical regions. Let the signs... The set of compatible regions is ,in It is a region index. For the set of regions in the anatomical region that exhibit the signs, select the compatible regions that have the highest semantic similarity. As a new anchor target:

[0017] in, Anatomical region With signs Semantic similarity between them Indicates semantic similarity Anatomical region that reaches its maximum value The cosine similarity is calculated after encoding by a pre-trained language model to obtain:

[0018] in, , These are semantic embedding vectors for the region name and the feature description, respectively.

[0019] Preferably, the verification process of the rule reflection module in S4 includes the following steps: assuming the rule base contains... Rules Each rule By a conditional predicate And a problem description Composition, for a given knowledge, enhances the structured representation Each rule's conditional predicate is evaluated one by one: if the conditional predicate evaluates to the given structured representation, the corresponding problem description is output; otherwise, an empty set is output. The problem descriptions of all triggering rules constitute the rule-level problem set. :

[0020] The above The index for the number of rules. For the first Description of the problems caused by the triggering of this rule; The semantic evaluation process of the reflection module of the large language model is modeled using conditional probability; let the output of the reflection model be a set of questions. Given knowledge-enhanced structured representation and retrieval knowledge The model generates the first The probability of each problem description is:

[0021] in, For decoders of large language models, As a template for reflection prompts, For splicing operations, Describe the first to the (i-1)th questions; A complete set of questions is generated through autoregressive sampling. :

[0022] Aggregating and deduplicating the rule-level question set and the semantic-level question set output by the large language model reflection module is the process of merging two heterogeneous question sets into a unified multi-source question set. The aggregation operation includes: calculating the semantic similarity of the questions in the two sets, merging question items with different expressions but pointing to the same question, deduplicating duplicate questions, and sorting them according to the severity of the questions.

[0023] Preferably, step S5 specifically includes the following steps: the structural integrity scoring is specifically as follows: assuming the structural checklist includes... Each inspection item The maximum score for each item is [number]. Candidate Report No. The score for the item is Then the structural integrity score for:

[0024] The inspection items include the existence of necessary fields, compliance of field format, complete correspondence between regions and symptoms, and correctness of structure hierarchy; The semantic consistency score is calculated based on the semantic logical consistency between the fields within the candidate report: Let the candidate report... Include fields First, extract the semantics of all field pairs. Calculate the semantic consistency matrix between fields. ,in and All are field indexes:

[0025] For field pairs that should be semantically consistent but have low actual similarity, and for field pairs that should be semantically complementary but are actually contradictory, a consistency penalty is applied respectively. The final semantic consistency score is:

[0026] in, For the set of field pairs that need to be subject to consistency penalties, The penalty weights for each field pair; The medical rationality score is calculated by comparing the candidate report with the retrieved medical knowledge: Let the set of retrieved knowledge items be... Candidate reports are The medical rationality score is then the weighted average of the maximum semantic match between the candidate report and all knowledge items:

[0027] in, The number of fields in the candidate report. A text encoder for the medical field; The scores from the three dimensions are weighted and fused according to preset weights to obtain the final comprehensive score of the candidate report: Let the weight of structural integrity be... The semantic consistency weight is The weight of medical rationality is And satisfy Then the first Candidate Reports The overall score is:

[0028] For all After calculating the comprehensive score for each candidate report, they are sorted in descending order of score:

[0029] Select the candidate report with the highest score The final output is a structured MRI report of cervical lesions. If multiple candidate reports have the same overall score and are all the highest, the candidate report with the higher structural integrity score will be selected first. If the structural integrity scores are also the same, the candidate report with the higher medical rationality score will be selected first.

[0030] A knowledge-enhanced report generation device for cervical lesion MRI images includes the following modules: The structured draft generation module preprocesses the input cervical lesion MRI images to obtain standardized image data; inputs the standardized image data into a visual-language model, extracts multi-scale visual feature representations through an image encoder; maps the visual features to the semantic space of the language model based on a cross-modal projection matrix to obtain visual-language alignment features; and uses an autoregressive text decoder to serialize and decode the visual-language alignment features to generate a structured draft containing anatomical structure fields, image sign fields, suspected abnormal area fields, and uncertainty fields. Structured Draft Standardization Module: Standardizes the structured draft, including removing negative descriptions and invalid information, establishing the correspondence between image features and anatomical regions, completing necessary structural fields, unifying the expression form and semantic granularity, and outputting the standardized structured intermediate representation; Structured representation output module: Retrieves standardized expressions and diagnostic rules related to the current image signs, anatomical structures and descriptive content from a pre-built medical knowledge base, and introduces the retrieved knowledge information as external constraints into the generation process to output a knowledge-enhanced structured representation; Structured Representation Correction Module: The structured representation after knowledge enhancement is evaluated in parallel by the rule reflection module and the large language model reflection module, forming a multi-source problem set, and automatically correcting the structured representation, outputting the corrected structured representation; The final structured report generation module generates multiple candidate structured results, scores and filters them based on structural integrity, semantic consistency and medical rationality indicators, and selects the result with the best score as the final output cervical lesion MRI structured report.

[0031] This invention offers the following advantages: It employs a layered, multi-layered optimization mechanism, providing numerous key benefits. By leveraging a visual-language model for structured extraction combined with structural constraint filtering, it achieves standardized expression of image information, overcoming the drawbacks of traditional reports' disorganized structures and arbitrary descriptions, thus improving report standardization and readability. Through negative description filtering, sign region alignment, and field completion, it reduces redundant information and anatomical location mismatches, improving the accuracy of intermediate processing results. The introduction of a medical knowledge base enhances knowledge constraints, integrates professional diagnostic rules to correct generated content, and balances data characteristics with clinical logic, enhancing the clinical rationality of the report. A multi-agent dual-reflection mechanism autonomously identifies defects such as anchoring failure and information omissions, strengthening the system's error detection capabilities; combined with automatic repair and multi-candidate optimization strategies, it iteratively optimizes the manuscript, improving the instability of single-generation results and comprehensively enhancing the accuracy, consistency, and output quality of image report generation. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0033] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention.

[0035] Example The following are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the following embodiments. All technical solutions that fall within the scope of the present invention are within the scope of protection of the present invention.

[0036] Reference manual attached Figure 1 This invention provides a method and apparatus for generating knowledge-enhanced reports from MRI images of cervical lesions, comprising the following steps: S1, Cross-modal structured information extraction based on visual-language model: The fine-tuned visual-language model is used to perform cross-modal semantic understanding on the input cervical lesion MRI images, automatically extract key elements such as anatomical structure information, visible image signs and suspected abnormal areas in the images, and organize them into a structured draft according to predefined fields. S2, Structural Consistency Constraints and Rule Filtering Mechanism: The structured draft is standardized, including removing negative descriptions and invalid information, establishing the correspondence between image features and anatomical regions, completing necessary structural fields, and unifying the expression form and semantic granularity. S3, a knowledge enhancement mechanism based on knowledge retrieval: retrieves standardized expressions and diagnostic rules related to current imaging signs, anatomical structures and descriptive content from a pre-built medical knowledge base, and introduces the retrieved knowledge information as an external constraint into the generation process; S4, a multi-agent collaborative self-reflection mechanism: through the rule reflection module and the large language model reflection module, the structured draft is evaluated in parallel to form a multi-source problem set, so as to realize the in-depth quality evaluation and automatic correction of the generated results. S5, a scoring and selection mechanism based on multiple candidate results: multiple candidate structured results are generated, and scores and filters are performed based on indicators such as structural integrity, semantic consistency and medical rationality. The result with the best score is selected as the final output cervical lesion MRI structured report.

[0037] The following provides a detailed explanation of each step.

[0038] S1, Cross-modal structured information extraction based on a vision-language model In this embodiment of the invention, the cross-modal structured information extraction based on a vision-language model includes: preprocessing the input cervical lesion MRI images to obtain standardized image data; inputting the standardized image data into a fine-tuned vision-language model and extracting multi-scale visual feature representations through an image encoder; mapping the visual features to the semantic space of the language model based on a cross-modal projection matrix to obtain vision-language alignment features; and using an autoregressive text decoder to serialize and decode the vision-language alignment features to generate a structured draft containing anatomical structure fields, image sign fields, suspected abnormal area fields, and uncertainty fields.

[0039] It should be noted that preprocessing the input cervical lesion MRI images is a fundamental operation that transforms the raw DICOM or NIfTI format medical image data into a standard form suitable for processing by the vision-language model. This preprocessing includes, but is not limited to: firstly, decomposing the raw 3D MRI volume data into a 2D slice sequence along the axial direction; then, normalizing the pixel intensity of each 2D slice, mapping the original pixel values ​​to the [0, 255] interval through a linear transformation; then, copying and expanding the single-channel grayscale image into a three-channel format to adapt to the input requirements of the pre-trained vision-language model; finally, adjusting all 2D slices to a preset fixed resolution (e.g., ...). (pixels), generating standardized image data tensors.

[0040] It should be noted that the fine-tuned visual-language model is a specialized model obtained by performing domain adaptation fine-tuning on a cervical lesion MRI image dataset based on a large-scale general visual-language pre-trained model (such as LLaVA or Qwen-VL). During the fine-tuning process, a cervical MRI image-report paired dataset was used, and the attention layer and projection layer of the model were efficiently fine-tuned using low-rank adaptation (LoRA) technology, enabling the model to master the professional visual representation of cervical anatomy and the linguistic expression patterns of medical imaging terminology.

[0041] It should be noted that the image encoder is the component in the vision-language model responsible for converting the input image into a high-dimensional visual feature representation, typically employing a Vision Transformer (ViT) or Convolutional Neural Network (CNN) architecture. When using the Vision Transformer architecture, the input image is first segmented into... Each non-overlapping image patch is transformed into a fixed-dimensional vector representation through a linear projection layer, with positional encoding information appended, and then fed into a multi-layer self-attention transformer network for processing.

[0042] Furthermore, the feature extraction process of the image encoder can be described by the following mathematical formula:

[0043] in, Indicates the first Flattened pixel vectors of an image patch For image patch embedding matrix, For the first The location encoding vector of each image patch For the embedded first Feature vectors of image patches.

[0044] Furthermore, in the multilayer self-attention transformer network, the first... The layer calculation process is as follows:

[0045]

[0046] in, For the first The output features of the layer are defined by LN (Layer Normalization), MHSA (Multi-Head Self-Attention), and FFN (Feed-Forward Network). The specific calculation expression for multi-head self-attention is as follows:

[0047]

[0048] in, , , , The first The query, key, and value projection matrix of each attention head. To output the projection matrix, Let be the dimension of the key vector. The number of attention heads.

[0049] It should be noted that the cross-modal projection matrix is ​​a crucial bridge connecting the visual encoder and the language decoder. Its function is to map visual features from the image semantic space to the language semantic space, achieving semantic alignment of cross-modal information. Specifically, the visual feature representation output by the image encoder... (in This represents the total number of layers in the visual encoder. (For visual feature dimensions) through a learnable linear projection matrix Mapped to the feature space of the language model:

[0050] in, For visual-language alignment features, This represents the feature dimension of the language model. This projection operation places visual features and text word embeddings in the same vector space, facilitating subsequent autoregressive text generation.

[0051] It should be noted that the autoregressive text decoder employs a causal language model integrated with a visual-language model (such as the decoder part of an LLaMA architecture), using visual-language alignment features as prefix prompts, and generates structured text token by token through an autoregressive approach. In the... At each generation time, the model calculates the conditional probability distribution for each candidate word in the vocabulary:

[0052] in, Let be the probability normalization function. For the first The tokens generated at each time step. For the generated word sequence, For the decoder in the The hidden state vector at time step 1. The output layer projection matrix. This is a temperature parameter used to control the randomness of the generated text. In structured report generation tasks, it is typically used... Set to a smaller value (e.g.) This is to improve the certainty of the generated results.

[0053] It should be noted that the structured draft is structured text organized according to a predefined JSON Schema format, containing the following core fields: anatomical structure field (recording the cervical-related anatomical regions visible in the images, such as the cervical stroma, endometrium, parametrial tissue, vaginal fornix, etc.), imaging features field (recording the imaging features of each anatomical region, such as T1WI signal intensity, T2WI signal intensity, enhancement features, etc.), suspected abnormal area field (recording the anatomical regions suspected of having lesions and their spatial location descriptions), and uncertainty field (recording the confidence score of the model for each judgment).

[0054] Furthermore, the generation process of the structured draft employs a constrained decoding strategy, which introduces JSON Schema grammatical constraints during the decoding process to ensure that the generated text always conforms to a predefined structured format. Specifically, at each decoding step, the set of the next valid tokens is dynamically calculated, and the probability of invalid tokens is forcibly set to zero, thereby ensuring the grammatical correctness of the structured output.

[0055] S2, Structural Consistency Constraints and Rule Filtering Mechanism In this embodiment of the invention, the structural consistency constraint and rule filtering mechanism includes: performing a negative description filtering operation on the structured draft to remove negative statements that contradict positive findings; performing a region-sign alignment verification operation to verify whether each image sign has been correctly anchored to the corresponding anatomical region; performing a field integrity check operation to automatically complete any missing necessary fields; and performing an expression normalization operation to unify terminology and semantic granularity to obtain a normalized structured intermediate representation.

[0056] It should be noted that the negative description filtering operation is a text cleansing process based on a combination of rule matching and a lightweight classifier. In medical image report generation, the visual-language model may mix in negative expressions (such as "no obvious abnormalities seen", "no obvious enhancement", "cannot be ruled out...") while generating positive descriptions due to the distribution characteristics of the training data. These negative descriptions contradict the actual positive findings and will seriously affect the clinical usability of the report.

[0057] Furthermore, the negative description filtering operation employs a two-level filtering strategy. The first level is keyword rule filtering, which performs precise matching based on a pre-built negative description keyword dictionary. This negative description keyword dictionary includes high-frequency negative words (such as "not seen", "no obvious", "not excluded", "cannot be ruled out", etc.), weakening modifiers (such as "mild", "a little", "possibly", etc.), and negative sentence templates. For the text content of each field in the structured draft, it is checked one by one whether it contains keywords or matching sentence templates from the dictionary. If a match is successful, the statement is marked as a candidate negative description.

[0058] Furthermore, the second level involves semantic classifier filtering, employing a lightweight binary classifier based on a pre-trained language model to perform fine-grained semantic-level judgments on candidate negative descriptions. The classifier's input consists of the candidate sentence and its surrounding context, and its output is the probability value that the sentence belongs to a negative description. .when (in The preset threshold is usually set to... When the statement is removed from the structured draft, the statement will be removed.

[0059] Furthermore, the judgment probability of the semantic classifier can be expressed by the following formula:

[0060] in, Candidate statements, For pre-trained BERT encoders, This is for pooling operations (usually using the hidden state marked with [CLS]). The classifier weight matrix is... For bias terms, This is the Sigmoid activation function.

[0061] It should be noted that the region-sign alignment verification operation is a crucial step in verifying the correctness of the correspondence between image sign descriptions and anatomical region annotations. In the structured draft, each image sign should clearly belong to one or more anatomical regions. If a sign lacks a corresponding region anchor, or if the anchored region is not in the predefined anatomical structure dictionary, it is considered an alignment anomaly.

[0062] Furthermore, the region-feature alignment verification operation is based on a pre-constructed anatomical structure-feature compatibility matrix. Let the predefined set of anatomical regions be... (Such as cervical stroma, endometrium, parametrial tissue, vaginal fornix, pelvic lymph nodes, etc.), the predefined set of sign types is (e.g., high signal on T2WI, signal on T1WI, significant enhancement, blurred boundaries, etc.), then the compatibility matrix Defined as: Among them, the compatibility matrix Defined as: if signs Can appear in the area If the value is 1, it is 0 otherwise.

[0063] For each sign-region pair in the structured draft Query the compatibility matrix to determine Whether it is true; if If the pair is incompatible, it will be marked as an incompatible pair, triggering a correction operation.

[0064] Furthermore, the correction operation first attempts to remap incompatible signs to compatible anatomical regions. Let the signs... The set of compatible regions is Then, the compatible region with the greatest semantic similarity is selected as the new anchoring target:

[0065] in, Anatomical region With signs Semantic similarity between them Indicates semantic similarity Anatomical region that reaches its maximum value The cosine similarity is calculated after encoding by a pre-trained language model to obtain:

[0066] in, , These are semantic embedding vectors for the region name and the feature description, respectively.

[0067] It should be noted that the field integrity check is an automated validation process that ensures the structured intermediate representation contains all necessary fields. These necessary fields include patient basic information, imaging information (sequence type, scan orientation, contrast agent usage), anatomical structure, imaging features, suspected abnormal areas, and confidence assessment. If a necessary field is detected as missing, a field completion operation is triggered.

[0068] Furthermore, the field completion operation employs a hybrid strategy based on default values ​​and contextual reasoning. For fields with standard default values ​​(such as scanning orientation, which defaults to "axis"), the default value is directly entered; for fields requiring contextual reasoning, a pre-trained language model is used to infer the reasonable value of the missing field based on the content of existing fields. For example, when the imaging sign field records "cervical stroma shows high signal on T2WI" while the suspected abnormal area field is empty, the model will infer the suspected abnormal area as "cervical stroma" based on anatomical knowledge.

[0069] S3, Knowledge Enhancement Mechanism Based on Knowledge Retrieval In this embodiment of the invention, the knowledge enhancement mechanism based on knowledge retrieval includes: vectorizing key entities (including anatomical region names and image feature descriptions) in the normalized structured intermediate representation to obtain a query feature vector; retrieving relevant standardized expressions and diagnostic rules from a pre-built medical knowledge base based on the query feature vector using a hybrid retrieval strategy (including semantic similarity retrieval and keyword enhancement retrieval); and fusing the retrieved knowledge entries with the original structured intermediate representation to obtain a knowledge-enhanced structured representation.

[0070] It should be noted that the pre-built medical knowledge base is a structured knowledge base specifically for the MRI imaging diagnosis of cervical lesions. Its content includes: standardized nomenclature and hierarchical relationships of cervical anatomical structures, descriptions of MRI signal characteristics of each anatomical region under normal and pathological conditions, imaging manifestations related to FIGO staging of cervical cancer, standardized MRI report writing templates for cervical lesions, and key points for common misdiagnosis and differential diagnosis. Each knowledge record in the knowledge base is stored in the form of a structured document and has undergone vectorized encoding processing for efficient retrieval.

[0071] It should be noted that the vectorization encoding is the process of converting key entities in text form into dense vector representations. For anatomical region names... and description of image signs The text is encoded using a pre-trained medical domain text encoder (such as PubMedBERT or a domain-adapted BERT model):

[0072] in, This indicates a splicing operation. and Special separator mark, This is the output query feature vector. This represents the output dimension of the encoder.

[0073] It should be noted that the hybrid retrieval strategy combines semantic similarity retrieval and keyword-enhanced retrieval, aiming to balance the comprehensiveness and accuracy of the retrieval. Semantic similarity retrieval, based on distance metrics in vector space, can capture the deep semantic relationships between queries and knowledge entries; keyword-enhanced retrieval, on the other hand, is based on precise keyword matching, ensuring that the search results contain key medical terms.

[0074] Furthermore, the semantic similarity retrieval uses cosine similarity as the metric. Let the first... The vector representation of a knowledge record is as follows The query vector is The semantic similarity between the two is calculated as follows:

[0075] Based on this similarity score, all entries in the knowledge base are sorted in descending order, and the entries with the highest similarity are selected. Knowledge records as a set of semantic retrieval results .

[0076] Furthermore, the keyword-enhanced retrieval employs a TF-IDF-weighted term matching scoring mechanism. Let the set of keywords contained in the query text be... The first in the knowledge base The set of keywords contained in each knowledge record is The keyword matching score is then:

[0077] in, For terms In knowledge records TF-IDF weights in For terms The pre-defined importance weights (anatomical region terms have higher weights than general descriptive terms) are used to select the preceding terms based on this score. Knowledge records as a set of keyword search results .

[0078] Furthermore, the final result of the hybrid retrieval is a weighted fusion of the two retrieval result sets:

[0079] in, The fusion weight of semantic retrieval and keyword retrieval (usually taken as...) ), For normalization operations (linearly mapping keyword scores to...) (interval) To select the top with the highest fusion score Operations on knowledge records.

[0080] It should be noted that the knowledge fusion described herein is a process of organically combining the retrieved knowledge entries with the original structured intermediate representation. For each retrieved knowledge record, key information (such as standardized expressions, diagnostic rules, and identification points) is first extracted, and then this information is embedded into the structured intermediate representation as additional fields. Specifically, for each anatomical region field and imaging sign field in the structured intermediate representation, relevant standardized descriptions and diagnostic rules are extracted from the search results and appended to the metadata of the corresponding field as "knowledge enhancement annotations".

[0081] Furthermore, the knowledge fusion process can be described by the following mathematical expression. Let the set of retrieved knowledge entries be... Structured intermediate representation is (each of which) If it is a structured field, then the fused knowledge-enhanced representation for:

[0082] in, For structured fields With knowledge entries Correlation score between them This is the relevance threshold. The relevance score is obtained by calculating the semantic similarity between the field content and the knowledge item:

[0083] It should be noted that the knowledge-enhanced structured representation not only retains the content of the original structured draft, but also adds external constraint information from the medical knowledge base. This information will play an important reference role in the subsequent self-reflection and multi-candidate scoring stages, ensuring that the generated results not only conform to the data-driven model output, but also meet the professional standards of medical knowledge-driven approaches.

[0084] S4, a self-reflection mechanism for multi-agent collaboration. In this embodiment of the invention, the multi-agent collaborative self-reflection mechanism includes: simultaneously inputting the knowledge-enhanced structured representation into a rule reflection module and a large language model reflection module; the rule reflection module, based on a preset anatomical rule base and report writing specifications, performing logical consistency verification on the structured representation and outputting a set of rule-level questions; the large language model reflection module, combined with retrieved medical knowledge, performing semantic-level quality assessment on the structured representation and outputting a set of semantic-level questions; aggregating and deduplicating the rule-level question set and the semantic-level question set to form a multi-source question set; and automatically correcting the structured representation based on the multi-source question set to obtain a corrected structured representation.

[0085] It should be noted that the rule reflection module is an expert system component based on deterministic rules. Its core function is to perform automated logical validation of structured representations through a predefined rule base. The rule base includes: anatomical integrity rules (checking whether any anatomical regions that must be described are missing, such as the cervical stroma, which must be described in a T2WI sequence), sign-region anchoring rules (checking whether each image sign has been correctly associated with its corresponding anatomical region), description consistency rules (checking whether the description of the same anatomical region is consistent in different fields), and format compliance rules (checking whether the output conforms to a predefined JSON Schema structure).

[0086] Furthermore, the verification process of the rule reflection module can be formally described as follows. Assume the rule base contains... Rules Each rule By a conditional predicate And a problem description Composition. For a given knowledge, an enhanced structured representation. Evaluate the conditional predicates of each rule one by one: Each rule By conditional predicate Problem description The output consists of the following: If the conditional predicate evaluates to the given structured representation as true, the corresponding problem description is output; otherwise, an empty set is output.

[0087] The problem descriptions of all triggering rules constitute a set of problem-level issues at the rule level. .

[0088] Furthermore, the key rules involved in the rule reflection module can be formally described using Boolean expressions. For example, the "cervical stroma cannot be omitted" rule in the anatomical integrity rule can be expressed as:

[0089] in, This is the operation for extracting the names of all anatomical regions from the structured representation. Similarly, the sign-region anchoring rule can be represented as:

[0090] It should be noted that the large language model reflection module is a semantic evaluation component based on a large-scale language model (such as GPT-4 or Qwen-72B-Chat). Its function is to perform in-depth quality evaluation of structured representations from a medical semantic perspective. Unlike the rule reflection module, the large language model reflection module can identify complex semantic problems not covered by the rule base, such as ambiguous descriptions, logical contradictions, clinical irrationality, and inappropriate terminology.

[0091] Furthermore, the large language model reflection module adopts a chain-of-thought prompting strategy to guide the model to analyze the quality of the structured representation step by step. The specific implementation method is as follows: construct a reflection prompt containing the following parts: (1) the structured representation content to be evaluated; (2) the relevant medical knowledge entries retrieved; (3) predefined reflection guidance questions (such as "Are the descriptions of each anatomical region complete?", "Do the descriptions of signs accurately reflect possible pathological changes?", "Are there any ambiguous or self-contradictory statements?"); (4) output format requirements (the model is required to output a list of problems found in a structured JSON format, with each problem including problem type, problem location, problem description and modification suggestions).

[0092] Furthermore, the semantic evaluation process of the reflection module of the large language model can be modeled using conditional probability. Let the output of the reflection model be a set of questions. Given a structured representation and retrieval knowledge The model generates the first The probability of each problem description is:

[0093] in, For decoders of large language models, This serves as a template for reflection prompts. A complete set of questions is generated through autoregressive sampling. .

[0094] It should be noted that aggregating and deduplicating the rule-level question set and the semantic-level question set output by the large language model reflection module is the process of merging two heterogeneous question sets into a unified multi-source question set. The aggregation operation includes: calculating the semantic similarity of questions in the two sets; merging question items with different expressions but pointing to the same question; deduplicating duplicate questions; and sorting them according to the severity of the questions.

[0095] Furthermore, the aggregation and deduplication process can be implemented using the following algorithm. First, semantically encode all question items: , Then, the similarity matrix between the rule-based problem and the large language model problem is calculated. :

[0096] For similarity exceeding the threshold The question is about ( These issues were then merged into a single comprehensive problem description, retaining the more detailed modification suggestions from both sources. Unmatched problem items were simply retained. The final multi-source problem set is as follows:

[0097] It should be noted that automatically correcting the structured representation based on the multi-source problem set is a process of converting problem descriptions into specific repair operations. For each problem... First, the problem type and location information are analyzed, and then the corresponding repair operation is selected according to the predefined repair strategy library. The repair strategies include: supplementing missing information (when the problem type is "information omission"), modifying the error description (when the problem type is "description error"), deleting redundant content (when the problem type is "redundant / contradictory"), and re-anchoring the region (when the problem type is "region mismatch").

[0098] Furthermore, the automatic correction operation can be implemented using a repair generator driven by a large language model. The input to the repair generator is the original structured representation. and multi-source problem set The output is the corrected structured representation. The correction process can be modeled as a conditional generation problem:

[0099] in, The correction is calculated by an autoregressive decoder (usually another large language model instance). When generating the corrected content, the correction generator is constrained to modify only the fields relevant to the problem, leaving other fields unchanged, to ensure the locality and controllability of the correction.

[0100] S5, a scoring selection mechanism based on multiple candidate results. In this embodiment of the invention, the scoring selection mechanism based on multiple candidate results includes: generating from the corrected structured representation by introducing diverse generation parameters (including different temperature parameters, Top-p sampling thresholds, and different reflective repair strategies). The report identifies 10 candidate structured reports; each candidate report is scored using a multi-dimensional quality assessment, including structural integrity, semantic consistency, and medical rationality; all candidate reports are ranked based on the weighted composite score of each dimension, and the candidate report with the highest composite score is selected as the final output cervical lesion MRI structured report.

[0101] It should be noted that the generation The operation of generating candidate structured reports is achieved by introducing controlled randomness during the generation phase. Specifically, in the text decoding phase of the visual-language model, multiple different candidate outputs are generated using different combinations of decoding parameters. The generation of candidate reports uses parameter combinations. ,in Temperature parameter controls the smoothness of the sampling distribution; The Top-p (nucleus sampling) threshold controls the truncation range of candidate terms. This is achieved by setting differentiated parameter combinations (such as...). , This can yield candidate outputs that achieve different balances between determinism and diversity.

[0102] It should be noted that the multi-dimensional quality scoring of each candidate report is a comprehensive automated evaluation process. The scoring system includes three core dimensions: structural integrity score. (Assess whether the candidate report contains all necessary fields, whether the JSON format is correct, and whether the field hierarchy is reasonable), semantic consistency score. (Assessing the semantic logic consistency between fields within the candidate report and identifying any contradictory descriptions) and medical rationality scoring. (Assess whether the medical descriptions in the candidate reports are consistent with clinical knowledge, whether the terminology is used correctly, and whether the diagnostic inferences are reasonable).

[0103] Furthermore, the structural integrity score is calculated based on a predefined structural checklist. Let the structural checklist include... Each inspection item The maximum score for each item is [number]. Candidate Report No. The score for the item is The structural integrity score is then:

[0104] The checks include the existence of necessary fields (such as whether the anatomical structure field exists, weights) ), field format compliance (such as JSON syntax correctness, weight) ), Region-Symptom Correspondence Completeness (whether each symptom has a region anchor, weight) And the correctness of the structure hierarchy (whether the nested relationships of fields conform to the schema definition, weights) )wait.

[0105] Furthermore, the semantic consistency score is calculated based on the semantic logical consistency between fields within the candidate report. Let the candidate report... Include fields First, extract the semantic embedding vectors of all field pairs. Calculate the semantic consistency matrix between fields. ,in and All are field indexes:

[0106] For field pairs that should be semantically consistent but have low actual similarity (e.g., the anatomical structure field and the suspected abnormal area field should match the anatomical area described), and for field pairs that should be semantically complementary but are actually contradictory (e.g., the imaging sign field records "significant enhancement" while the diagnostic impression field records "benign lesion"), consistency penalties are applied respectively. The final semantic consistency score is:

[0107] in, For the set of field pairs that need to be subject to consistency penalties, The penalty weights for each field pair.

[0108] Furthermore, the medical rationality score is calculated by comparing the candidate report with the retrieved medical knowledge. Let the set of retrieved knowledge items be... Candidate reports are The medical rationality score is then the weighted average of the maximum semantic match between the candidate report and all knowledge items:

[0109] in, The number of fields in the candidate report. This is a text encoder for the medical field. The score reflects the degree to which each field in the candidate report matches the most relevant knowledge entry in the medical knowledge base.

[0110] It should be noted that the weighted composite score is obtained by weighting and fusing the scores from the three dimensions according to preset weights to obtain the final composite score of the candidate report. Let the structural integrity weight be... The semantic consistency weight is The weight of medical rationality is And satisfy Then the first Candidate Reports The overall score is:

[0111] Furthermore, the values ​​of the weighting coefficients reflect the relative importance of different dimensions in the overall quality assessment. In a preferred embodiment of the invention, the structural integrity weight... Semantic consistency weight Medical rationality weight This weighting configuration ensures that the report, while first meeting structural specifications, also possesses a high degree of semantic and logical consistency and medical professional rationality.

[0112] It should be noted that selecting the candidate report with the highest overall score as the final output is a ranking-based selection process. For all... After calculating the comprehensive score for each candidate report, they are sorted in descending order of score:

[0113] Select the candidate report with the highest score The final output is a structured MRI report of cervical lesions. If multiple candidate reports have the same overall score and are all the highest, the candidate report with the higher structural integrity score is selected first; if the structural integrity scores are also the same, the candidate report with the higher medical rationality score is selected first.

[0114] It should be noted that the final output cervical lesion MRI structured report is a structured document conforming to a predefined JSON Schema format, containing a complete anatomical description, imaging feature record, annotation of suspected abnormal areas, diagnostic impression, and confidence assessment. This report can be directly reviewed, modified, and confirmed by radiologists, serving as a reference basis for formal imaging diagnostic reports.

[0115] Closed-loop optimization mechanism for the entire process In this embodiment of the invention, steps S1 to S5 constitute a closed-loop report generation process from initial generation to gradual correction and optimization. Specifically, step S1 is responsible for converting unstructured MRI image data into a preliminary structured representation; step S2 normalizes this representation, removes noise, and establishes correct structured relationships; step S3 introduces external medical knowledge for enhancement; step S4 performs in-depth quality assessment and automatic correction through a multi-agent collaborative reflection mechanism; and step S5 ensures optimal quality of the final output through a multi-candidate optimization mechanism.

[0116] It should be noted that there is bidirectional information flow between the steps of this closed-loop process. The forward flow is responsible for passing the processing results from the upstream step to the downstream step; the backward correction flow allows the downstream step (especially the self-reflection mechanism in step S4) to provide feedback on problems to the upstream step, triggering iterative correction. For example, when the rule reflection module finds a serious error in the description of an anatomical region, it will not only correct it in step S4, but also feed the problem information back to step S2, triggering a re-verification of the region-sign alignment.

[0117] Furthermore, the iterative process of the closed-loop optimization mechanism can be controlled by the following convergence condition. Let the... The structured representation after the next iteration is as follows The number of issues identified by the reflection module is The iteration termination condition is:

[0118] in, The maximum number of iterations (usually taken as...) This condition ensures that the system terminates the loop when the problem is completely solved or the maximum number of iterations is reached, thus avoiding infinite iteration.

[0119] In the several embodiments provided in this invention, it should be understood that the disclosed methods can be implemented in other ways. It will be apparent to those skilled in the art that this invention is not limited to the details of the exemplary embodiments described above, and that it can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within this invention.

[0120] In summary, this invention achieves structured expression and closed-loop optimization of the medical image report generation process by constructing a multi-stage collaborative processing mechanism of "visual understanding - structure generation - rule constraint - knowledge enhancement - self-reflection - iterative correction", which effectively improves the accuracy, consistency and clinical applicability of report generation.

[0121] The above description is merely a preferred embodiment of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial modifications made to the present invention by those skilled in the art within the scope of the technology disclosed in the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A method for generating knowledge-enhanced reports from MRI images of cervical lesions, characterized in that, Includes the following steps: S1. Preprocess the input cervical lesion MRI images to obtain standardized image data; input the standardized image data into the vision-language model, and extract multi-scale visual feature representations through the image encoder; The visual features are mapped to the semantic space of the language model based on the cross-modal projection matrix to obtain visual-language aligned features; The visual-language alignment features are serialized and decoded using an autoregressive text decoder to generate a structured draft containing anatomical structure fields, image feature fields, suspected abnormal region fields, and uncertainty fields. S2. Standardize the structured draft, including removing negative descriptions and invalid information, establishing the correspondence between image features and anatomical regions, completing necessary structural fields, unifying the expression form and semantic granularity, and outputting the standardized structured intermediate representation. S3. Retrieve standardized expressions and diagnostic rules related to the current image signs, anatomical structures and descriptive content from the pre-built medical knowledge base, and introduce the retrieved knowledge information as external constraints into the generation process to output the knowledge-enhanced structured representation; S4. The structured representation enhanced by the knowledge is evaluated in parallel by the rule reflection module and the large language model reflection module to form a multi-source problem set, and the structured representation is automatically corrected and the corrected structured representation is output. S5. Generate multiple candidate structured results, score and screen them based on structural integrity, semantic consistency and medical rationality indicators, and select the result with the best score as the final output cervical lesion MRI structured report; wherein, steps S1 to S5 constitute a closed-loop report generation process, and there is bidirectional information transmission between each step, including forward transmission flow and backward correction flow.

2. The method for generating knowledge-enhanced reports from cervical lesion MRI images according to claim 1, characterized in that, The preprocessing of the input cervical lesion MRI image in S1 specifically includes the following steps: First, the original three-dimensional MRI volume data is decomposed into a two-dimensional slice sequence along the axis; then, the pixel intensity of each two-dimensional slice is normalized, and the original pixel values ​​are mapped to the [0, 255] interval through linear transformation. Then, the single-channel grayscale image is copied and expanded into a three-channel format to adapt to the input requirements of the pre-trained vision-language model; finally, all two-dimensional slices are adjusted to a preset fixed resolution to generate a standardized image data tensor.

3. The method for generating knowledge-enhanced reports of cervical lesion MRI images according to claim 2, characterized in that, The visual-language model in S1 is a dedicated model obtained by performing domain adaptation on a cervical lesion MRI image dataset using LLaVA or Qwen-VL. The adaptation involves using a cervical MRI image-report paired dataset and adjusting the parameters of the model's attention layer and projection layer through low-rank adaptation technology, so that the model can master the professional visual representation of cervical anatomy and the linguistic expression patterns of medical imaging terminology. The image encoder adopts a Vision Transformer or convolutional neural network architecture.

4. The method for generating knowledge-enhanced reports of cervical lesion MRI images according to claim 3, characterized in that, The image encoder in S1 extracts multi-scale visual feature representations: in, The total number of image patches, Indicates the first Flattened pixel vectors of an image patch For image patch embedding matrix, For the first The location encoding vector of each image patch For the embedded first Image patch feature vectors; visual feature representations output by the image encoder. ,in This represents the total number of layers in the visual encoder. For visual feature dimensions, Given a real matrix space, through a learnable linear projection matrix Mapped to the feature space of the language model: in, For visual-language alignment features, To define the feature dimension of the language model, a projection operation is used to place the visual features and text word embeddings in the same vector space, facilitating subsequent autoregressive text generation. The autoregressive text decoder employs a causal language model integrated with the visual-language model, using visual-language alignment features as prefix cues, and generates structured text token by token through an autoregressive approach. At each generation time, the model calculates the conditional probability distribution for each candidate word in the vocabulary: in, Let be the probability normalization function. For the first The word units generated at each time point For the generated word sequence, For the decoder in the The hidden state vector at time step 1. The output layer projection matrix. The temperature parameter is used to control the randomness of the generated text. The structured draft is structured text organized according to a predefined JSON Schema format. The generation process of the structured draft adopts a constraint decoding strategy, that is, JSON Schema syntax constraints are introduced during the decoding process to ensure that the generated text always conforms to the predefined structured format.

5. The method for generating knowledge-enhanced reports from cervical lesion MRI images according to claim 1, characterized in that, The removal of negative descriptions and invalid information in S2 includes two levels of filtering: the first level is keyword rule filtering, which performs precise matching based on a pre-built negative description keyword dictionary; the negative description keyword dictionary includes: negative words, weakened modifiers, and negative sentence templates. For the text content of each field in the structured draft, it is checked one by one whether it contains keywords or matching sentence templates from the dictionary. If the match is successful, the statement is marked as a candidate negative description. The second level is semantic classifier filtering, which uses a lightweight binary classifier based on a pre-trained language model to perform fine-grained semantic-level judgment on candidate negative descriptions. The input of the classifier is the candidate sentence and its context fragment, and the output is the probability value of the sentence belonging to a negative description. ,when When that happens, remove the statement from the structured draft, where This is a preset threshold.

6. The method for generating knowledge-enhanced reports of cervical lesion MRI images according to claim 5, characterized in that, The judgment probability filtered by the semantic classifier This can be expressed by the following formula: in, Candidate statements, For pre-training Encoder, For pooling operations, The classifier weight matrix is... For bias terms, This is the Sigmoid activation function.

7. The method for generating knowledge-enhanced reports of cervical lesion MRI images according to claim 6, characterized in that, The correspondence between imaging features and anatomical regions established in S2 is based on a pre-constructed anatomical structure-feature compatibility matrix. The predefined set of anatomical regions is... The predefined set of symptom types is Then the compatibility matrix Defined as if signs Appearing in the area If the condition is met, the value is 1; otherwise, the value is 0. Represents a set of predefined anatomical regions The number of regions in This represents the number of task types in the predefined symptom type set S; For each region-symptom pair in the structured draft Query the compatibility matrix to determine Whether it is true; if If the signs are incompatible, they are marked as incompatible pairs, triggering a correction operation. This correction operation first attempts to remap the incompatible signs to compatible anatomical regions. Let the signs... The set of compatible regions is ,in It is a region index. For the set of regions in the anatomical region that exhibit the signs, select the compatible regions that have the highest semantic similarity. As a new anchor target: in, Anatomical region With signs Semantic similarity between them Indicates semantic similarity Anatomical region that reaches its maximum value The cosine similarity is calculated after encoding by a pre-trained language model to obtain: in, , These are semantic embedding vectors for the region name and the feature description, respectively.

8. The method for generating knowledge-enhanced reports of cervical lesion MRI images according to claim 1, characterized in that, The verification process of the rule reflection module in S4 includes the following steps: Assume the rule base contains... Rules Each rule By a conditional predicate And a problem description Composition, for a given knowledge, enhances the structured representation Each rule's conditional predicate is evaluated one by one: if the conditional predicate evaluates to the given structured representation, the corresponding problem description is output; otherwise, an empty set is output. The problem descriptions of all triggering rules constitute the rule-level problem set. : The above The index for the number of rules. For the first Description of the problems caused by the triggering of this rule; The semantic evaluation process of the reflection module of the large language model is modeled using conditional probability; let the output of the reflection model be a set of questions. Given knowledge-enhanced structured representation and retrieval knowledge The model generates the first The probability of each problem description is: in, For decoders of large language models, As a template for reflection prompts, For splicing operations, Describe the first to the (i-1)th problems; A complete set of questions is generated through autoregressive sampling. : Aggregating and deduplicating the rule-level question set and the semantic-level question set output by the large language model reflection module is the process of merging two heterogeneous question sets into a unified multi-source question set. The aggregation operation includes: calculating the semantic similarity of the questions in the two sets, merging question items with different expressions but pointing to the same question, deduplicating duplicate questions, and sorting them according to the severity of the questions.

9. The method for generating knowledge-enhanced reports of cervical lesion MRI images according to claim 1, characterized in that, S5 specifically includes the following steps: The structural integrity scoring is specifically as follows: Assume the structural checklist includes... Each inspection item The maximum score for each item is [number]. Candidate Report No. The score for the item is Then the structural integrity score for: The inspection items include the existence of necessary fields, compliance of field format, complete correspondence between regions and symptoms, and correctness of structure hierarchy; The semantic consistency score is calculated based on the semantic logical consistency between the fields within the candidate report: Let the candidate report... Include fields First, extract the semantics of all field pairs. Calculate the semantic consistency matrix between fields. ,in and All are field indexes: For field pairs that should be semantically consistent but have low actual similarity, and for field pairs that should be semantically complementary but are actually contradictory, a consistency penalty is applied respectively. The final semantic consistency score is: in, For the set of field pairs that need to be subject to consistency penalties, The penalty weights for each field pair; The medical rationality score is calculated by comparing the candidate report with the retrieved medical knowledge: Let the set of retrieved knowledge items be... Candidate reports are The medical rationality score is then the weighted average of the maximum semantic match between the candidate report and all knowledge items: in, The number of fields in the candidate report. A text encoder for the medical field; The scores from the three dimensions are weighted and fused according to preset weights to obtain the final comprehensive score of the candidate report: Let the weight of structural integrity be... The semantic consistency weight is The weight of medical rationality is And satisfy Then the first Candidate Reports The overall score is: For all After calculating the comprehensive score for each candidate report, they are sorted in descending order of score: Select the candidate report with the highest score The final output is a structured MRI report of cervical lesions. If multiple candidate reports have the same overall score and are all the highest, the candidate report with the higher structural integrity score will be selected first. If the structural integrity scores are also the same, the candidate report with the higher medical rationality score will be selected first.

10. A knowledge-enhanced report generation device for MRI images of cervical lesions, characterized in that, Includes the following modules: Structured draft generation module: preprocesses the input cervical lesion MRI images to obtain standardized image data; inputs the standardized image data into the vision-language model, and extracts multi-scale visual feature representations through the image encoder; The visual features are mapped to the semantic space of the language model based on the cross-modal projection matrix to obtain visual-language aligned features; The visual-language alignment features are serialized and decoded using an autoregressive text decoder to generate a structured draft containing anatomical structure fields, image feature fields, suspected abnormal region fields, and uncertainty fields. Structured Draft Standardization Module: Standardizes the structured draft, including removing negative descriptions and invalid information, establishing the correspondence between image features and anatomical regions, completing necessary structural fields, unifying the expression form and semantic granularity, and outputting the standardized structured intermediate representation; Structured representation output module: Retrieves standardized expressions and diagnostic rules related to the current image signs, anatomical structures and descriptive content from a pre-built medical knowledge base, and introduces the retrieved knowledge information as external constraints into the generation process to output a knowledge-enhanced structured representation; Structured Representation Correction Module: The structured representation after knowledge enhancement is evaluated in parallel by the rule reflection module and the large language model reflection module, forming a multi-source problem set, and automatically correcting the structured representation, outputting the corrected structured representation; The final structured report generation module generates multiple candidate structured results, scores and filters them based on structural integrity, semantic consistency and medical rationality indicators, and selects the result with the best score as the final output cervical lesion MRI structured report.

Images