AI-based medical image report quality automatic scoring system
By using multi-level similarity calculation to evaluate the quality of AI-generated medical image reports, the problem of inaccurate evaluation results in traditional methods is solved, the practicality of image reports in clinical settings is improved, and the evaluation results are ensured to meet clinical quality requirements.
Patent Information
- Application Number
- CN202511670604.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-14
- Publication Date
- 2026-02-24
AI Technical Summary
Existing methods based on traditional text similarity calculations cannot accurately assess the quality of AI-generated medical image reports, resulting in a significant gap between the assessment results and actual clinical quality requirements, thus limiting the practical value of image reports in clinical settings.
By acquiring multiple medical term entities and sentence entities, the clinical importance coefficient and entity-level similarity of medical term entities are calculated. Relational network similarity and logical flow sequence similarity are constructed, and multi-level similarity calculations are integrated to evaluate the quality of image reports.
It improves the accuracy of image report quality assessment, enhances its practical value in clinical settings, ensures that important diagnostic information is not buried in secondary descriptions, and captures the strength of clinical associations between medical term entities and the rationality of diagnostic reasoning processes.
Smart Images
Figure CN121565366A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical information technology, specifically to an AI-based automated scoring system for medical image report quality. Background Technology
[0002] With the widespread application of artificial intelligence technology in the field of medical imaging, the number of medical imaging reports generated by AI (Artificial Intelligence) has increased dramatically. How to automatically evaluate the quality of these reports has become an important need in clinical practice.
[0003] Currently, most automated assessment methods rely on traditional text similarity calculations, comparing the text overlap between AI-generated reports and expert-written reports to assess quality. However, these methods treat medical reports as ordinary text, failing to grasp the specialized medical semantics and clinical logical structure they contain. This leads to a significant gap between the assessment results and actual clinical quality requirements. Particularly when dealing with imaging reports containing complex medical entities, clinical relationship networks, and diagnostic reasoning processes, traditional text similarity methods often fail to accurately capture the key factors of imaging report quality, resulting in low accuracy of the automated assessment results and limiting the practical value of imaging reports in clinical settings. Summary of the Invention
[0004] To address the technical problem that traditional text similarity calculations for quality assessment in related technologies result in low accuracy of automated assessment results, limiting the practical value of image reports in clinical settings, this invention provides an AI-based automated scoring system for medical image report quality.
[0005] The systems used include:
[0006] The acquisition module is used to acquire multiple target image reports to be evaluated and extract multiple medical term entities and medical sentence entities from the target image reports;
[0007] The first calculation module is used to calculate the clinical importance coefficient of each medical term entity based on its frequency of occurrence and importance.
[0008] The second calculation module is used to calculate the entity-level similarity between different target image reports based on the clinical importance coefficient and the degree of local semantic consistency between different target image reports;
[0009] The third calculation module is used to calculate the relationship network similarity between different target image reports based on the association strength between different medical term entities and the standardized entity set between different target image reports;
[0010] The fourth calculation module is used to calculate the logical flow sequence similarity between different target image reports based on the semantic similarity between each medical statement entity and the number of medical statement entities in the target image report.
[0011] The integration module is used to integrate and calculate entity-level similarity, relational network similarity, and logical flow sequence similarity to obtain the comprehensive similarity between different target image reports;
[0012] The evaluation module is used to evaluate the quality of each target image report based on comprehensive similarity and preset standard reports, and obtain the quality evaluation results.
[0013] In one possible implementation of this application, the first computing module is specifically used for:
[0014] Based on preset document standards, the frequency of occurrence of each medical term entity is determined;
[0015] The frequency of occurrence is normalized to obtain the diagnostic frequency factor for each medical term entity;
[0016] The distribution of medical term entities in the target image report is analyzed and processed to obtain the clinical significance score of each medical term entity;
[0017] Based on the diagnostic frequency factor and clinical significance score, the clinical importance coefficient of each medical term entity was calculated.
[0018] In one possible implementation of this application, the second computing module is specifically used for:
[0019] Determine the intersection set of medical terminology entities shared by different target image reports, and the union set of medical terminology entities formed by the sum of all medical terminology entities among different target image reports;
[0020] Calculate the first sum of the clinical importance coefficients of the intersection set of entities, and the second sum of the clinical importance coefficients of the union set of entities;
[0021] The entity-level similarity between different target image reports is calculated by multiplying the first sum by the ratio between the degree of local semantic consistency and the second sum.
[0022] In one possible implementation of this application, the third computing module includes:
[0023] The first construction submodule is used to construct an entity association strength matrix based on the frequency of occurrence and local semantic consistency between different medical term entities. The entity association strength matrix is used to characterize the association strength between two medical term entities in the target image report.
[0024] The second construction submodule is used to construct a standardized entity set based on the union and symmetric difference sets between different target image reports;
[0025] The first calculation submodule is used to calculate the relationship network similarity between different target image reports based on the entity association strength matrix and the standardized entity set.
[0026] In one possible implementation of this application, the first construction submodule is specifically used for:
[0027] The frequency of any two medical term entities occurring simultaneously is normalized and used as the co-occurrence frequency weight between medical term entities.
[0028] Determine the semantic distance and the first semantic consistency value between any two medical term entities;
[0029] The entity association strength matrix is constructed based on the product of co-occurrence frequency weights, semantic distance, and the first semantic consistency value.
[0030] In one possible implementation of this application, the first computing submodule is specifically used for:
[0031] Determine the weighting factors for clinical relationships between different medical term entities;
[0032] Calculate the first product between the clinical relationship weight factor and the entity association strength matrix, and the second product between the clinical relationship weight factor and the square of the entity association strength matrix;
[0033] Based on the first product, the second product, and the number of entities corresponding to the standardized entity set, the relationship network similarity between different target image reports is calculated.
[0034] In one possible implementation of this application, the fourth calculation module is specifically used for:
[0035] Based on the clinical importance coefficient between any two medical statement entities, the semantic similarity between different medical statement entities is calculated.
[0036] Determine the relative positions between different medical statement entities, and then perform inverse proportional normalization on the relative positions to obtain similarity weights;
[0037] Based on the number of medical sentence entities in different target image reports, and by weighting and averaging semantic similarity and similarity weights, the logical flow sequence similarity between different target image reports is obtained.
[0038] In one possible implementation of this application, the integration module includes:
[0039] The second calculation submodule is used to calculate the first logical similarity based on the similarity of the relational network and the similarity of the logical flow sequence.
[0040] The third calculation submodule is used to calculate the third product between the first preset weight, the first logical similarity, and the relationship network similarity;
[0041] The fourth calculation submodule is used to obtain the comprehensive similarity between different target image reports based on entity-level similarity and the product between the third product.
[0042] In one possible implementation of this application, the second calculation submodule is specifically used for:
[0043] Determine the preset adjustment coefficients of the relational network on the logical flow;
[0044] Calculate the fourth product between the preset adjustment coefficient and the similarity of the relational network;
[0045] The first logical similarity is calculated based on the product between the logical flow sequence similarity and the fourth product.
[0046] In one possible implementation of this application, the evaluation module is specifically used for:
[0047] Based on comprehensive similarity, the target image reports are clustered to obtain multiple clusters;
[0048] Calculate the overall similarity between the preset standard report and each cluster, and use it as the first similarity.
[0049] The first similarity is used as the basic quality score for each cluster. Based on the basic quality score and the relative position of each target image report in its respective cluster, a quality assessment result is generated.
[0050] This application has, but is not limited to, the following technical effects:
[0051] This application addresses the core issues of traditional assessment methods by constructing a multi-level similarity calculation. Specifically, it calculates the clinical importance coefficient of each medical term entity based on its frequency and importance. Then, it calculates the entity-level similarity between different target image reports by using the clinical importance coefficient and the degree of local semantic consistency between different target image reports, thus preventing important diagnostic information from being buried in secondary descriptions. By calculating the similarity of relational networks, it captures the clinical correlation strength between medical term entities. Furthermore, by calculating the similarity of logical flow sequences, it assesses the rationality of the diagnostic reasoning process from the perspective of narrative order and semantic coherence. Finally, it integrates and processes the features of these three levels to obtain the comprehensive similarity between different target image reports. Then, it uses the comprehensive similarity and a preset standard report to conduct a quality assessment of the target image reports, improving the accuracy of similarity determination between the preset standard report and the target image reports. Based on accurately identifying the characteristic patterns of high-quality reports, it improves the accuracy of automated quality assessment and enhances the practical value of image reports in the clinical environment. Attached Figure Description
[0052] Figure 1 This is a schematic diagram of the system architecture of the AI-based automated scoring system for medical image report quality in this application.
[0053] Figure 2 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application. Detailed Implementation
[0054] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.
[0055] This application provides an AI-based automated scoring system for medical image report quality. In the first embodiment of this AI-based automated scoring system for medical image report quality, referring to... Figure 1 The system includes:
[0056] The acquisition module 10 is used to acquire multiple target image reports to be evaluated and extract multiple medical term entities and medical sentence entities from the target image reports.
[0057] As an example, an AI-based automated scoring system for medical image reports can be used to execute AI-based automated scoring methods for medical image reports. The target image reports are AI-generated medical image reports, but the quality of these reports varies, requiring quality assessment. A medical named entity recognition model (such as the NER model) is used to extract a set of medical entities from the reports, including anatomical locations, lesion types, and image features. The set of medical entities can be medical term entities, medical statement entities, etc. Among them, medical statement entities can include multiple medical term entities, and the recognition scope includes core medical concepts such as anatomical locations, lesion types, image features, and quantitative indicators.
[0058] The first calculation module 20 is used to calculate the clinical importance coefficient of each medical term entity based on the frequency and importance of its occurrence.
[0059] As an example, the clinical importance coefficient is used to represent the importance of different medical term entities to a disease. For example, the degree centrality (number of connections) or betweenness centrality (number of times it acts as a bridge) of entity nodes in a medical knowledge graph can reflect its importance. The purpose of this operation is to avoid treating two reports that both mention "pulmonary nodules" as the same similarity as two reports that both mention "pleural thickening", because the overlap of the former is more clinically significant.
[0060] The first calculation module is specifically used for:
[0061] Based on preset document standards, the frequency of occurrence of each medical term entity is determined.
[0062] The frequency of occurrence is normalized to obtain the diagnostic frequency factor for each medical term entity.
[0063] As an example, the preset document standard can be the current standard medical diagnostic guidelines and pathway documents. According to this standard, the frequency of occurrence of each medical term entity in the document / target image report is counted, and the frequency of occurrence of the medical term entity is normalized to serve as the diagnostic frequency factor for each medical term entity.
[0064] The distribution of medical term entities in the target image report was analyzed and processed to obtain the clinical significance score of each medical term entity.
[0065] Based on the diagnostic frequency factor and clinical significance score, the clinical importance coefficient of each medical term entity was calculated.
[0066] As an example, by extracting expert surveys and anomaly report distributions from the database and analyzing the distribution of medical term entities in the target image reports, a clinical significance score is obtained for each medical term entity. The clinical significance score is used to capture the perceived importance of different entities in expert experience.
[0067] As an example, taking the medical term entity 'e' as an example, the clinical importance coefficient... The calculation method can be:
[0068]
[0069] in, The diagnostic frequency factor represents the entity's role in quantifying its participation in clinical diagnostic decisions. This is because entities that frequently participate in key diagnostic decisions are considered to receive higher weights. This represents the clinical significance score, thus integrating subjective expert experience with objective statistical evidence as a calibration value, and through... The function is normalized to differentiate the clinical importance of different medical entities, thereby avoiding treating "benign and malignant lesions" the same.
[0070] The second calculation module 30 is used to calculate the entity-level similarity between different target image reports based on the clinical importance coefficient and the degree of local semantic consistency between different target image reports.
[0071] As an example, the degree of local semantic consistency can be calculated using a local semantic consistency function to represent the semantic consistency of the same entity in any two reports. If the meaning and degree of the entity are similar in the two reports, then its semantic description should also be close. If the same entity is described as "ground-glass nodule" in one report and "solid nodule" in another report, then they should not be considered completely consistent. This avoids ignoring descriptive differences simply because the entity name is the same, as these descriptive differences may change the clinical meaning.
[0072] As an example, entity-level similarity is used to represent the similarity between entities in different target image reports. The higher the entity-level similarity, the higher the deep semantic similarity between the target image reports, and the more they belong to the same type of report.
[0073] The second calculation module is specifically used for:
[0074] Determine the intersection set of medical terminology entities shared by different target image reports, and the union set of medical terminology entities formed by the sum of all medical terminology entities among different target image reports;
[0075] Calculate the first sum of the clinical importance coefficients of the intersection set of entities, and the second sum of the clinical importance coefficients of the union set of entities.
[0076] As an example, let's consider two target image reports, a and b, where the intersection set of entities is represented as... The union set of entities is represented as ,in, and These represent the entity sets in report a and report b, respectively.
[0077] The entity-level similarity between different target image reports is calculated by multiplying the first sum by the ratio between the degree of local semantic consistency and the second sum.
[0078] As an example, taking two target image reports a and b as an example, entity-level similarity The calculation method can be:
[0079]
[0080] in, This represents the clinical importance coefficient. By adjusting the weights of different entities in the similarity calculation, it is assumed that important entities (such as lung nodules) contribute more to the similarity than less important entities (such as pleural thickening). Indicates report (In different target image reports) the same medical term entity The local semantic consistency function, that is, the degree of local semantic consistency. Indicates the first sum. Indicates the second sum. This indicates that the medical term entity 'e' belongs to the intersection set of entities in the two reports. The medical term entity e represents the union of the entity sets in the two reports.
[0081] As an example, the formula captures the deep semantic similarity of the two reports at the medical entity level as much as possible, rather than just a surface match, by calculating the similarity of each medical term entity in the two reports, weighted by the clinical importance of co-occurring entities, taking into account the similarity of the context, and then normalizing by the sum of the clinical importance of all medical term entities (co-occurring and unique), so that the similarity value is between 0 and 1.
[0082] The third calculation module 40 is used to calculate the relationship network similarity between different target image reports based on the association strength between different medical term entities and the standardized entity set between different target image reports.
[0083] As an example, after establishing the semantic foundation at the entity level, the evaluation of reports needs to extend upwards to the relationships between entities. Even if two reports are highly similar at the entity level, their clinical logical integrity may differ significantly. This difference in the integrity of the relationship network directly reflects the structured quality of the report, but traditional methods and simple entity matching cannot effectively capture this. Based on this, relationship network similarity is calculated according to the entity set, importance coefficient, and the association strength between different medical term entities output by entity-level similarity. Relationship network similarity is used to evaluate the structural integrity of the internal clinical logic of the report, representing the degree of similarity of the internal clinical logic between different target image reports.
[0084] The third calculation module includes:
[0085] The first construction submodule is used to construct an entity association strength matrix based on the frequency of occurrence and local semantic consistency between different medical term entities. The entity association strength matrix is used to characterize the association strength between two medical term entities in the target image report.
[0086] As an example, local semantic consistency is used to represent the degree of semantic consistency between different medical term entities. The construction process of the entity association strength matrix is determined by a variety of factors. Here, frequency of occurrence, local semantic consistency, and semantic distance between entities are selected as the basis to construct the entity association strength matrix between different medical term entities, so as to characterize the association strength between two medical term entities in the target image report.
[0087] The first construction submodule is specifically used for:
[0088] The frequency of any two medical terms occurring simultaneously is normalized and used as the co-occurrence frequency weight between the medical terms.
[0089] As an example, the frequency of any two medical term entities appearing simultaneously in the same semantic unit (such as a diagnostic description sentence) is normalized and used as the co-occurrence frequency weight between medical term entities.
[0090] Determine the semantic distance and first semantic consistency value between any two medical term entities.
[0091] The entity association strength matrix is constructed based on the product of co-occurrence frequency weights, semantic distance, and the first semantic consistency value.
[0092] As an example, a decay function is constructed based on the semantic distance between medical word entities in a sentence, and an exponential decay model is used. ,in The semantic distance between two entities. The attenuation coefficient can be... The attenuation coefficient can be adjusted automatically. Increasing it will cause the attenuation to be too fast and widen the difference between different semantic distances. If it is too small, it will not be sensitive to changes in semantic distance. Substitute the normalized semantic distance into the calculation formula of the entity association strength matrix. The first semantic consistency value is the local semantic consistency between any two medical word entities, which can be calculated by the local semantic consistency function mentioned above.
[0093] As an example, taking medical term entities i and j as an example, the entity association strength matrix... The calculation method can be:
[0094]
[0095] in, Represents two medical terms Co-occurrence frequency weights between them This represents the normalized semantic distance between two entities. This is the first semantic consistency value, representing the differences in the local semantic environment of the entities in their respective reports.
[0096] The second construction submodule is used to construct a standardized entity set based on the union and symmetric difference sets between different target image reports.
[0097] As an example, taking target image reports a and b as examples, the standardized entity set The construction method can be:
[0098]
[0099] in, This represents the set of common entities between two target image reports, also known as the union. This indicates the number of entities in the symmetric difference set between two target image reports. = ( - )∪( - ), This is a pre-set difference entity discount factor, which can be 0.2 or 0.3, to avoid unique entities from excessively affecting similarity calculation. Similarly, the standardized entity set between other target image reports can be calculated.
[0100] The first calculation submodule is used to calculate the relationship network similarity between different target image reports based on the entity association strength matrix and the standardized entity set.
[0101] The first calculation submodule is specifically used for:
[0102] Determine the weighting factors for clinical relationships between different medical term entities.
[0103] As an example, a clinical relationship weighting factor is constructed based on the standard medical diagnostic pathway. Specifically, firstly, key relationship pairs in the core diagnostic pathway (such as lesion-location, lesion-nature) are identified and assigned higher weights; secondly, supporting descriptive relationships (such as lesion-size, lesion-number) are identified and assigned medium weights; finally, supplementary descriptive relationships (such as lesion-morphology, lesion-boundary) are identified and assigned basic weights. At this point, the higher weight, medium weight, and basic weight are set to (1.0, 0.7, 0.3) respectively. These weights can also be set by existing doctors or experts and are preset values. They can be adjusted according to actual needs.
[0104] Calculate the first product between the clinical relationship weight factor and the entity association strength matrix, and the second product between the clinical relationship weight factor and the square of the entity association strength matrix.
[0105] Based on the first product, the second product, and the number of entities corresponding to the standardized entity set, the relationship network similarity between different target image reports is calculated.
[0106] As an example, taking target image reports a and b as an example, the relationship network similarity The calculation method can be:
[0107]
[0108] in, Representing medical terms The entity association strength matrix is used to quantify the clinical association strength between two entities in the report, capturing a continuous spectrum from simple co-occurrence to deep semantic association. This is because, based on the linguistic characteristics of medical reports, the association between entities depends not only on whether they appear at the same time, but also on their grammatical roles and semantic distance in the sentence.
[0109] This represents medical term entities. The clinical relationship weighting factor between them is determined by setting the importance of different medical term entity associations to the diagnostic logic, ensuring that key clinical relationships occupy appropriate weights in similarity calculation. For example, it is believed that entity associations in the critical diagnostic path (such as nodule-malignant tumor) should receive higher weights than secondary associations (such as nodule-calcification).
[0110] N represents the number of entities corresponding to the standardized entity set. This is used to ensure the comparability of similarity calculations across reports of different sizes, and by standardizing the set size, it prevents large reports from obtaining artificially high similarity scores simply because they contain more entities. Represents the first product. This represents the second product. Similarly, the network similarity between other different target image reports can be obtained.
[0111] As an example, the current feature captures the similarity features of image reports at the clinical relationship network level through multi-level analysis from entity association strength to clinical relationship weight. The numerator calculates the inner product of the two image report relationship networks to capture the consistency of association patterns; the denominator is standardized through weighted norm product to ensure that the result is in the range of [0,1].
[0112] The fourth calculation module 50 is used to calculate the logical flow sequence similarity between different target image reports based on the semantic similarity between each medical statement entity and the number of medical statement entities in the target image report.
[0113] As an example, the quality assessment of medical reports also needs to consider the dynamic process of diagnostic deduction. Therefore, it is necessary to further construct the logical coherence similarity between different target image reports, that is, the logical flow sequence similarity. The logical flow sequence similarity is used to characterize the similarity of the target image reports in the logical order of their presentation. Logical coherence is a deep core dimension for assessing report quality. It reflects the dynamic deduction process from identifying problems, describing findings to forming conclusions. The logical flow sequence similarity in this part is the highest level in the quality assessment system. It directly depends on the results of the first two layers (entity-level similarity and relational network similarity): the semantic similarity of sentences reuses the entity-level similarity and clinical importance coefficient of the basic layer, while the association pattern implied by the sentence sequence is affected by the relational network structure assessed by the intermediate layer. The features of this layer capture the "narrative" quality of diagnostic reasoning. Only when the first two layers are solid and reliable can the assessment of this dynamic process be fully meaningful.
[0114] The fourth calculation module is specifically used for:
[0115] The semantic similarity between different medical statement entities is calculated based on the clinical importance coefficient between any two medical statement entities.
[0116] The relative positions between different medical statement entities are determined, and the relative positions are inversely normalized to obtain the similarity weights.
[0117] As an example, the way to obtain medical statement entities can be:
[0118] Sentence sequence segmentation: Each target image report is segmented into a sentence sequence according to punctuation marks such as periods and semicolons, ensuring that each sentence is a complete semantic unit. At the same time, empty sentences and sentences with only punctuation are filtered out, retaining the effective narrative content, and then the required medical sentence entities are extracted.
[0119] As an example, consider the i-th sentence obtained after sentence segmentation of target image report a. and The j-th sentence obtained after sentence segmentation For example, semantic similarity The calculation method can be:
[0120]
[0121] in, This represents the clinical importance coefficient of the medical term entity e. express Belongs to statement entity and statement entities The intersection, express Belongs to statement entity and statement entities The union of .
[0122] As an example, consider the i-th sentence obtained after sentence segmentation of target image report a. and The j-th sentence obtained after sentence segmentation For example, similarity weight The calculation method can be:
[0123]
[0124] in, This represents the location sensitivity coefficient, which is obtained as follows: Its main function is to control the strictness of sequence position matching, adapting to the comparison of sequences reported in different lengths. To train the average sentence length of the training set, an exponential decay function is used to penalize positional bias, ensuring that sentences with similar relative positions contribute more, thereby capturing the consistency of logical order. This indicates the relative positions of two sentences. When the weights are close to 1, the weights approach 1; otherwise, the weights decay exponentially. This mainly characterizes the target image report. and The number of sentence entities after sentence segmentation is normalized to ensure comparability between reports of different lengths.
[0125] Based on the number of medical sentence entities in different target image reports, and by weighting and averaging semantic similarity and similarity weights, the logical flow sequence similarity between different target image reports is obtained.
[0126] As an example, consider the i-th sentence obtained after sentence segmentation of target image report a. and The j-th sentence obtained after sentence segmentation For example, logical flow sequence similarity The calculation method can be:
[0127]
[0128] Here, i and j are natural sequence indices, primarily serving as the basic units for logical flow comparison to capture the content features of the report at the micro-semantic level. This is because sentences are the basic carriers of the narrative logic in medical reports, and the sequential relationships and semantic content between sentences directly reflect the coherence and logicality of diagnostic reasoning. This mainly characterizes the target image report. and The number of sentence entities after sentence segmentation. Indicates semantic similarity. This represents the similarity weight.
[0129] As an example, this allows for an effective assessment of logical coherence based on the sequence of report texts and the importance of entities.
[0130] The integration module 60 is used to integrate and calculate entity-level similarity, relational network similarity, and logical flow sequence similarity to obtain the comprehensive similarity between different target image reports.
[0131] As an example, by constructing a progressive evaluation feature system from "entity basis" to "relationship structure" and then to "logical flow", entity-level similarity, relationship network similarity and logical flow sequence similarity are calculated. In traditional methods, these three different levels of features are simply fused in parallel, but their inherent vertical dependencies are ignored, which may cause the evaluation results to deviate from clinical reality.
[0132] For example, even if a report has a smooth logical flow, its overall quality will be significantly reduced if its basic entity descriptions are inaccurate or key diagnostic entities are missing. Similarly, an incomplete relationship network between entities will directly affect the rationality of the logical flow.
[0133] Based on this, entity-level similarity, relational network similarity, and logical flow sequence similarity are integrated and calculated according to the vertical dependency relationship, so as to obtain the comprehensive similarity between different target image reports. The comprehensive similarity reflects the degree of similarity between different target image reports.
[0134] The integration module includes:
[0135] The second calculation submodule is used to calculate the first logical similarity based on the similarity of the relational network and the similarity of the logical flow sequence.
[0136] As an example, the effectiveness of logical flow similarity depends on the integrity of the relational network. Therefore, the first logical similarity is obtained by fine-tuning the logical flow sequence similarity through relational network similarity.
[0137] Specifically, the second calculation submodule is used for:
[0138] Determine the preset adjustment coefficients of the relational network on the logical flow;
[0139] Calculate the fourth product between the preset adjustment coefficient and the similarity of the relationship network;
[0140] The first logical similarity is calculated based on the product between the logical flow sequence similarity and the fourth product.
[0141] As an example, the preset adjustment coefficient is the adjustment coefficient of the relational network on the logical flow, which can be set to 0.2, and there is no specific limitation.
[0142] As an example, take target image report a and For example, the first logical similarity The calculation method can be:
[0143]
[0144] in, Indicates target image report a and Logical flow sequence similarity between them This indicates the preset adjustment coefficient. Indicates the similarity of a relational network. This represents the fourth product.
[0145] The third calculation submodule is used to calculate the third product between the first preset weight, the first logical similarity, and the relationship network similarity.
[0146] The fourth calculation submodule is used to obtain the comprehensive similarity between different target image reports based on entity-level similarity and the product between the third product.
[0147] As an example, take target image report a and For example, overall similarity The calculation method can be:
[0148]
[0149] in, Indicates target image report a and Entity-level similarity between them Indicates target image report a and Relationship network similarity This represents the first preset weight, which serves as the overall enhancement coefficient. It can be 0.3, 0.4, etc., and there is no specific limitation. Indicates the first logical similarity. This represents the third product.
[0150] As an example, by using the entity foundation (S) as the core cornerstone and directly as the product basis, its fundamental position in quality assessment is established. Any defects at the entity level will directly affect the final quality score. The relational structure (G) serves as an intermediate bridge, not only directly participating in quality enhancement but also influencing the effectiveness of the logical flow through moderating factors. The final logical process (H) directly serves as the top-level representation, and its effect is constrained by the integrity of the relational network, reflecting the clinical understanding that "a logical process lacking good relational support has limited value."
[0151] The evaluation module 70 is used to evaluate the quality of each target image report based on the comprehensive similarity and the preset standard report, and obtain the quality evaluation results.
[0152] As an example, multiple clusters are obtained by clustering each target image report based on comprehensive similarity. Then, each cluster is scored based on a preset standard report to obtain the quality assessment result.
[0153] The evaluation module is specifically used for:
[0154] Based on comprehensive similarity, the target image reports are clustered to obtain multiple clusters;
[0155] Calculate the overall similarity between the preset standard report and each cluster, and use it as the first similarity.
[0156] As an example, a hierarchical clustering algorithm is used to stratify the quality of target image reports. Through a bottom-up aggregation strategy, the reports with the highest similarity are continuously merged to form a cluster structure with clear quality hierarchy, resulting in multiple clusters. During the clustering process, we set a dynamic clustering termination condition: when the average similarity within a cluster is lower than a preset threshold or the separation between clusters reaches a stable state, the merging stops, ensuring that the reports within each cluster have consistent quality characteristics.
[0157] As an example, after clustering is completed, the pre-defined standard report (high-quality report) annotated by experts is used as the gold standard. The comprehensive similarity between each cluster center and the gold standard cluster is calculated to obtain the first similarity, and the first similarity is used as the basic quality score of the cluster.
[0158] The first similarity is used as the basic quality score for each cluster. Based on the basic quality score and the relative position of each target image report in its respective cluster, a quality assessment result is generated.
[0159] As an example, for the quality assessment of a single report, the quality score of its cluster and its relative position in the cluster are taken into account to generate a standardized quality score of 0-100, thus obtaining the quality assessment result.
[0160] This application provides an AI-based automated scoring system for medical image report quality. By constructing a multi-level similarity calculation, this application addresses the core problems faced by traditional assessment methods. Specifically, based on the frequency and importance of medical term entities, it calculates the clinical importance coefficient of each medical term entity. Then, using the clinical importance coefficient and the degree of local semantic consistency between different target image reports, it calculates the entity-level similarity between different target image reports, preventing important diagnostic information from being buried in secondary descriptions. By calculating relational network similarity, it captures the clinical association strength between medical term entities. Furthermore, by calculating logical flow sequence similarity, it assesses the rationality of the diagnostic reasoning process from the perspective of narrative order and semantic coherence. Finally, it integrates and processes these three levels of features to obtain a comprehensive similarity between different target image reports. The comprehensive similarity and a preset standard report are then used to assess the quality of the target image reports, improving the accuracy of similarity determination between the preset standard report and the target image reports. Based on accurately identifying the characteristic patterns of high-quality reports, this system enhances the accuracy of automated quality assessment and improves the practical value of image reports in clinical settings.
[0161] Reference Figure 2 , Figure 2 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application.
[0162] like Figure 2 As shown, the AI-based automated scoring device for medical image report quality may include: a processor 1001, a memory 1003, and a communication bus 1002. The communication bus 1002 is used to enable communication between the processor 1001 and the memory 1003.
[0163] Optionally, the AI-based automated medical image report quality scoring device may also include a user interface, a network interface, a camera, RF (Radio Frequency) circuitry, sensors, a WiFi module, etc. The user interface may include a display screen and an input submodule such as a keyboard; optional user interfaces may also include standard wired or wireless interfaces. The network interface may include standard wired or wireless interfaces (such as a Wi-Fi interface).
[0164] Those skilled in the art will understand that Figure 2 The structure of the AI-based automated medical image report quality scoring device shown in the figure does not constitute a limitation on the AI-based automated medical image report quality scoring device. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0165] like Figure 2 As shown, the memory 1003, serving as a storage medium, may include an operating system, a network communication module, and an AI-based automated medical image report quality scoring program. The operating system is a program that manages and controls the hardware and software resources of the AI-based automated medical image report quality scoring device, supporting the operation of the AI-based automated medical image report quality scoring program and other software and / or programs. The network communication module is used to enable communication between the various components within the memory 1003, as well as communication with other hardware and software in the AI-based automated medical image report quality scoring system.
[0166] exist Figure 2 In the AI-based automated scoring device for medical image report quality shown, the processor 1001 is used to execute the AI-based automated scoring program for medical image report quality stored in the memory 1003 to implement the steps of the AI-based automated scoring system for medical image report quality described above.
[0167] The specific implementation method of the AI-based automated scoring device for medical image report quality in this application is basically the same as the embodiments of the AI-based automated scoring system for medical image report quality described above, and will not be repeated here.
[0168] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0169] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0170] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0171] The above are merely preferred embodiments of this application and do not limit the scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the scope of protection of this application.
[0172] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0173] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.
Claims
1. An AI-based automated scoring system for medical image report quality, characterized in that, The system includes: The acquisition module is used to acquire multiple target image reports to be evaluated, and extract multiple medical term entities and medical sentence entities from the target image reports; The first calculation module is used to calculate the clinical importance coefficient of each medical term entity based on the frequency and importance of the occurrence of the medical term entities; The second calculation module is used to calculate the entity-level similarity between different target image reports based on the clinical importance coefficient and the degree of local semantic consistency between different target image reports; The third calculation module is used to calculate the relationship network similarity between different target image reports based on the association strength between different medical term entities and the standardized entity set between different target image reports; The fourth calculation module is used to calculate the logical flow sequence similarity between different target image reports based on the semantic similarity between each of the medical statement entities and the number of statement entities of the medical statement entities in the target image report; The integration module is used to integrate and calculate the entity-level similarity, relational network similarity, and logical flow sequence similarity to obtain the comprehensive similarity between different target image reports; The evaluation module is used to evaluate the quality of each target image report based on the comprehensive similarity and the preset standard report, and obtain the quality evaluation result.
2. The AI-based automated scoring system for medical image report quality as described in claim 1, characterized in that, The first calculation module is specifically used for: Based on preset document standards, the frequency of occurrence of each of the aforementioned medical term entities is determined; The frequency of occurrence is normalized to obtain the diagnostic frequency factor for each medical term entity; The distribution of medical term entities in the target image report is analyzed and processed to obtain the clinical significance score of each medical term entity; Based on the diagnostic frequency factor and the clinical significance score, the clinical importance coefficient of each of the medical term entities is calculated.
3. The AI-based automated scoring system for medical image report quality as described in claim 1, characterized in that, The second calculation module is specifically used for: Determine the intersection set of medical terminology entities shared by different target image reports, and the union set of medical terminology entities formed by the sum of all medical terminology entities among different target image reports; Calculate the first sum of the clinical importance coefficients of the intersection set of entities and the second sum of the clinical importance coefficients of the union set of entities; The entity-level similarity between different target image reports is calculated based on the product of the first sum and the ratio between the local semantic consistency degree and the second sum.
4. The AI-based automated scoring system for medical image report quality as described in claim 1, characterized in that, The third computing module includes: The first construction submodule is used to construct an entity association strength matrix based on the frequency of occurrence and local semantic consistency between different medical term entities. The entity association strength matrix is used to characterize the association strength between two medical term entities in the target image report. The second construction submodule is used to construct a standardized entity set based on the union and symmetric difference sets between different target image reports; The first calculation submodule is used to calculate the relationship network similarity between different target image reports based on the entity association strength matrix and the standardized entity set.
5. The AI-based automated scoring system for medical image report quality as described in claim 4, characterized in that, The first construction submodule is specifically used for: The frequency of any two medical term entities occurring simultaneously is normalized and used as the co-occurrence frequency weight between the medical term entities. Determine the semantic distance and the first semantic consistency value between any two medical term entities; The entity association strength matrix is constructed based on the product of the co-occurrence frequency weight, semantic distance, and the first semantic consistency value.
6. The AI-based automated scoring system for medical image report quality as described in claim 4, characterized in that, The first calculation submodule is specifically used for: Determine the weighting factors for clinical relationships between different medical term entities; Calculate the first product between the clinical relationship weight factor and the entity association strength matrix, and the second product between the clinical relationship weight factor and the square of the entity association strength matrix; Based on the first product, the second product, and the number of entities corresponding to the standardized entity set, the relationship network similarity between different target image reports is calculated.
7. The AI-based automated scoring system for medical image report quality as described in claim 1, characterized in that, The fourth calculation module is specifically used for: Based on the clinical importance coefficient between any two medical statement entities, the semantic similarity between different medical statement entities is calculated. Determine the relative positions between different medical statement entities, and perform inverse proportional normalization on the relative positions to obtain similarity weights; Based on the number of medical statement entities in different target image reports, and by weighting and averaging the semantic similarity and similarity weights, the logical flow sequence similarity between different target image reports is obtained.
8. The AI-based automated scoring system for medical image report quality as described in claim 1, characterized in that, The integration module includes: The second calculation submodule is used to calculate the first logical similarity based on the relationship network similarity and the logical flow sequence similarity; The third calculation submodule is used to calculate the third product between the first preset weight, the first logical similarity, and the relationship network similarity; The fourth calculation submodule is used to obtain the comprehensive similarity between different target image reports based on the product between the entity-level similarity and the third product.
9. The AI-based automated scoring system for medical image report quality as described in claim 8, characterized in that, The second calculation submodule is specifically used for: Determine the preset adjustment coefficients of the relational network on the logical flow; Calculate the fourth product between the preset adjustment coefficient and the similarity of the relationship network; The first logical similarity is calculated based on the product between the logical flow sequence similarity and the fourth product.
10. The AI-based automated scoring system for medical image report quality as described in claim 1, characterized in that, The evaluation module is specifically used for: Based on the comprehensive similarity, the target image reports are clustered to obtain multiple clusters. Calculate the overall similarity between the preset standard report and each of the aforementioned clusters, and use this as the first similarity. The first similarity is used as the basic quality score for each cluster. Based on the basic quality score and the relative position of each target image report in its respective cluster, a quality assessment result is generated.