Image report analysis method, device, equipment and medium

Abstract

The invention discloses an image report analysis method and device, equipment and a medium, and relates to the technical field of image analysis, and the method comprises the steps: receiving a plurality of unstructured medical image reports arranged by the same user according to a time sequence; and performing natural language processing on each unstructured medical image report by using a preset domain medical ontology to generate a corresponding structured data instance, performing longitudinal fusion analysis on each structured data instance according to a time sequence, and performing longitudinal fusion analysis on the same clinical discovery entity of the structured data instances at different time points by associating the same clinical discovery entity of the structured data instances at different time points. Generating a time sequence index for describing disease changes and an overall state evaluation result, and obtaining a longitudinal fusion analysis result; and integrating the longitudinal fusion analysis result to obtain a dynamic change track representing the disease change of the user. The same entity is associated across time, and the dynamic index is automatically calculated. And the evolution trend of the illness state can be seen clearly.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, and in particular to an image report analysis method, apparatus, device, and medium. Background Technology

[0002] Pancreatic cancer is one of the most malignant tumors, ranking among the leading causes of death for all cancers. For patients undergoing surgical resection, the risk of postoperative recurrence and metastasis is extremely high; therefore, close monitoring through regular computed tomography (CT) imaging is a core aspect of clinical management. Radiologists review the images and then write imaging reports in a free-text narrative format. While this report format aligns with human physician communication habits, its unstructured and non-standardized nature makes it difficult for computer systems to automatically parse and utilize key clinical information (tumor size, vascular invasion, lymph node status, etc.). This presents a significant data barrier for large-scale clinical research, treatment efficacy evaluation, and medical quality control.

[0003] To address the structuring challenges of medical texts, applying natural language processing (NLP) techniques to the structuring of radiology reports has become a recognized technological direction. Early solutions primarily relied on predefined structured templates, rule-based systems based on medical dictionaries, or dictionary matching methods based on named entity recognition. While effective in specific scenarios, these methods exhibit limited robustness and generalization capabilities when faced with the diversity and complexity of language expressions. Although some dedicated information extraction systems exist for oncology, they mostly focus on extracting relatively static information from pathology reports or initial diagnostic documents, such as cancer stage, histological type, and differentiation grade. They are not specifically designed for the dynamic and continuous monitoring scenario of postoperative imaging follow-up. Furthermore, they suffer from significant technological gaps in deeply and longitudinally structuring text reports describing the outcomes of these diagnostic and treatment processes. They lack the ability to accurately understand postoperative anatomical changes, the capacity for longitudinal tracking and quantitative analysis of key oncology information, and the difficulty in encoding complex clinical logic. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide an image report analysis method, apparatus, device, and medium capable of transforming a series of isolated text reports into a user's disease evolution history. The specific solution is as follows:

[0005] In a first aspect, this application discloses an image report analysis method applied to a computer device, comprising:

[0006] Receive multiple unstructured medical image reports from the same user, arranged in time sequence;

[0007] Each unstructured medical image report is processed using natural language to generate a corresponding structured data instance. The predefined medical ontology defines the anatomical structure concept, clinical discovery concept, and the relationship between these concepts in the target medical scenario.

[0008] The structured data instances are subjected to longitudinal fusion analysis according to time series, so as to generate time series indicators describing disease changes and overall status assessment results by associating the same clinical discovery entity of the structured data instances at different time points, so as to obtain the longitudinal fusion analysis results.

[0009] The longitudinal fusion analysis results are integrated to obtain the dynamic trajectory of changes in the user's disease.

[0010] Optionally, the step of performing natural language processing on each of the unstructured medical image reports using a preset domain medical ontology to generate corresponding structured data instances includes:

[0011] Natural language processing is performed on each of the unstructured medical image reports using a predefined domain medical ontology to extract medical entities, medical attributes, and relationships to obtain initial extraction results.

[0012] Using a preset clinical knowledge verification layer, the initial extraction results are verified and corrected based on clinical rules to obtain structured data instances determined based on the corrected extraction results; the verification and correction include postoperative anatomical correction of anatomical structure concepts based on the user's surgical history.

[0013] Optionally, the structured data instance determined based on the corrected extraction result includes:

[0014] The corrected extraction results are mapped and filled into the hierarchical fields corresponding to the medical entities to generate a structured data instance of the unstructured medical image report.

[0015] Optionally, the step of using a preset clinical knowledge verification layer to verify and correct the initial extraction results based on clinical rules includes:

[0016] The initial extraction results are verified and corrected based on clinical rules using a preset clinical knowledge verification layer according to a preset priority verification order; wherein the preset priority verification order is a verification and correction order of preset first priority, preset second priority, preset third priority, and preset fourth priority.

[0017] The preset first priority is used to perform negation word and uncertainty detection;

[0018] The preset second priority is used to perform postoperative anatomical correction of the anatomical structure concept based on the user's surgical history;

[0019] The preset third priority is used to perform clinically appropriate examinations;

[0020] The preset fourth priority is used to perform cross-field logical consistency checks.

[0021] Optionally, the longitudinal fusion analysis of each of the structured data instances according to time series, to generate time-series indicators describing disease changes and overall status assessment results by associating the same clinical discovery entity of the structured data instances at different time points, to obtain the longitudinal fusion analysis results, includes:

[0022] The clinical findings entities in each of the structured data instances are matched and linked across time with the corresponding clinical findings entities in historical reports according to the time series, so as to obtain the corresponding linking results.

[0023] Based on the linking results, the change in the time-series indicators of the clinical discovery entity over time is calculated;

[0024] Based on the amount of change, the overall state of the disease change is determined to obtain an overall state assessment result;

[0025] The longitudinal fusion analysis results are generated based on the time-series indicators and the overall state assessment results.

[0026] Optionally, the step of matching and linking the clinical findings entities in each of the structured data instances with the corresponding clinical findings entities in historical reports across time according to the time series to obtain the corresponding linking results includes:

[0027] Based on the time series, the clinical discovery entities in each of the structured data instances are weighted feature similarity scores with the corresponding clinical discovery entities in historical reports to obtain the corresponding link results;

[0028] The weighted features include: anatomical location, entity type, and morphological description.

[0029] Optionally, integrating the longitudinal fusion analysis results to obtain a dynamic trajectory characterizing changes in the user's disease includes:

[0030] The structured information of single reports and the cross-report longitudinal analysis results from the longitudinal fusion analysis results were integrated to obtain the dynamic trajectory of disease changes in user image reports.

[0031] Secondly, this application discloses an image report analysis device applied to a computer device, comprising:

[0032] The report receiving module is used to receive multiple unstructured medical image reports from the same user, arranged in a time sequence.

[0033] The instance generation module is used to perform natural language processing on each of the unstructured medical image reports using a preset domain medical ontology to generate corresponding structured data instances. The preset domain medical ontology defines the anatomical structure concept, clinical discovery concept, and the relationship between the concepts in the target medical scenario.

[0034] The instance fusion module is used to perform longitudinal fusion analysis on each of the structured data instances according to the time series, so as to generate time series indicators describing disease changes and overall status assessment results by associating the same clinical discovery entity of the structured data instances at different time points, so as to obtain the longitudinal fusion analysis results.

[0035] The results generation module is used to integrate the longitudinal fusion analysis results to obtain a dynamic trajectory characterizing the changes in the user's disease.

[0036] Thirdly, this application discloses an electronic device, including:

[0037] Memory, used to store computer programs;

[0038] A processor is used to execute the computer program to implement the steps of the aforementioned disclosed image report analysis method.

[0039] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed image report analysis method.

[0040] As can be seen, this application discloses receiving multiple unstructured medical image reports from the same user, arranged in a time sequence; using a pre-defined domain medical ontology to perform natural language processing on each of the unstructured medical image reports to generate corresponding structured data instances, wherein the pre-defined domain medical ontology defines anatomical structure concepts, clinical discovery concepts, and the relationships between these concepts in the target medical scenario; performing longitudinal fusion analysis on each of the structured data instances according to the time sequence, so as to generate time-series indicators describing disease changes and overall state assessment results by associating the same clinical discovery entity of the structured data instances at different time points, thereby obtaining longitudinal fusion analysis results; integrating the longitudinal fusion analysis results to obtain the dynamic trajectory of the user's disease changes. Thus, by accurately identifying and distinguishing postoperative concepts through a pre-defined domain medical ontology with domain-specific knowledge, serious clinical misjudgments caused by a lack of anatomical knowledge are avoided, greatly improving the accuracy and clinical credibility of the structured results, and automatically calculating dynamic indicators by associating the same entity across time. It allows users to clearly see the evolution of the disease without manually comparing multiple reports or performing logical reasoning, outputting a high-level, quantitative dynamic trajectory. This provides strong objective evidence, improving the efficiency and accuracy of decision-making. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0042] Figure 1 This is a flowchart of an image report analysis method disclosed in this application;

[0043] Figure 2 This application discloses a workflow diagram of a hybrid NLP engine.

[0044] Figure 3 This application discloses a flowchart of a vertical reporting fusion process.

[0045] Figure 4 This is a flowchart of a specific image report analysis method disclosed in this application;

[0046] Figure 5 This is a schematic diagram of the structure of an image report analysis device disclosed in this application;

[0047] Figure 6 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0049] Pancreatic cancer is one of the most malignant tumors, ranking among the leading causes of death for all cancers. For patients undergoing surgical resection, the risk of postoperative recurrence and metastasis is extremely high; therefore, close monitoring through regular computed tomography (CT) imaging is a core aspect of clinical management. Radiologists review the images and then write imaging reports in a free-text narrative format. While this report format aligns with human physician communication habits, its unstructured and non-standardized nature makes it difficult for computer systems to automatically parse and utilize key clinical information (tumor size, vascular invasion, lymph node status, etc.). This presents a significant data barrier for large-scale clinical research, treatment efficacy evaluation, and medical quality control.

[0050] To address the challenge of structuring medical texts, applying natural language processing (NLP) techniques to the structuring of radiology reports has become a recognized technological direction. Early solutions primarily relied on predefined structured templates, rule-based systems based on medical dictionaries, or dictionary matching methods based on named entity recognition. While these methods are effective in specific scenarios, their robustness and generalization capabilities are limited when faced with the diversity and complexity of language expressions.

[0051] In recent years, with the development of machine learning, especially deep learning technology, models based on conditional random fields or Transformer architectures have been used for information extraction, significantly improving the ability to handle language variations.

[0052] However, these general-purpose radiology reporting structured techniques reveal fundamental shortcomings when dealing with the highly specialized clinical scenario of post-pancreatic cancer surgery:

[0053] 1. Limitations of Anatomical Knowledge: Existing systems are typically trained based on standard human anatomy. Radical pancreatic resection (such as pancreaticoduodenectomy) significantly alters the anatomical structures within the abdominal cavity. General-purpose models cannot recognize postoperative-specific concepts such as "operating table," "anastomosis," and "metal clips," often misinterpreting normal postoperative changes as abnormalities or failing to accurately locate the anatomical position of recurrent lesions, leading to serious clinical errors in structured results.

[0054] 2. Lack of temporal correlation analysis: Existing methods typically treat each report as an independent, static data point. They cannot automatically correlate multiple reports from the same patient at different time points, thus failing to determine whether a lesion is "new," "stable," or "enlarging," nor can they calculate its rate of change. However, in oncology assessment, this longitudinal dynamic change is the core basis for judging disease progression, stabilization, or remission, and is crucial for clinical decision-making.

[0055] 3. Inability to understand complex oncology concepts: Pancreatic cancer assessment involves many complex clinical concepts, such as the "surrounding" or "adjacent" relationship of the tumor to important blood vessels (e.g., superior mesenteric artery, portal vein), signs of minor peritoneal seeding metastasis, or overall assessment based on criteria for evaluating the efficacy of treatment for solid tumors. General NLP (Natural Language Processing) models lack this depth of domain knowledge and struggle to accurately extract and encode this information, which is crucial for prognosis and treatment selection.

[0056] Although some information extraction systems exist specifically for the field of oncology, most of them focus on extracting relatively static information from pathology reports or initial diagnostic documents, such as cancer stage, histological type, and differentiation grade. They are not specifically designed for the dynamic and continuous monitoring scenario of postoperative imaging follow-up. Furthermore, they have significant technological gaps in the in-depth and longitudinal structuring of text reports describing the results of these diagnostic and treatment processes. They cannot accurately understand the anatomical structures changed after surgery, lack the ability to longitudinally track and quantify key oncology information, and have the deficiency of being unable to encode complex clinical logic.

[0057] To this end, the present invention provides an image report analysis scheme that can transform a series of isolated text reports into a user's disease evolution history.

[0058] Reference Figure 1 As shown, this invention discloses an image report analysis method applied to a computer device, comprising:

[0059] Step S11: Receive multiple unstructured medical image reports from the same user, arranged in time series.

[0060] In this embodiment, a sequence of unstructured, free-text postoperative CT (Computed Tomography) reports for pancreatic cancer, sent in a time sequence from the same user through a hospital information system or a radiology information system, is received to obtain multiple unstructured medical image reports. The input format of the unstructured medical image reports can be plain text, document, or conform to medical information standards such as HL7 (Health Level Seven), and there are no specific limitations on this.

[0061] Step S12: Perform natural language processing on each of the unstructured medical image reports using a preset domain medical ontology to generate corresponding structured data instances. The preset domain medical ontology defines the anatomical structure concept, clinical discovery concept, and the relationship between these concepts in the target medical scenario.

[0062] In this embodiment, the input unstructured medical image report is subjected to standardized preprocessing, including but not limited to: sentence boundary detection, word segmentation, and report chapter recognition. For example, different parts such as "clinical history", "image findings", and "diagnostic opinion" are automatically distinguished to obtain a preprocessed unstructured medical image report.

[0063] In this embodiment, a preset domain medical ontology is used to perform natural language processing on each of the unstructured medical image reports to extract medical entities, medical attributes, and relationships, resulting in an initial extraction result. A preset clinical knowledge verification layer is then used to validate and correct the initial extraction result based on clinical rules, resulting in a structured data instance determined based on the corrected extraction result. The validation and correction include postoperative anatomical corrections to the anatomical structure concept based on the user's surgical history. Figure 2As shown, the preprocessed unstructured medical image report is input into a hybrid NLP engine, which is a preset domain medical ontology. The hybrid NLP engine contains two sub-modules, one of which is a deep learning extractor: it uses a Transformer base model (such as BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical TextMining)) that has been fine-tuned on a large number of pancreatic cancer image reports. The deep learning extractor performs named entity recognition and relation extraction tasks. Its purpose is to identify core medical concepts in preprocessed unstructured medical image reports, such as "lesion," "lymph node," and "pancreatic duct," as well as their attributes (e.g., "size," "location," "enhancement level") and interrelationships (e.g., "located in," "adjacent"). The second layer is the clinical knowledge validation layer, which receives the initial output of the deep learning extractor. Because the deep learning extractor identifies medical concepts from the input text, it may not fully understand the specific clinical context, producing preliminary but potentially erroneous extraction results. Therefore, it needs to be input into the clinical knowledge validation layer, which uses the pancreatic cancer post-operative ontology as a knowledge base containing the user's surgical history information. This knowledge base performs logical and medical corrections to these preliminary results, performing a series of post-processing verifications. This layer is responsible for handling negative and uncertain descriptions, resolving entity ambiguities, and verifying and correcting the extraction results according to clinical logic rules to ensure the medical accuracy of the final output.

[0064] In this embodiment, the ontology-driven structured template after pancreatic cancer surgery is a dynamic template. Traditional methods based on fixed templates are too rigid and cannot adapt to the complexity and individual differences in postoperative imaging of pancreatic cancer. Therefore, this invention proposes an ontology-based, dynamic, hierarchical structured data model, specifically designed for the clinical scenario after pancreatic cancer surgery. This ontology is not only a list of data fields but also a machine-readable knowledge model that defines various medical concepts, their attributes, and the logical relationships and constraints that must be followed between them. The construction of this ontology integrates the radiological vocabulary system and oncology staging standards. Compared with traditional static templates, the core advantage of this ontology lies in its "driving" and "constraining" capabilities. During information extraction, the ontology provides context and prior knowledge for the natural language processing engine. For example, when the engine identifies the entity "superior mesenteric artery," the ontology limits its possible relationships to predefined ranges such as "clear," "adjacent," and "encircling," thereby guiding the model to make more accurate judgments and avoiding clinically meaningless associations. The "dynamic" characteristic of the ontology in this invention is reflected in its scalability and maintainability. This ontology is not static but designed as a knowledge base that can be continuously updated and improved by domain experts (such as oncologists or radiologists) through a dedicated interface. When new imaging findings, diagnostic terms, or assessment criteria emerge in clinical practice, experts can add new concepts, attributes, or relationships to the ontology. This dynamic maintenance mechanism ensures that the system can adapt to evolving medical knowledge, maintaining its long-term effectiveness and accuracy—something that static templates or hard-coded rule systems cannot achieve. Furthermore, the ontology's design inherently supports longitudinal analysis. Fields such as ChangeFromPrior (change compared to the previous time) and IsNew (whether it is a new occurrence) are not directly populated by the user or in a single extraction process but must be calculated by the "longitudinal fusion engine" of this invention before they can be generated. This demonstrates that this invention is a complete system with tightly integrated modules, rather than a simple data form design. Table 1 shows an example of the structured report ontology hierarchy after pancreatic cancer surgery.

[0065] Table 1. Example of ontology hierarchy in structured reports after pancreatic cancer surgery

[0066]

[0067]

[0068] In this embodiment, the step of using a preset clinical knowledge verification layer to verify and correct the initial extraction results based on clinical rules includes: using the preset clinical knowledge verification layer to verify and correct the initial extraction results based on clinical rules according to a preset priority verification order; wherein, the preset priority verification order is a verification and correction order of preset first priority, preset second priority, preset third priority, and preset fourth priority; the preset first priority is used to perform negative word and uncertainty detection; the preset second priority is used to perform postoperative anatomical correction of anatomical structure concepts based on the user's surgical history; the preset third priority is used to perform clinical rationality checks; and the preset fourth priority is used to perform cross-field logical consistency checks. It can be understood that the deep learning extractor outputs candidate entities, attributes, and relationships, and attaches a confidence score to each extraction result. Subsequently, the clinical knowledge verification layer receives these high-confidence candidate results and applies a rule set with clear priorities and execution order for verification. The clinical knowledge verification layer employs a multi-stage, ordered screening method to execute the rules, ensuring the rigor of the logic and the accuracy of the results. The execution of the rules strictly follows the following priority order:

[0069] 1. First priority: Basic semantic correction. This stage deals with the most basic linguistic phenomena.

[0070] Negation and uncertainty detection: First, rules are applied to identify and bind negation words (such as "not seen" or "none") and uncertainty descriptions (such as "suspected" or "possible"), marking the status of the corresponding entity as "negative" or "uncertain". This is the basis for all subsequent judgments.

[0071] 2. Second priority: Postoperative anatomical correction based on surgical history. This is a key step that distinguishes this invention from the general model, as it uses individualized patient information to correct the extracted results.

[0072] Anatomical entity replacement: The system first retrieves the patient's surgical history from the PatientInfo.SurgicalProcedure field. Then, based on the "surgical-anatomical mapping rules" defined in the ontology, the extracted anatomical entities are corrected. For example, the rule is: if SurgicalProcedure is 'Whipple', and the deep learning extractor identifies the entity 'pancreatic head', then the entity is automatically corrected to 'pancreatic head region operating table'.

[0073] Identification of normal postoperative changes: If an entity is described as a “metal clip” or “suture granuloma” and its location is close to the expected anastomosis location for this type of surgery (based on ontology knowledge), it is classified as a “normal postoperative change” rather than a “suspected recurrence.”

[0074] Anatomical impossibility exclusion: If the Surgical Procedure is “distal pancreatectomy” and the extractor identifies a lesion at the “tail of the pancreas”, mark the finding as a high-risk error and request manual review, because the anatomical structure has theoretically been removed.

[0075] 3. Third priority: Clinical rationality check. After anatomical correction, the rationality of the extracted values ​​and conditions is judged.

[0076] Numerical range verification: If Findings.NodalStatus.Size > 50mm, mark it as "Dimensional abnormality, review recommended".

[0077] 4. Fourth priority: Cross-field logical consistency check. This is the highest level of check, ensuring that there are no logical inconsistencies within the final output structured data.

[0078] Conclusion and detail matching: If the value of Impression.OverallAssessment.RECIST_Status is 'PD' (disease progression), and the IsNew attribute of all lesions is false and the ChangeFromPrior value is <= 20%, then it is marked as "potential logical contradiction".

[0079] Specifically, the structured data instance determined based on the corrected extraction results includes: mapping and filling the corrected extraction results into the hierarchical fields corresponding to the medical entity to generate a structured data instance of the unstructured medical image report. It can be understood that the verified and corrected information is accurately mapped and filled into the corresponding hierarchical fields of the pancreatic cancer post-operative ontology, thereby generating a complete and standardized structured data instance for a single report.

[0080] Step S13: Perform longitudinal fusion analysis on each of the structured data instances according to the time series, so as to generate time series indicators describing disease changes and overall status assessment results by associating the same clinical discovery entity of the structured data instances at different time points, so as to obtain the longitudinal fusion analysis results.

[0081] In this embodiment, clinical discovery entities in each structured data instance are matched and linked across time according to the time series with corresponding clinical discovery entities in historical reports to obtain corresponding linking results. Based on the linking results, the change in the time series indicators of the clinical discovery entities over time is calculated. Based on the change, the overall state of disease change is determined to obtain an overall state assessment result. A longitudinal fusion analysis result is generated based on the time series indicators and the overall state assessment result. Specifically, weighted feature similarity is performed between clinical discovery entities in each structured data instance and corresponding clinical discovery entities in historical reports according to the time series to obtain corresponding linking results. The weighted features include anatomical location, entity type, and morphological description. It is understood that when the system receives a new imaging report and identifies it as belonging to a patient with existing historical records using PatientInfo.ID, the longitudinal fusion engine is triggered. Initialization logic (processing the first report): For the first report of a patient processed by the system for the first time, the entity linking and comparison functions of the longitudinal fusion engine are skipped. The system only executes modules 1 to 3 to generate a baseline structured report. In this baseline report, the IsNew field for all identified clinical findings (such as lesions and lymph nodes) is automatically set to true, while the ChangeFromPrior field is set to null or "Not Applicable (N / A)" because no previous data is available for comparison. This report will serve as the initial time point (T0) for all subsequent comparisons. Incremental processing logic (processing subsequent reports): When processing the second and subsequent reports for this patient (Tn, n>0), after completing the structured processing of the current report in modules 1 to 3, the system will automatically retrieve and load the structured report data of the patient's most recent report (Tn-1) from the database. Subsequently, the longitudinal fusion engine uses these two reports (Tn and Tn-1) as input to perform entity linking and time-series index calculations. It is important to note that longitudinal fusion relies on precise cross-report entity linking; when a matching historical entity cannot be found, the system will classify it as a new lesion.

[0082] For newly diagnosed lesions, the core steps for vertical fusion are as follows:

[0083] 1. Cross-report entity linking and coreference resolution:

[0084] a. Candidate Entity Identification and Feature Vector Construction: For each clinical finding entity identified in the current report (Tn) and each entity in historical reports (Tn-1), the system constructs a multi-dimensional feature vector. This vector contains the following weighted features:

[0085] For anatomical locations (weight 0.5), the standardized anatomical codes from the ontology are used.

[0086] Entity type (weighted at 0.3), such as "liver metastases" and "retroperitoneal lymph nodes".

[0087] Dimension (weight 0.1), normalized maximum or minimum diameter.

[0088] Morphological description (weight 0.1): Descriptive terms (such as “cystic”, “enhanced”, “unclear boundaries”) are converted into TF-IDF vectors.

[0089] b. Similarity Calculation and Linking: The system uses a weighted cosine similarity algorithm to calculate the similarity score between the feature vectors of each entity in the current report and all similar entities in the historical reports. ;in, and These are the feature vectors of the current and previous entities, respectively. This indicates that the current entity feature vector is at the [number]th [node]. Values ​​in each feature dimension This indicates that the previous entity feature vector is at the th position. Values ​​in each feature dimension Represents the dot product. It is the weight of the i-th feature dimension.

[0090] c. Matching Decision and Threshold Processing: The system sets a matching threshold (e.g., 0.85). For an entity in the current report, if its highest similarity score with an entity in a historical report is higher than this threshold, the two are successfully linked.

[0091] d. New Entity Determination and Field Population: If an entity's highest similarity score is lower than or equal to the threshold, the system determines it as a newly discovered entity. In this case, the system will automatically populate its longitudinal analysis fields: the IsNew field is set to true, and the ChangeFromPrior field is set to null or "Not Applicable (N / A)".

[0092] Step S14: Integrate the longitudinal fusion analysis results to obtain the dynamic change trajectory characterizing the user's disease changes.

[0093] In this embodiment, the structured information of single reports and the cross-report longitudinal analysis results from the longitudinal fusion analysis results are integrated to obtain the dynamic trajectory of disease changes in the user's image reports. It is understood that when entities at two time points are successfully linked, the system automatically performs quantitative calculations and fills the time-related fields in the ontology. For example, if the size of the corresponding liver lesion in the previous report was 1.8 cm, the system will calculate the size change rate: (2.1−1.8) / 1.8=+16.7%, and fill this value into the ChangeFromPrior field. The system further applies clinical rules to determine the status based on these quantitative indicators. For example, if no matching entity is found in the previous report, the IsNew field of the lesion will be set to true. More importantly, the system will automatically calculate and fill the Impression.OverallAssessment.RECIST_Status field based on standards such as RECIST 1.1, comprehensively considering the sum of the diameters of all target lesions and their change rates. This function automates one of the most core and complex tasks in tumor image assessment. Figure 3 As shown, when the longitudinal fusion engine receives structured reports (Report A and Report B) from two time points, the entity linker first identifies that the two reports describe the same liver lesion based on features such as location and size. Subsequently, the time-series indicator calculator automatically calculates the percentage change in size in the reports and determines that the lesion is not newly developed. These calculated longitudinal indicators are then populated into the latest structured report, generating the final updated report, thereby transforming isolated data points into a continuous record of disease evolution.

[0094] like Figure 4 As shown, the complete processing flow from receiving unstructured CT reports to outputting structured data containing longitudinal analysis results is disclosed. Specifically, the system first receives a series of unstructured report texts, which, after preprocessing, are sent to a hybrid NLP engine. Within this engine, a deep learning extractor performs initial information extraction, and the results are then sent to a clinical knowledge validation layer for correction using ontology knowledge. The corrected information is then populated into the ontology, forming a single structured report. Finally, the longitudinal fusion engine associates multiple reports through its cross-report entity linker and time-series indicator calculator, calculates temporal changes, and ultimately outputs a complete structured longitudinal data object containing the disease evolution history, such as a JSON (JavaScript Object Notation) or XML (eXtensible Markup Language) file.

[0095] As can be seen, this application discloses receiving multiple unstructured medical image reports from the same user, arranged in a time sequence; using a pre-defined domain medical ontology to perform natural language processing on each of the unstructured medical image reports to generate corresponding structured data instances, wherein the pre-defined domain medical ontology defines anatomical structure concepts, clinical discovery concepts, and the relationships between these concepts in the target medical scenario; performing longitudinal fusion analysis on each of the structured data instances according to the time sequence, so as to generate time-series indicators describing disease changes and overall state assessment results by associating the same clinical discovery entity of the structured data instances at different time points, thereby obtaining longitudinal fusion analysis results; integrating the longitudinal fusion analysis results to obtain the dynamic trajectory of the user's disease changes. Thus, by accurately identifying and distinguishing postoperative concepts through a pre-defined domain medical ontology with domain-specific knowledge, serious clinical misjudgments caused by a lack of anatomical knowledge are avoided, greatly improving the accuracy and clinical credibility of the structured results, and automatically calculating dynamic indicators by associating the same entity across time. It allows users to clearly see the evolution of the disease without manually comparing multiple reports or performing logical reasoning, outputting a high-level, quantitative dynamic trajectory. This provides strong objective evidence, improving the efficiency and accuracy of decision-making.

[0096] Reference Figure 5 As shown, the present invention provides an image report analysis device applied to a computer device, comprising:

[0097] The report receiving module 11 is used to receive multiple unstructured medical image reports from the same user, arranged in a time sequence.

[0098] The instance generation module 12 is used to perform natural language processing on each of the unstructured medical image reports using a preset domain medical ontology to generate corresponding structured data instances. The preset domain medical ontology defines the anatomical structure concept, clinical discovery concept, and the relationship between the concepts in the target medical scenario.

[0099] The instance fusion module 13 is used to perform longitudinal fusion analysis on each of the structured data instances according to the time series, so as to generate time series indicators describing disease changes and overall status assessment results by associating the same clinical discovery entity of the structured data instances at different time points, so as to obtain the longitudinal fusion analysis results.

[0100] The results generation module 14 is used to integrate the longitudinal fusion analysis results to obtain a dynamic change trajectory characterizing the user's disease changes.

[0101] Therefore, it is evident that by integrating a proprietary domain-specific medical ontology through the instance generation module 12, which possesses clinical expertise, it can accurately identify postoperative anatomical structures and complex clinical findings, avoiding misjudgments caused by a lack of knowledge and improving the clinical accuracy and reliability of structured results. Secondly, this device achieves a paradigm shift from static description to dynamic tracking. The instance fusion module 13, by associating the same clinical entity across time points, automatically quantifies the rate of change of key indicators and generates an overall status assessment, transforming previously isolated multiple reports into an intuitive, quantifiable trajectory of disease evolution. This significantly reduces the cognitive load on doctors manually comparing historical reports, providing unprecedented dynamic decision support for disease progression assessment, efficacy judgment, and prognosis prediction. Finally, this device transforms massive amounts of unstructured text into high-quality, time-series structured data assets, directly serving medical quality control and big data analysis.

[0102] Furthermore, embodiments of this application also disclose an electronic device, Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0103] Figure 6 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the image report analysis method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0104] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0105] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0106] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0107] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system 221 can be Windows Server, Netware, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the image report analysis method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0108] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned image report analysis method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0110] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software module may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable disks, CD-ROMs (Compact Disc-Read Only Memory), or any other form of storage medium known in the art.

[0111] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0112] The solution provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An image report analysis method, characterized in that, Applied to computer devices, including: Receive multiple unstructured medical image reports from the same user, arranged in time sequence; Natural language processing is performed on each of the unstructured medical image reports using a predefined domain medical ontology to generate corresponding structured data instances. The predefined domain medical ontology defines the anatomical structure concept, clinical discovery concept, and the relationship between these concepts in the target medical scenario. The structured data instances are subjected to longitudinal fusion analysis according to time series, so as to generate time series indicators describing disease changes and overall status assessment results by associating the same clinical discovery entity of the structured data instances at different time points, so as to obtain the longitudinal fusion analysis results. The longitudinal fusion analysis results are integrated to obtain the dynamic trajectory of changes in the user's disease.

2. The image report analysis method according to claim 1, characterized in that, The step of performing natural language processing on each of the unstructured medical image reports using a preset domain medical ontology to generate corresponding structured data instances includes: Natural language processing is performed on each of the unstructured medical image reports using a predefined domain medical ontology to extract medical entities, medical attributes, and relationships to obtain initial extraction results. Using a preset clinical knowledge verification layer, the initial extraction results are verified and corrected based on clinical rules to obtain structured data instances determined based on the corrected extraction results; the verification and correction include postoperative anatomical correction of anatomical structure concepts based on the user's surgical history.

3. The image report analysis method according to claim 2, characterized in that, The structured data instance determined based on the corrected extraction results includes: The corrected extraction results are mapped and filled into the hierarchical fields corresponding to the medical entities to generate a structured data instance of the unstructured medical image report.

4. The image report analysis method according to claim 2, characterized in that, The step of using a preset clinical knowledge verification layer to verify and correct the initial extraction results based on clinical rules includes: The initial extraction results are verified and corrected based on clinical rules using a preset clinical knowledge verification layer according to a preset priority verification order; wherein the preset priority verification order is a verification and correction order of preset first priority, preset second priority, preset third priority, and preset fourth priority. The preset first priority is used to perform negation word and uncertainty detection; The preset second priority is used to perform postoperative anatomical correction of the anatomical structure concept based on the user's surgical history; The preset third priority is used to perform clinically appropriate examinations; The preset fourth priority is used to perform cross-field logical consistency checks.

5. The image report analysis method according to claim 1, characterized in that, The longitudinal fusion analysis of each structured data instance according to time series, by associating the same clinical discovery entity of the structured data instances at different time points, generates time-series indicators describing disease changes and overall status assessment results, to obtain the longitudinal fusion analysis results, including: The clinical findings entities in each of the structured data instances are matched and linked across time with the corresponding clinical findings entities in historical reports according to the time series, so as to obtain the corresponding linking results. Based on the linking results, the change in the time-series indicators of the clinical discovery entity over time is calculated; Based on the amount of change, the overall state of the disease change is determined to obtain an overall state assessment result; The longitudinal fusion analysis results are generated based on the time-series indicators and the overall state assessment results.

6. The image report analysis method according to claim 5, characterized in that, The step of matching and linking the clinical findings entities in each of the structured data instances with the corresponding clinical findings entities in historical reports across time according to the time series to obtain the corresponding linking results includes: Based on the time series, the clinical discovery entities in each of the structured data instances are weighted feature similarity scores with the corresponding clinical discovery entities in historical reports to obtain the corresponding link results; The weighted features include: anatomical location, entity type, and morphological description.

7. The image report analysis method according to any one of claims 1 to 6, characterized in that, The integration of the longitudinal fusion analysis results to obtain a dynamic trajectory characterizing changes in the user's disease includes: The structured information of single reports and the cross-report longitudinal analysis results from the longitudinal fusion analysis results were integrated to obtain the dynamic trajectory of disease changes in user image reports.

8. An image report analysis device, characterized in that, Applied to computer devices, including: The report receiving module is used to receive multiple unstructured medical image reports from the same user, arranged in a time sequence. The instance generation module is used to perform natural language processing on each of the unstructured medical image reports using a preset domain medical ontology to generate corresponding structured data instances. The preset domain medical ontology defines the anatomical structure concept, clinical discovery concept, and the relationship between the concepts in the target medical scenario. The instance fusion module is used to perform longitudinal fusion analysis on each of the structured data instances according to the time series, so as to generate time series indicators describing disease changes and overall status assessment results by associating the same clinical discovery entity of the structured data instances at different time points, so as to obtain the longitudinal fusion analysis results. The results generation module is used to integrate the longitudinal fusion analysis results to obtain a dynamic trajectory characterizing the changes in the user's disease.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the image report analysis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when executed by a processor, the computer programs implement the steps of the image report analysis method as described in any one of claims 1 to 7.

Images