Method and system for extracting accidental discovery in image structured report

Abstract

The invention provides a method for extracting accidental discovery in an image structured report. The method comprises the following steps: creating an accidental discovery dictionary corresponding to an anatomical part and an image examination type based on the anatomical part and the image examination type; creating a big language model question template dictionary; after the image structured report is audited, outputting keywords and / or synonyms of the keywords corresponding to the free text in the image structured report based on the accidental discovery dictionary; constructing a big language model question statement corresponding to the free text; and according to the question statement of the large language model, extracting accidental discovery content from the free text, and outputting a result. The invention further discloses a system for extracting the accidental discovery in the image structured report. Under the condition of not increasing extra filling cost, the iconography accidental discovery is automatically tracked, follow-up visit suggestions are set according to the accidental discovery, a diagnosis doctor does not need to actively mark and search follow-up visit rules, the diagnosis quality is improved, and great help is provided for disease prevention and early screening and early diagnosis.

Description

Technical Field

[0001] The present invention relates to the field of medical information, and more specifically, to a method and system for extracting incidental findings in structured image reports. Background Art

[0002] In medical imaging, incidental findings refer to other lesions or abnormalities discovered in addition to the main purpose of the diagnostic imaging examination. These incidental findings may not be related to the patient's current symptoms or diseases, but may represent other important medical conditions. Incidental findings in imaging examinations are of particular importance for early prevention, early screening and early diagnosis of diseases. In recent years, some international guidelines have been issued on how to follow up incidental findings in imaging, such as the 2017 Fleischner Guidelines, which mainly focus on the follow-up plan for lung nodules; the ACR (American College of Radiology) IF (incidental findings) committee also issued guidelines in 2022, covering the classification and follow-up plans for incidental findings of solid organs such as the liver, pancreas, kidneys, adrenal glands, spleen, and thyroid.

[0003] Incidental findings in imaging are rarely tracked and followed up in a standardized manner. There are two main reasons for this. First, there is a lack of standards and training on what constitutes an incidental finding. Second, the cost of extracting these incidental findings and applying follow-up rules is too high. Diagnostic doctors have to complete a report every 5 minutes, and the time is tight and the task is heavy. It is not practical for diagnostic doctors to manually set up follow-up arrangements based on incidental findings and follow-up rules.

[0004] Although the method of using AI to extract unstructured report text can also extract descriptive features similar to those of accidental discoveries, it is impossible to determine whether they have been described as the main features in the report, that is, it is impossible to determine whether it is an accidental discovery. Summary of the invention

[0005] In view of this, the main purpose of the present invention is to provide a method and system for extracting incidental findings in structured image reports. The method utilizes the technical features of structured image reports and a large language model (LLM) to extract incidental findings in image manifestations and set follow-up recommendations based on the incidental findings, thereby solving the problems in the prior art of being unable to determine whether it is an incidental finding and the lack of operability of manually extracting incidental findings and setting follow-up arrangements for diagnostic doctors due to time constraints and heavy tasks.

[0006] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] On the one hand, the present invention provides a method for extracting incidental findings in an image structured report, comprising: creating an incidental findings dictionary corresponding to anatomical sites and imaging examination types based on the anatomical sites and imaging examination types, the incidental findings dictionary including keywords and synonyms of the keywords; creating a large language model question template dictionary; wherein the large language model question template dictionary includes keywords in the incidental findings dictionary and synonyms of the keywords; when the image structured report is reviewed, outputting keywords and / or synonyms of keywords corresponding to free text in the image structured report based on the incidental findings dictionary; constructing a large language model question sentence corresponding to the free text; extracting incidental findings content from the free text according to the question sentence of the large language model, and outputting the result.

[0008] Preferably, based on the incidental discovery dictionary, keywords and / or synonyms of keywords corresponding to the free text in the image structured report are output, and the steps include: automatically identifying the free text in the text box of the image structured report and the anatomical parts and imaging examination types of the corresponding chapters; matching the anatomical parts and imaging examination types of the corresponding chapters with the incidental discovery dictionary, and outputting keywords and / or synonyms of keywords corresponding to the free text.

[0009] Preferably, constructing a large language model question sentence corresponding to the free text comprises the following steps: based on the keywords and / or synonyms of the keywords corresponding to the free text, searching for a large language model question template corresponding to the free text from the large language model question template dictionary; constructing a large language model question sentence according to the large language model question template, the content of the free text, and the keywords and / or synonyms corresponding to the free text.

[0010] Preferably, the method further comprises: automatically extracting the incidental discovery content, calling a follow-up plan matching the incidental discovery content, generating a follow-up suggestion, and pushing the suggestion to the doctor.

[0011] On the other hand, the present invention also provides a system for extracting incidental findings in structured image reports, the system comprising: an incidental findings dictionary creation module, a question template dictionary creation module, a keyword output module, a question sentence construction module and an incidental findings output module, wherein the incidental findings dictionary creation module is connected to the keyword output module and is used to create an incidental findings dictionary corresponding to anatomical parts and imaging examination types based on them, wherein the incidental findings dictionary includes keywords and synonyms of the keywords; the question template dictionary creation module is connected to the question sentence construction module and is used to create a large language model question template dictionary; wherein the large language model question template dictionary includes keywords in the incidental findings dictionary and synonyms of the keywords; the The keyword output module is respectively connected to the accidental discovery dictionary creation module, the question sentence construction module and the accidental discovery output module, and is used to output the keywords and / or synonyms of the keywords corresponding to the free text in the image structured report based on the accidental discovery dictionary after the image structured report is reviewed; the question sentence construction module is respectively connected to the question template dictionary creation module, the keyword output module and the accidental discovery output module, and is used to construct a large language model question sentence corresponding to the free text; the accidental discovery output module is respectively connected to the keyword output module and the question sentence construction module, and is used to extract accidental discovery content from the free text according to the question sentence of the large language model, and output the result.

[0012] Preferably, the keyword output module also includes a recognition unit and a matching unit, wherein the recognition unit is connected to the matching unit and is used to automatically recognize the free text in the text box in the image structured report and the anatomical parts and imaging examination types of the corresponding chapters; the matching unit is connected to the recognition unit and is used to match the anatomical parts and imaging examination types of the corresponding chapters with the incidental discovery dictionary, and output the keywords and / or synonyms of the keywords corresponding to the free text.

[0013] Preferably, the question sentence construction module includes a search unit and a construction unit, wherein the search unit is connected to the construction unit and is used to search for a large language model question template corresponding to the free text from the large language model question template dictionary based on the keywords and / or synonyms of the keywords corresponding to the free text; the construction unit is connected to the search unit and is used to construct a question sentence of the large language model according to the large language model question template, the content of the free text, and the keywords and / or synonyms corresponding to the free text.

[0014] Preferably, the system further comprises a follow-up generation module, which is connected to the incidental discovery output module and is used for automatically extracting the incidental discovery content, calling a follow-up plan matching the incidental discovery content, generating a follow-up suggestion, and pushing it to the doctor.

[0015] Technical effects of the present invention:

[0016] 1. After the review of the structured imaging report is completed, the method of the present invention outputs the keywords or synonyms of the keywords corresponding to the free text in the structured imaging report based on the pre-created incidental findings dictionary; constructs the large language model question sentence corresponding to the free text according to the large language model question template dictionary, extracts the incidental findings content from the free text based on the large language model question sentence, and outputs the result, automatically tracks the incidental findings of imaging without increasing the additional filling cost, sets follow-up suggestions based on the incidental findings, and does not require the diagnostic doctor to actively mark and search for follow-up rules, thereby improving the diagnosis quality and greatly helping in disease prevention, early screening and early diagnosis;

[0017] 2. The method of the present invention utilizes the characteristics of structured imaging reports to extract incidental findings from the free text in structured imaging reports, and does not regard the imaging manifestations described in the main features of the imaging report as incidental findings, thus avoiding logical errors or legal disputes when directly adding follow-up descriptions;

[0018] 3. The method of the present invention utilizes the fact that the number of incidental findings keywords that can be used for follow-up in the same anatomical part is limited, which is precisely the prerequisite for extracting incidental findings in structured imaging reports using LLM. There is no need to fine-tune LLM or add an external knowledge base, which saves development costs and improves the accuracy of extracting incidental findings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 A flow chart of a method for extracting incidental findings in an image structured report according to Embodiment 1 of the present invention is shown;

[0021] Figure 2 A schematic diagram of the system structure for extracting incidental findings from structured image reports according to Embodiment 2 of the present invention is shown;

[0022] Figure 3 A schematic diagram of the system structure for extracting incidental findings from structured image reports according to Embodiment 3 of the present invention is shown;

[0023] Figure 4 A schematic diagram of the system structure for extracting incidental findings from structured image reports according to a fourth embodiment of the present invention is shown;

[0024] Figure 5 A schematic diagram of the system structure for extracting incidental findings in an image structured report according to Embodiment 5 of the present invention is shown. DETAILED DESCRIPTION

[0025] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0026] Embodiment 1

[0027] Figure 1 FIG. 4 shows a flow chart of a method for extracting incidental findings in an image structured report according to Embodiment 1 of the present invention; Figure 1 As shown, the method comprises the following steps:

[0028] Overall Narrative:

[0029] At present, only the findings that are included in the main imaging features of the structured imaging report can automatically apply the follow-up rules. For example, the imaging report of lung nodules, the main business goal is the location, size, number and other information of the lung nodules. Even in conjunction with the AI ​​of lung nodules, the above information of lung nodules can be automatically obtained. Usually, the sensitivity of imaging AI is relatively high, and many small nodules that are not worthy of attention will be listed. After manual judgment and elimination of some unimportant nodules, the follow-up rules can be automatically applied. At present, only the diagnosis of lung nodules has specially produced a structured reporting tool due to the development of health screening business. It is completely inapplicable in other fields, and the follow-up rules for accidental discoveries cannot be used at all.

[0030] Our company has corresponding imaging structured report templates for follow-up of lung nodules, and CT / MR examinations of the liver, pancreas, kidneys, adrenal glands, spleen, thyroid, etc. However, the existing structure of the structured report mainly describes the imaging manifestations and diagnostic logic of a specific disease. For example, liver cancer mainly describes the filling of the lesion before and after contrast injection, and does not focus the structure on something like an incidental cyst. In order to describe these supplementary findings, in the design of the structured report, a free text box is usually added after each major anatomical part, allowing doctors to freely fill in supplementary information outside the structured description. Only the information in these free text boxes is the extracted information of accidental discoveries that is of concern in this invention.

[0031] The core technology of the present invention relies on the characteristics of structured reports. These reports generally contain detailed descriptions and diagnostic logic for specific anatomical sites, although they may not cover all potential imaging manifestations. The present invention specifically utilizes the information provided by free text boxes in structured reports, which not only enable physicians to record incidental findings that fall outside the templated structure, but also effectively distinguish these incidental findings from other findings that have been described as major features in the report, which are usually elaborated in detail in the structured content area. In addition, the location of the chapter in which the text box is located can also provide key anatomical site information, which is crucial for subsequent analysis.

[0032] The number of incidental findings keywords available for follow-up within the same anatomical site is limited, which is precisely the prerequisite for using LLM to extract incidental findings in structured reports.

[0033] The reason why incidental findings usually appear in a free text box under a certain anatomical site in a structured report is that when designing the report, it is considered that there are many types of imaging manifestations that may appear in each specific anatomical site, and the actual diagnostic logic only needs to focus on a part of them. Based on efficiency, the report usually only contains descriptions of those that appear more frequently, and incidental findings are left to be recorded in the free text box.

[0034] However, if the present invention is applied to the text of imaging findings in unstructured reports, the most critical challenge is to distinguish incidental findings from imaging findings that have been described as the main features in the report, so as to avoid logical errors or legal disputes when directly adding follow-up descriptions. This precise distinction is the key to ensuring the smooth implementation of the present invention.

[0035] Step S101, creating an incidental discovery dictionary corresponding to the anatomical part and the imaging examination type, wherein the incidental discovery dictionary includes keywords and synonyms of the keywords;

[0036] Because the number of medical words involved in incidental findings for follow-up rules in a specific anatomical site is small, in order to accurately extract incidental findings, it is first necessary to create a dictionary of anatomical sites and possible incidental findings keywords and synonyms.

[0037] For example, if the examination type is CT and the anatomical site is the lung, the words involved may include:

[0038] Nodules, cysts, tumors, space-occupying lesions, abnormal growth, density changes, calcifications, infiltration, irregular margins, enhancement, parenchymal changes, vascular lesions, architectural distortions.

[0039] Step S102, creating a large language model question template dictionary; wherein the large language model question template dictionary contains keywords in the accidental discovery dictionary and synonyms of the keywords;

[0040] Considering the differences in incidental findings and follow-up rules for different anatomical sites and examination types, the corresponding LLM question templates may be different. Therefore, a dictionary of LLM question templates corresponding to anatomical sites and examination types is defined. The question template contains possible incidental findings keywords and their synonyms. For example, when the examination type is CT and the anatomical site is lung, the LLM question template may be:

[0041] "Please extract information containing the following keywords and their synonyms based on the following radiology report text, and return the results in the format of 'finding name-size-location-feature description'. If there is no matching information, return 'none'. The keywords and synonyms of the finding name include:

[0042] [Keyword list]

[0043] For example: nodule - 1mm - lung - cystic and solid.

[0044] The report text is: [report text]"

[0045] The question template gives the output format in detail, thus ensuring to the greatest extent possible that the accidentally discovered keywords can be output in the specified format.

[0046] These large language model question templates are stored in databases such as SQLSERVER and are retrieved by anatomical part and examination type when used. For easy maintenance, corresponding maintenance tools are provided for maintenance.

[0047] Step S103, when the image structured report is reviewed, outputting keywords and / or synonyms of keywords corresponding to the free text in the image structured report based on the incidental discovery dictionary;

[0048] Among them, based on the incidental discovery dictionary, the keywords and / or synonyms of the keywords corresponding to the free text in the image structured report are output, and the steps include: automatically identifying the free text in the text box of the image structured report and the anatomical parts and imaging examination types of the corresponding chapters; matching the anatomical parts and imaging examination types of the corresponding chapters with the incidental discovery dictionary, and outputting the keywords and / or synonyms of the keywords corresponding to the free text.

[0049] Step S104, constructing a large language model question sentence corresponding to the free text;

[0050] Among them, constructing a large language model question sentence corresponding to the free text, the steps include: based on the keywords and / or synonyms of the keywords corresponding to the free text, searching for the corresponding large language model question template from the large language model question template dictionary; constructing a large language model question sentence according to the large language model question template, the content of the free text, and the keywords and / or synonyms corresponding to the free text.

[0051] Step S105 , extracting incidental discovery content from the free text according to the question statement of the large language model, and outputting the result.

[0052] The method also includes: deploying a local area network LLM environment and a web service: selecting a suitable open source large language model (LLM), such as ChatGLM-6B, and deploying it in a local environment for text analysis. Using a web service to encapsulate the call to the large model, the web service guarantees user access authority control and data security and provides services for the local area network.

[0053] Through the WEB service, LLM is used to automatically extract incidental findings. When the structured report is reviewed, the background program automatically extracts incidental findings from the free text in the free text box of the structured report and the anatomical site and examination type of the corresponding chapter.

[0054] During this process, a list of incidental findings keywords was obtained from a defined dictionary of anatomical sites and incidental findings keywords according to the anatomical site and examination technique, for example: nodule, cyst, tumor, space-occupying lesion, abnormal growth, density change, calcification, infiltration, irregular margin, enhancement, parenchymal change, vascular lesion, architectural distortion.

[0055] Then, according to the corresponding LLM question statement template, the content of the free text edit box and the list of accidentally discovered keywords are replaced to construct the LLM question statement.

[0056] Finally, after LLM questions, the result is obtained, which should be accidental discovery content or "none".

[0057] The method further includes: automatically extracting the incidental discovery content, calling a follow-up plan that matches the incidental discovery content, generating a follow-up suggestion, and pushing the suggestion to the doctor.

[0058] First, define the follow-up rules for incidental findings:

[0059] Based on international guidelines and clinical practice, define a set of follow-up rules that will guide how to follow up different incidental findings.

[0060] For example:

[0061] 1. Pulmonary nodule size (mm) ≤ 4. Follow-up of sub-centimeter nodules first discovered on CT scan within 12 months

[0062] 2Lungs Size of pulmonary nodules (mm) 4-6CT scan 6-12 months Follow-up of first medium-sized nodules

[0063] 3. Pulmonary nodule size (mm) 6-8 CT scan 3-6 months follow-up of nodules approaching critical size for the first time

[0064] Follow-up rules are stored in databases such as SQL SERVER, and corresponding maintenance tools are also provided for maintenance. The background program generates corresponding follow-up suggestions by matching the accidental findings identified by AI.

[0065] When LLM extracts incidental findings, the system will automatically set a follow-up plan based on the anatomical site and examination type information according to the follow-up rules defined above. For example, a follow-up plan for sub-centimeter nodules with a 12-month cycle can be set based on nodules-2mm-left upper lobe-ground glass. Follow-up rules are set through the RIS / PACS system.

[0066] After the follow-up rules are confirmed, a table will be generated and attached to the end of the report. It will be submitted to the clinician as a follow-up recommendation, and the clinician will decide whether to adopt it, as shown in Table 1:

[0067] Table 1

[0068]

[0069] The specific implementation process of this embodiment is described below in the form of examples:

[0070] 1. Clinical scenario: The patient reported that he had difficulty breathing during activities and occasional chest pain. Based on the patient's symptoms and medical history (such as a history of long-distance travel, obesity, long-term use of hormones, etc.), the clinician suspected that he might be at risk of pulmonary embolism.

[0071] 2. Arrange the patient to undergo enhanced thin-layer CT scan of the pulmonary artery and three-dimensional imaging. The imaging doctor then writes the diagnosis in the structured report. During the diagnosis, pulmonary embolism was ruled out, but a 5mm nodule was unexpectedly found, so it was entered in the free text edit box: A solid nodule with a diameter of about 5mm was found in the right upper lobe (S1 segment) with smooth edges.

[0072] 3. Extract incidental findings. The background program retrieves the LLM question template based on the lung and CT information and replaces it with the question statement:

[0073] "Please extract the information containing the keywords 'nodule' and its synonyms 'mass' and 'space-occupying' according to the following imaging report text, and return the results in the format of 'finding name-size-location-characteristic description', such as nodule-1mm-lung-cystic solid or none. The report text is: A solid nodule with a diameter of about 5mm was found in the right upper lobe (S1 segment). Please extract the information containing the following keywords and their synonyms according to the following imaging report text, and return the results in the format of 'finding name-size-location-characteristic description'. The keywords and synonyms of the finding name include: -nodule (mass, space-occupying, space-occupying lesion, abnormal growth) -cyst -The keywords and synonyms of the tumor feature description include: -density change -calcification -infiltration -irregular margin -enhancement -parenchymal change -vascular lesion -structural distortion -solidity -cystic solid For example: nodule-1mm-lung-cystic solid. The report text is: [A solid nodule with a diameter of about 5mm and smooth margin was found in the right upper lobe (S1 segment)]"

[0074] Then, questions were asked to the LLM model in the local area network through WEBSERVICE, and answers were obtained.

[0075] "Nodule - 5mm - right upper lobe (S1 segment) - solid, smooth margins"

[0076] Since the diagnosis corresponding to the purpose of the examination is written in the structured part outside the free text edit box, this part can be guaranteed to be an incidental finding.

[0077] 4. Generate follow-up recommendations

[0078] Based on the information of 5 mm solid nodules, follow-up recommendations were retrieved and a table was generated:

[0079]

[0080] After the image structured report is reviewed, the embodiment of the present invention outputs the keywords or synonyms of the keywords corresponding to the free text in the image structured report based on the pre-created incidental findings dictionary; constructs the large language model question sentence corresponding to the free text according to the large language model question template dictionary, extracts the incidental findings content from the free text based on the large language model question sentence, and outputs the result, so that the incidental findings of imaging are automatically tracked without increasing the additional filling cost, and follow-up suggestions are set according to the incidental findings, without the need for the diagnosing doctor to actively mark and search for follow-up rules, thereby improving the quality of diagnosis and greatly helping in the prevention, early screening and early diagnosis of diseases; the embodiment of the present invention utilizes the characteristics of the image structured report to extract incidental findings from the free text in the image structured report, and does not regard the image manifestations described in the main features of the image report as incidental findings, thus avoiding logical errors or legal disputes when directly adding follow-up descriptions; the embodiment of the present invention utilizes the characteristic that the number of incidental findings keywords that can be used for follow-up in the same anatomical part is limited, which constitutes the prerequisite for extracting incidental findings in the image structured report using LLM, without fine-tuning the LLM and plug-in knowledge base, saving development costs and improving the accuracy of extracting incidental findings.

[0081] Embodiment 2

[0082] Overall Narrative:

[0083] At present, only the findings that are included in the main imaging features of the structured imaging report can automatically apply the follow-up rules. For example, the imaging report of lung nodules, the main business goal is the location, size, number and other information of the lung nodules. Even in conjunction with the AI ​​of lung nodules, the above information of lung nodules can be automatically obtained. Usually, the sensitivity of imaging AI is relatively high, and many small nodules that are not worthy of attention will be listed. After manual judgment and elimination of some unimportant nodules, the follow-up rules can be automatically applied. At present, only the diagnosis of lung nodules has specially produced a structured reporting tool due to the development of health screening business. It is completely inapplicable in other fields, and the follow-up rules for accidental discoveries cannot be used at all.

[0084] Our company has corresponding imaging structured report templates for follow-up of lung nodules, and CT / MR examinations of the liver, pancreas, kidneys, adrenal glands, spleen, thyroid, etc. However, the existing structure of the structured report mainly describes the imaging manifestations and diagnostic logic of a specific disease. For example, liver cancer mainly describes the filling of the lesion before and after contrast injection, and does not focus the structure on something like an incidental cyst. In order to describe these supplementary findings, in the design of the structured report, a free text box is usually added after each major anatomical part, allowing doctors to freely fill in supplementary information outside the structured description. Only the information in these free text boxes is the extracted information of accidental discoveries that is of concern in this invention.

[0085] The core technology of the present invention relies on the characteristics of structured reports. These reports generally contain detailed descriptions and diagnostic logic for specific anatomical sites, although they may not cover all potential imaging manifestations. The present invention specifically utilizes the information provided by free text boxes in structured reports, which not only enable physicians to record incidental findings that fall outside the templated structure, but also effectively distinguish these incidental findings from other findings that have been described as major features in the report, which are usually elaborated in detail in the structured content area. In addition, the location of the chapter in which the text box is located can also provide key anatomical site information, which is crucial for subsequent analysis.

[0086] The number of incidental findings keywords available for follow-up within the same anatomical site is limited, which is precisely the prerequisite for using LLM to extract incidental findings in structured reports.

[0087] The reason why incidental findings usually appear in a free text box under a certain anatomical site in a structured report is that when designing the report, it is considered that there are many types of imaging manifestations that may appear in each specific anatomical site, and the actual diagnostic logic only needs to focus on a part of them. Based on efficiency, the report usually only contains descriptions of those that appear more frequently, and incidental findings are left to be recorded in the free text box.

[0088] However, if the present invention is applied to the text of imaging findings in unstructured reports, the most critical challenge is to distinguish incidental findings from imaging findings that have been described as the main features in the report, so as to avoid logical errors or legal disputes when directly adding follow-up descriptions. This precise distinction is the key to ensuring the smooth implementation of the present invention.

[0089] Figure 2 FIG. 4 shows a schematic diagram of a system structure for extracting incidental findings from an image structured report according to Embodiment 2 of the present invention; Figure 3As shown, the system includes: an accidental discovery dictionary creation module 10, a question template dictionary creation module 20, a keyword output module 30, a question sentence construction module 40 and an accidental discovery output module 50, wherein:

[0090] The incidental discovery dictionary creation module 10 is connected to the keyword output module 30 and is used to create an incidental discovery dictionary corresponding to the anatomical part and the imaging examination type, wherein the incidental discovery dictionary includes keywords and synonyms of the keywords;

[0091] Because the number of medical words involved in incidental findings for follow-up rules in a specific anatomical site is small, in order to accurately extract incidental findings, it is first necessary to create a dictionary of anatomical sites and possible incidental findings keywords and synonyms.

[0092] For example, if the examination type is CT and the anatomical site is the lung, the words involved may include:

[0093] Nodules, cysts, tumors, space-occupying lesions, abnormal growth, density changes, calcifications, infiltration, irregular margins, enhancement, parenchymal changes, vascular lesions, architectural distortions.

[0094] The question template dictionary creation module 20 is connected to the question sentence construction module 40 and is used to create a large language model question template dictionary; wherein the large language model question template dictionary contains the keywords in the accidental discovery dictionary and the synonyms of the keywords;

[0095] Considering the differences in incidental findings and follow-up rules for different anatomical sites and examination types, the corresponding LLM question templates may be different. Therefore, a dictionary of LLM question templates corresponding to anatomical sites and examination types is defined. The question template contains possible incidental findings keywords and their synonyms. For example, when the examination type is CT and the anatomical site is lung, the LLM question template may be:

[0096] "Please extract information containing the following keywords and their synonyms based on the following radiology report text, and return the results in the format of 'finding name-size-location-feature description'. If there is no matching information, return 'none'. The keywords and synonyms of the finding name include:

[0097] [Keyword list]

[0098] For example: nodule - 1mm - lung - cystic and solid.

[0099] The report text is: [report text]"

[0100] The question template gives the output format in detail, thus ensuring to the greatest extent possible that the accidentally discovered keywords can be output in the specified format.

[0101] These large language model question templates are stored in databases such as SQLSERVER and are retrieved by anatomical part and examination type when used. For easy maintenance, corresponding maintenance tools are provided for maintenance.

[0102] The keyword output module 30 is connected to the accidental discovery dictionary creation module 10, the question sentence construction module 40, and the accidental discovery output module 50, respectively, and is used to output the keywords and / or synonyms of the free text in the image structured report based on the accidental discovery dictionary after the image structured report is reviewed;

[0103] The question sentence construction module 40 is respectively connected to the question template dictionary creation module 10, the keyword output module 30, and the accidental discovery output module 50, and is used to construct a large language model question sentence corresponding to the free text;

[0104] The accidental discovery output module 50 is connected to the keyword output module 30 and the question sentence construction module 40 respectively, and is used to extract accidental discovery content from the free text according to the question sentence of the large language model and output the result.

[0105] Before executing the system, you need to deploy the LAN LLM environment and WEB services: select a suitable open source large language model (LLM), such as ChatGLM-6B, and deploy it in the local environment for text analysis. Use WEB services to encapsulate calls to large models, and the WEB services guarantee user access permission control and data security and provide services for the LAN.

[0106] Through the WEB service, LLM is used to automatically extract incidental findings. When the structured report is reviewed, the background program automatically extracts incidental findings from the free text in the free text box of the structured report and the anatomical site and examination type of the corresponding chapter.

[0107] During this process, a list of incidental findings keywords was obtained from a defined dictionary of anatomical sites and incidental findings keywords according to the anatomical site and examination technique, for example: nodule, cyst, tumor, space-occupying lesion, abnormal growth, density change, calcification, infiltration, irregular margin, enhancement, parenchymal change, vascular lesion, architectural distortion.

[0108] Then, according to the corresponding LLM question statement template, the content of the free text edit box and the list of accidentally discovered keywords are replaced to construct the LLM question statement.

[0109] Finally, after LLM questions, the result is obtained, which should be accidental discovery content or "none".

[0110] After the review of the structured imaging report is completed, the embodiment of the present invention outputs the keywords or synonyms of the keywords corresponding to the free text in the structured imaging report based on the pre-created incidental findings dictionary; constructs the large language model question sentences corresponding to the free text according to the large language model question template dictionary, extracts the incidental findings content from the free text based on the large language model question sentences, and outputs the results. Without increasing the additional filling cost, the incidental findings in imaging are automatically tracked, and follow-up suggestions are set according to the incidental findings. There is no need for the diagnostic doctor to actively mark and search for follow-up rules, which improves the quality of diagnosis and is of great help in disease prevention, early screening and early diagnosis.

[0111] Embodiment 3

[0112] Figure 3 FIG. 4 shows a schematic diagram of a system structure for extracting incidental findings from a structured image report according to Embodiment 3 of the present invention; Figure 3 As shown, the keyword output module 30 also includes a recognition unit 302 and a matching unit 304, wherein:

[0113] The recognition unit 302 is connected to the matching unit 304 and is used to automatically recognize the free text in the text box in the image structured report and the anatomical part and imaging examination type of the corresponding chapter;

[0114] The matching unit 304 is connected to the recognition unit 302, and is used to match the anatomical parts and imaging examination types of the corresponding chapters with the incidental findings dictionary, and output keywords and / or synonyms of keywords corresponding to the free text.

[0115] Embodiment 4

[0116] Figure 4 FIG. 4 shows a schematic diagram of a system structure for extracting incidental findings from an image structured report according to a fourth embodiment of the present invention; Figure 4 As shown, the question sentence construction module 40 includes a search unit 402 and a construction unit 404, wherein:

[0117] The search unit 402 is connected to the construction unit 404 and is used to search for a corresponding large language model question template from the large language model question template dictionary based on the keywords and / or synonyms of the keywords corresponding to the free text;

[0118] The construction unit 404 is connected to the search unit 402, and is used to construct a question sentence of the large language model according to the large language model question template, the content of the free text, and the keywords and / or synonyms corresponding to the free text.

[0119] Embodiment 5

[0120] Figure 5 FIG. 4 shows a schematic diagram of a system structure for extracting incidental findings from an image structured report according to Embodiment 5 of the present invention; Figure 5 As shown, the system also includes a follow-up generation module 60, which is connected to the incidental discovery output module 50, and is used to automatically extract the incidental discovery content, call a follow-up plan matching the incidental discovery content, generate a follow-up suggestion, and push it to the doctor.

[0121] First, define the follow-up rules for incidental findings:

[0122] Based on international guidelines and clinical practice, define a set of follow-up rules that will guide how to follow up different incidental findings.

[0123] For example:

[0124] 1. Pulmonary nodule size (mm) ≤ 4. Follow-up of sub-centimeter nodules first discovered on CT scan within 12 months

[0125] 2Lungs Size of pulmonary nodules (mm) 4-6CT scan 6-12 months Follow-up of first medium-sized nodules

[0126] 3. Pulmonary nodule size (mm) 6-8 CT scan 3-6 months follow-up of nodules approaching critical size for the first time

[0127] Follow-up rules are stored in databases such as SQL SERVER, and corresponding maintenance tools are also provided for maintenance. The background program generates corresponding follow-up suggestions by matching the accidental findings identified by AI.

[0128] When LLM extracts incidental findings, the system will automatically set a follow-up plan based on the anatomical site and examination type information according to the follow-up rules defined above. For example, a follow-up plan for sub-centimeter nodules with a 12-month cycle can be set based on nodules-2mm-left upper lobe-ground glass. Follow-up rules are set through the RIS / PACS system.

[0129] After the follow-up rules are confirmed, a table will be generated and attached to the end of the report. It will be submitted to the clinician as a follow-up recommendation, and the clinician will decide whether to adopt it, as shown in Table 2:

[0130] Table 2

[0131]

[0132] The specific implementation process of this embodiment is described below in the form of examples:

[0133] 1. Clinical scenario: The patient reported that he had difficulty breathing during activities and occasional chest pain. Based on the patient's symptoms and medical history (such as a history of long-distance travel, obesity, long-term use of hormones, etc.), the clinician suspected that he might be at risk of pulmonary embolism.

[0134] 2. Arrange the patient to undergo enhanced thin-layer CT scan of the pulmonary artery and three-dimensional imaging. The imaging doctor then writes the diagnosis in the structured report. During the diagnosis, pulmonary embolism was ruled out, but a 5mm nodule was unexpectedly found, so it was entered in the free text edit box: A solid nodule with a diameter of about 5mm was found in the right upper lobe (S1 segment) with smooth edges.

[0135] 3. Extract incidental findings. The background program retrieves the LLM question template based on the lung and CT information and replaces it with the question statement:

[0136] "Please extract the information containing the keywords 'nodule' and its synonyms 'mass' and 'space-occupying' according to the following imaging report text, and return the results in the format of 'finding name-size-location-characteristic description', such as nodule-1mm-lung-cystic solid or none. The report text is: A solid nodule with a diameter of about 5mm was found in the right upper lobe (S1 segment). Please extract the information containing the following keywords and their synonyms according to the following imaging report text, and return the results in the format of 'finding name-size-location-characteristic description'. The keywords and synonyms of the finding name include: -nodule (mass, space-occupying, space-occupying lesion, abnormal growth) -cyst -The keywords and synonyms of the tumor feature description include: -density change -calcification -infiltration -irregular margin -enhancement -parenchymal change -vascular lesion -structural distortion -solidity -cystic solid For example: nodule-1mm-lung-cystic solid. The report text is: [A solid nodule with a diameter of about 5mm and smooth margin was found in the right upper lobe (S1 segment)]"

[0137] Then, questions were asked to the LLM model in the local area network through WEBSERVICE, and answers were obtained.

[0138] "Nodule - 5mm - right upper lobe (S1 segment) - solid, smooth margins"

[0139] Since the diagnosis corresponding to the purpose of the examination is written in the structured part outside the free text edit box, this part can be guaranteed to be an incidental finding.

[0140] 4. Generate follow-up recommendations

[0141] Based on the information of 5 mm solid nodules, follow-up recommendations were retrieved and a table was generated:

[0142]

[0143]

[0144] The embodiment of the present invention utilizes the characteristics of structured image reports to extract incidental findings from the free text in structured image reports, and does not regard the image manifestations described in the main features of the image report as incidental findings. In this way, logical errors or legal disputes when directly adding follow-up descriptions are avoided; the embodiment of the present invention utilizes the fact that the number of incidental finding keywords that can be used for follow-up in the same anatomical part is limited, which constitutes the prerequisite for extracting incidental findings in structured image reports using LLM. There is no need to fine-tune LLM or add an external knowledge base, which saves development costs and improves the accuracy of extracting incidental findings.

[0145] From the above description, it can be seen that the above-mentioned embodiments of the present invention achieve the following technical effects: after the review of the structured imaging report is completed, the embodiments of the present invention output the keywords or synonyms of the free text in the structured imaging report based on the pre-created incidental findings dictionary; construct the large language model question sentences corresponding to the free text according to the large language model question template dictionary, extract the incidental findings content from the free text based on the large language model question sentences, and output the results, so that incidental findings in imaging are automatically tracked without increasing additional filling costs, and follow-up suggestions are set according to incidental findings, without the need for diagnostic doctors to actively mark and search for follow-up rules, thereby improving the quality of diagnosis. The amount of incidental findings is of great help in disease prevention, early screening and early diagnosis; the embodiment of the present invention utilizes the characteristics of image structured reports to extract incidental findings from the free text in image structured reports, and does not regard the image manifestations described in the main features of the image report as incidental findings, thus avoiding logical errors or legal disputes when directly adding follow-up descriptions; the embodiment of the present invention utilizes the fact that the number of incidental finding keywords that can be used for follow-up in the same anatomical part is limited, which constitutes the prerequisite for extracting incidental findings in image structured reports using LLM, without the need for fine-tuning of LLM and plug-in knowledge base, saving development costs and improving the accuracy of extracting incidental findings.

[0146] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0147] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for extracting incidental findings in a structured imaging report, characterized in that: include: Creating an incidental finding dictionary corresponding to the anatomical site and the imaging examination type, wherein the incidental finding dictionary includes keywords and synonyms of the keywords; Creating a large language model question template dictionary; wherein the large language model question template dictionary contains keywords in the accidental discovery dictionary and synonyms of the keywords; When the image structured report is reviewed, outputting keywords and / or synonyms of keywords corresponding to the free text in the image structured report based on the incidental discovery dictionary; Constructing a large language model question sentence corresponding to the free text; According to the question sentence of the large language model, accidental discovery content is extracted from the free text, and the result is output.

2. The method for extracting incidental findings in a structured imaging report according to claim 1, characterized in that: Based on the incidental discovery dictionary, outputting keywords and / or synonyms of keywords corresponding to the free text in the image structured report comprises: Automatically identify the free text in the text box in the structured image report and the anatomical part and imaging examination type of the corresponding chapter; The anatomical parts and imaging examination types of the corresponding chapters are matched with the incidental findings dictionary, and the keywords and / or synonyms of the keywords corresponding to the free text are output.

3. The method for extracting incidental findings in a structured imaging report according to claim 1, characterized in that: Constructing a large language model question sentence corresponding to the free text, the steps include: Based on the keywords and / or synonyms of the keywords corresponding to the free text, searching the large language model question template corresponding to the keywords from the large language model question template dictionary; A question sentence of the large language model is constructed according to the large language model question template, the content of the free text, and the keywords and / or synonyms corresponding to the free text.

4. The method for extracting incidental findings in a structured imaging report according to claim 1, characterized in that: The method also includes: automatically extracting the incidental discovery content, calling a follow-up plan that matches the incidental discovery content, generating a follow-up suggestion, and pushing it to a doctor.

5. A system for extracting incidental findings in structured imaging reports, characterized in that: The system includes: an accidental discovery dictionary creation module, a question template dictionary creation module, a keyword output module, a question sentence construction module and an accidental discovery output module, wherein: The incidental discovery dictionary creation module is connected to the keyword output module and is used to create an incidental discovery dictionary corresponding to the anatomical part and the imaging examination type, wherein the incidental discovery dictionary includes keywords and synonyms of the keywords; The question template dictionary creation module is connected to the question sentence construction module and is used to create a large language model question template dictionary; wherein the large language model question template dictionary contains keywords in the accidental discovery dictionary and synonyms of the keywords; The keyword output module is connected to the accidental discovery dictionary creation module, the question sentence construction module, and the accidental discovery output module, respectively, and is used to output keywords and / or synonyms of keywords corresponding to the free text in the image structured report based on the accidental discovery dictionary after the image structured report is reviewed; The question sentence construction module is connected to the question template dictionary creation module, the keyword output module, and the accidental discovery output module, respectively, and is used to construct a large language model question sentence corresponding to the free text; The accidental discovery output module is connected to the keyword output module and the question sentence construction module respectively, and is used to extract accidental discovery content from the free text according to the question sentence of the large language model, and output the result.

6. The system for extracting incidental findings in structured imaging reports according to claim 5, characterized in that: The keyword output module also includes a recognition unit and a matching unit, wherein: The recognition unit is connected to the matching unit and is used to automatically recognize the free text in the text box in the image structured report and the anatomical part and imaging examination type of the corresponding chapter; The matching unit is connected to the recognition unit, and is used to match the anatomical parts and imaging examination types of the corresponding chapters with the incidental findings dictionary, and output keywords and / or synonyms of the keywords corresponding to the free text.

7. The system for extracting incidental findings in structured imaging reports according to claim 5, characterized in that: The question sentence construction module includes a search unit and a construction unit, wherein: The search unit is connected to the construction unit and is used to search for a large language model question template corresponding to the free text from the large language model question template dictionary based on the keywords and / or synonyms of the keywords corresponding to the free text; The construction unit is connected to the search unit and is used to construct a question sentence of the large language model according to the large language model question template, the content of the free text, and the keywords and / or synonyms corresponding to the free text.

8. The system for extracting incidental findings in structured imaging reports according to claim 5, characterized in that: The system also includes a follow-up generation module, which is connected to the accidental discovery output module and is used to automatically extract the accidental discovery content, call a follow-up plan that matches the accidental discovery content, generate a follow-up suggestion, and push it to the doctor.

Images