Abnormal disease label labeling system and method based on image report
The system uses a large language model and neural network classification to iteratively refine label extraction for medical imaging reports, addressing the challenges of inconsistent terminology and high annotation costs, improving data set construction for AI-assisted medical care.
Patent Information
- Application Number
- CN202411105083.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-12-13
AI Technical Summary
Medical imaging data sets are difficult to obtain and costly to annotate due to varying doctor writing habits and inconsistent terminology, posing challenges for machine learning labeling of abnormal lesions in medical imaging reports.
A system and method utilizing a large language model for sentence segmentation, regular expression rules, and neural network classification to iteratively refine label extraction, incorporating medical knowledge for accurate annotation of abnormal diseases in imaging reports.
Enhances the efficiency and accuracy of medical data set construction by automating label extraction and reducing the need for extensive manual annotation, leveraging existing hospital data for AI-assisted medical care.
Smart Images
Figure CN119131521A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition, and in particular to an abnormal disease labeling system and method based on image reports. Background Art
[0002] With the improvement of medical standards and the continuous development of medical imaging technology, medical imaging data has become an important part of patients' electronic files. Writing a qualified medical imaging report requires doctors to have medical imaging knowledge, a certain ability to evaluate the changes of lesions over time, essential clinical medical knowledge and clinical experience, and the ability to make comprehensive judgments on the patient's medical history and other examination results. However, medical imaging data is growing exponentially, and medical institutions have accumulated a large number of image-text data pairs that have been diagnosed and reviewed by professional doctors. Faced with the current situation where medical data sets are difficult to obtain and the cost of annotation is too high, how to use existing data at a lower cost to build an efficient and reliable data set to help AI-assisted medical care continue to improve is an urgent problem that needs to be solved.
[0003] However, there are also many problems with medical imaging data. First, when writing an imaging report, doctors need to carefully observe the current reading image, identify the abnormal lesion areas and features, and write the imaging report according to certain clinical diagnosis and treatment processes and medical description habits. These text messages naturally contain image abnormality information, but not all abnormal descriptions are directly mapped to clear and specific abnormal labels. For example, "atelectasis" may be described in the imaging report as: "reduced lobe volume", "incomplete expansion of the lungs", "lung compression" and other descriptions. Secondly, many different doctors have different writing habits and inconsistent terminology, so it is impossible to use fixed specifications for label extraction. These problems bring difficulties to machine learning labeling. Summary of the invention
[0004] The purpose of the present invention is to provide a system and method for labeling abnormal diseases based on image reports to solve the problems raised in the above background technology.
[0005] To achieve the above-mentioned invention object, the present invention provides an abnormal disease labeling system based on image report, including an LLM report disassembly module, a label rule iteration module, a label extraction module, and a neural network module, wherein:
[0006] The LLM report disassembly module uses the semantic understanding capability of the large language model to perform sentence segmentation of the image report text;
[0007] The label extraction module builds regular expression rules based on the image report split in the LLM report disassembly module to extract abnormal labels;
[0008] The neural network classification module uses the extracted labels and the abnormal disease dataset constructed from the corresponding medical imaging reports to perform neural network classification model prediction;
[0009] The label rule iteration module corrects and improves the regular expression rules through multiple rounds of iterations according to the prediction results of the neural network classification model until no new rules are added or modified, and the iteration ends.
[0010] Furthermore, in the LLM report disassembly module, each sentence only retains a specific anatomical location or a single abnormality description.
[0011] Furthermore, in the label rule iteration module, the prediction result of the neural network classification model is compared with the abnormal disease label. If they are inconsistent, the regular expression is corrected and then the label extraction module is entered to extract the abnormal label.
[0012] Furthermore, the label rule iteration module is manually checked by personnel with medical background knowledge to correct the regular expression rules.
[0013] Another aspect of the present invention provides a method for labeling abnormal disease based on image reports, comprising the following steps:
[0014] Step S1, using the semantic understanding capability of the large language model to perform sentence segmentation of the image report text;
[0015] Step S2, construct regular expression rules and complete preliminary label extraction based on the sentence segmentation results;
[0016] Step S3, establishing a neural network classification model, using the extracted labels and the abnormal disease data set constructed from the corresponding medical imaging reports to perform neural network classification model training and prediction;
[0017] Step S4, correct and improve the regular expression rules in multiple rounds of iterations according to the prediction results of the neural network classification model until no new rules are added or modified, and the iteration ends.
[0018] Furthermore, in step S1, each sentence only retains a specific anatomical location or a single abnormality description.
[0019] Furthermore, in step S4, the prediction result of the neural network classification model is compared with the label, and if they are inconsistent, the regular expression is corrected and steps S2 and S3 are repeated.
[0020] Furthermore, in step S4, personnel with medical background knowledge perform manual inspection and correct regular expression rules.
[0021] Compared with the prior art, the present system and method have the following advantages:
[0022] 1. The text report sentence segmentation scheme based on a large language model can be used to segment complex text descriptions into sentences based on semantics, ensuring the information integrity and independence of each clause description and solving the problem of regular expression coverage accuracy;
[0023] 2. Regular expressions are used for label extraction. Compared with the keyword matching method, the accuracy is higher. When facing abnormal values, ignore conditions can be written specifically to exclude them, ensuring the correctness of the extracted labels;
[0024] 3. Due to the use of a multi-round iteration method of the neural network model, model training and data set iteration can be carried out alternately, and the model prediction results can be used to promote data updates, which is more targeted, greatly reduces the amount of data involved in the update, and greatly reduces the demand for labeling personnel, which solves the problem of high cost of medical data set labeling to a certain extent;
[0025] In summary, the present invention adopts a combination of the above-mentioned technologies, utilizes the semantic understanding ability of a large language model, uses a neural network model to actively select data, constructs a data set annotation pipeline, and automates multiple rounds of iterations. This can achieve more accurate automatic annotation of medical abnormal disease labels based on imaging reports, and overcomes the problems of difficulty, high cost, and long time consumption in existing medical data set annotation. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 Diagram of a system for labeling abnormal diseases based on imaging reports.
[0027] Figure 2 The flowchart is a method for labeling abnormal diseases based on imaging reports. DETAILED DESCRIPTION
[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0029] The medical field has accumulated a large amount of "medical image-text report" data, which has a strong internal correlation. We use this massive amount of hospital data, use computers to analyze the information of the examination report, and then automatically extract the labels of the image report. After that, the neural network model analyzes and processes the medical image, and iterates the model training according to the labels extracted from the report, so that doctors only review and modify the results of the wrong predictions, and iterate the label extraction rules, which can greatly relieve the annotation pressure of doctors and greatly improve the efficiency of building medical data sets.
[0030] like Figure 1 The system structure diagram of the present invention is shown, and the system includes an LLM report disassembly module, a label rule iteration module, a label extraction module, and a neural network module. The LLM report disassembly module uses the semantic understanding ability of the large language model to perform text sentence segmentation of the image report, and divides the original image report into independent clause descriptions. During the extraction process, it is ensured that each sentence only retains a specific anatomical position or a single abnormal description to prevent the regular expression from having too wide a scope of action and causing label extraction errors. The label extraction module is based on medical knowledge to write preliminary regular expression rules, and then extracts labels from the disassembled text report according to the regular expression. The neural network classification module constructs a neural network classification model, and uses the extracted labels and the abnormal disease data set constructed by the corresponding medical image report to perform neural network classification model prediction. The label rule iteration module corrects and improves the regular expression rules according to the prediction results of the neural network classification model for multiple rounds of iterations until no new rules are added or modified, and the iteration ends. This module manually extracts the data whose reasoning results are inconsistent with the extracted labels, and then iterates the regular expression rules. After multiple rounds of iterations, the label extraction regular expression is no longer updated further.
[0031] Based on the above system, the usage method is as follows Figure 2 As shown, the following steps are included:
[0032] Step S1, using the semantic understanding capability of the large language model to perform sentence segmentation of the image report text.
[0033] Example 1 is an example of using a large language model to segment the image report text into sentences:
[0034] enter:
[0035] As a senior radiologist, your goal is to structure the imaging reports input by users. You need to disassemble the imaging reports, and the disassembled clauses need to be accurate to the specific anatomical location and the corresponding imaging description of the location.
[0036] Original imaging report: The lung textures on both sides are thickened, the lower lobes of both sides are honeycombed, and the interlobular septa are thickened; multiple small cystic translucent areas can be seen in both lungs; scattered linear shadows in both lungs, and cavitation shadows are seen under the pleura of the left upper lobe and left lower lung, and the cavitation shape is irregular. The pleura on both sides is locally thickened and adhered. \nNo abnormalities were found in the hilar areas of both lungs. \nMultiple lymph nodes in the mediastinum, some of which were enlarged. \nCalcification can be seen in the aortic wall and coronary artery wall. \nThe density of the vertebral bodies shown is uneven. Low-density lesions in the liver. High-density shadows in both kidneys, low-density shadows in the left kidney. Multiple punctate dense shadows in the bilateral renal calyces, and thickening of the bilateral adrenal glands.
[0037] Output:
[0038] Image report after splitting: [
[0040] "Both sides of the lung texture thickened",
[0041] "Both lower lobes show honeycomb changes and interlobular septa are thickened",
[0042] "Multiple small cystic lucencies were seen in both lungs",
[0043] "Both lungs have scattered linear shadows",
[0044] "There are cavities under the pleura of the left upper lobe and left lower lung, and the cavities are irregular in shape",
[0045] "Local thickening and adhesion of pleura on both sides",
[0046] "No abnormality was found in the hilar areas of both lungs",
[0047] "Multiple lymph nodes in the mediastinum, some of which are enlarged",
[0048] "Calcification can be seen in the aortic wall and coronary artery wall",
[0049] "The vertebral bodies shown have uneven density",
[0050] "Low-density lesions in the liver",
[0051] "High-density shadows of both kidneys",
[0052] "Low-density shadow of left kidney",
[0053] "Multiple punctate dense shadows in bilateral renal calyces",
[0054] "Bilateral adrenal enlargement" ]
[0056] Step S2, preliminarily construct regular expression rules based on medical knowledge, and extract labels from the disassembled text report according to the regular expression.
[0057] Example 2 is to extract disease labels using constructed regular expressions and report sentences.
[0058] Disease: Atelectasis
[0059] The regular expression is: "(lung|lobe).*(atelectasis|(volume|morphology).*shrink|incomplete|compressed|expanded.*poor|compressed)"
[0060] Report clause: "Patterns and linear shadows can be seen in the lower lobes of both lungs, the middle lobe of the right lung, and the lingular segment of the left lung, and some lung segments have atelectasis"
[0061] According to the matching rules, the report clause satisfies the regular expression matching rules of atelectasis, so the abnormal disease label can be extracted from the clause description.
[0062] Step S3, establish a neural network classification model. Given that the task involved in this project is medical images, where X-Ray images are 2D images and CT and MR images are 3D images, different classification models are selected for different types of images. For 2D images, a 2D ResNet classification model can be considered, and for 3D images, a 3D ResNet model can be used. The extracted labels and the abnormal disease data set constructed from the corresponding medical imaging reports are used to train and predict the neural network classification model.
[0063] Step S4, according to the prediction results of the neural network classification model, the regular expression rules are iterated and improved for multiple rounds until no new rules are added or modified, and the iteration ends. The data that is inconsistent with the inference results and the extracted labels are manually extracted, and the regular expression rules are iterated again. After multiple rounds of iterations, the label extraction regular expression is no longer updated.
[0064] The original imaging report is processed by using a large language model, and the label extraction of the text after the sentence segmentation is performed using the regular expression method. The obtained labels are trained and inferred by the neural network model. The data that is inconsistent with the inference results and the extracted labels are extracted for manual inspection, and the regular expression rules are iterated until they are accurate. This method can greatly simplify the construction of medical data sets and can quickly utilize a large amount of image-text data in the hospital, thereby accelerating the digitization of medical care.
[0065] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. An abnormal disease labeling system based on image reports, characterized in that: It includes LLM report disassembly module, label rule iteration module, label extraction module, and neural network module, among which: The LLM report disassembly module uses the semantic understanding capability of the large language model to perform sentence segmentation of the image report text; The label extraction module builds regular expression rules based on the image report split in the LLM report disassembly module to extract abnormal labels; The neural network classification module uses the extracted labels and the abnormal disease dataset constructed from the corresponding medical imaging reports to perform neural network classification model prediction; The label rule iteration module corrects and improves the regular expression rules through multiple rounds of iterations according to the prediction results of the neural network classification model until no new rules are added or modified, and the iteration ends.
2. According to claim 1, a system for labeling abnormal diseases based on imaging reports is characterized in that: In the LLM report disassembly module, each clause only retains a specific anatomical location or a single abnormality description.
3. According to claim 1, a system for labeling abnormal diseases based on image reports is characterized in that: In the label rule iteration module, the prediction result of the neural network classification model is compared with the abnormal disease label. If they are inconsistent, the regular expression is corrected and then the label extraction module is entered to extract the abnormal label.
4. According to claim 1, a system for labeling abnormal diseases based on imaging reports is characterized in that: The label rule iteration module is manually checked by personnel with medical background knowledge to correct the regular expression rules.
5. A method for labeling abnormal diseases based on image reports, characterized in that: The following steps are involved: Step S1, using the semantic understanding capability of the large language model to perform sentence segmentation of the image report text; Step S2, construct regular expression rules and complete preliminary label extraction based on the sentence segmentation results; Step S3, establishing a neural network classification model, using the extracted labels and the abnormal disease data set constructed from the corresponding medical imaging reports to perform neural network classification model training and prediction; Step S4, correct and improve the regular expression rules in multiple rounds of iterations according to the prediction results of the neural network classification model until no new rules are added or modified, and the iteration ends.
6. The method for labeling abnormal diseases based on imaging reports according to claim 5, characterized in that: In step S1, each sentence only retains a specific anatomical location or a single abnormality description.
7. The method for labeling abnormal diseases based on imaging reports according to claim 5, characterized in that: In step S4, the prediction result of the neural network classification model is compared with the label. If they are inconsistent, the regular expression is corrected and steps S2 and S3 are repeated.
8. The method for labeling abnormal diseases based on imaging reports according to claim 5, characterized in that: In step S4, personnel with medical background knowledge perform manual inspection and correct regular expression rules.