Method and device for generating natural language processing model for analyzing spine radiology report

Abstract

The present invention relates to a method and device for generating a natural language processing model for analyzing a spine radiology report. The method and device according to an aspect have the effect that it is possible to analyze various radiology reports, including those for CTs as well as X-rays and MRIs, and analyze an improved rule-based spine radiology report compared to previous research systems.

Description

Method and device for creating a natural language processing model for analyzing radiological reports on the spine

[0001] The present disclosure relates to a method and apparatus for generating a natural language processing model for analyzing a radiology report on the spine.

[0002] 70-80% of adults experience some form of low back pain during their lifetime. More than 85% of people under the age of 45 experience at least one episode of low back pain requiring medication or therapeutic intervention. Low back pain is classified as acute or chronic based on its onset. Acute low back pain usually occurs suddenly, while chronic low back pain lasts for more than three months. Low back pain is the most common reason for hospital visits and the second leading cause of sick leave for workers. Diagnosing and treating low back pain incurs significant direct and indirect costs, resulting in a significant economic impact on individuals, families, and society. In particular, 5-10% of chronic low back pain cases require expensive and long-term management.

[0003] Furthermore, the most common methods for evaluating chronic low back pain are X-rays, computed tomography (CT), and magnetic resonance imaging (MRI). These imaging data are reviewed and interpreted by medical professionals, resulting in a radiological report. A radiological report is a formal interpretation of the radiological examinations and includes radiological findings relevant to clinical diagnosis and treatment decisions. However, many radiological reports use free-form language and are unstructured, requiring manual review to extract clinical information. Manual information extraction is labor-intensive and impractical for large-scale studies. Therefore, creating a structured pipeline to extract clinical information from radiological reports for large-scale data studies is necessary.

[0004] Meanwhile, Natural Language Processing (NLP) is a branch of artificial intelligence that uses machines, such as computers, to process human language patterns. The ultimate goal of NLP is to build systems that enable computers to understand and analyze human language and solve various problems. Challenges in implementing NLP systems include the diverse expressions, ambiguity due to word ambiguity and insufficient sentence information, and discontinuous data. The introduction of machine learning has led to advances in NLP technologies related to data processing and analysis, and deep learning is accelerating the pace of NLP development.

[0005] Natural Language Processing (NLP), a computer technology specialized in text processing, can be used to extract valuable information from the unstructured text of electronic medical records (EMRs). NLP systems fall into three categories: rule-based, machine learning-based, and hybrid approaches. For example, an NLP pipeline utilizing a state-of-the-art deep learning model based on bidirectional encoder representations (BERT, bidirectional encoder representations form transformer) has been used to identify key findings in chest radiograph reports of intensive care unit (ICU) patients.

[0006] Previous research suggests several areas for improvement. First, rule-based approaches have not been studied in NLP systems for extracting words related to low back pain. Specifically, the design cost of rule-based approaches is lower than that of machine learning approaches, and the training dataset required for rule-based approaches is smaller than that of machine learning approaches. Therefore, developing a rule-based NLP system requires less cost and time than a machine learning approach. Previous studies have shown that when a rule-based NLP system was used to extract words from radiology reports related to low back pain, it achieved high specificity but moderate sensitivity. Second, the development of an NLP system for identifying low back pain-related findings in CT radiology reports is needed.

[0007] Therefore, there is a need to develop a natural language processing model that analyzes radiological reports on the spine to solve the above problems.

[0008] In order to solve the above-mentioned problems, the present invention provides a method and / or device for analyzing radiology reports including X-rays, MRIs, and CTs to generate a natural language processing model for analyzing radiology reports on the spine based on rules that are improved over systems of previous studies.

[0009] Additionally, the present specification provides a computer-readable recording medium containing a program for executing the above-described method on a computer. The technical problems to be solved are not limited to the technical problems described above, and other technical problems may exist.

[0010] One aspect provides a method for generating a natural language processing (NLP) model for analyzing radiology reports on the spine, the method comprising: preprocessing texts included in radiology reports included in a training data set selected from among radiology reports on the spine; mapping terms related to the spine extracted from the preprocessed texts to terms included in a pre-established term library; and summarizing the radiology reports by combining the mapped terms according to a pre-established format.

[0011] Another aspect provides a computing device comprising a memory storing at least one program; and a processor executing the at least one program to perform at least one operation, wherein the processor preprocesses texts included in radiology reports included in a training data set selected from among radiology reports on the spine, maps terms related to the spine extracted from the preprocessed texts to terms included in a pre-built terminology library, and summarizes the radiology reports by combining the mapped terms according to a preset format.

[0012] Another aspect provides a computer-readable recording medium having recorded thereon a program for executing the above-described method on a computer.

[0013] A method and device for generating a natural language processing model for analyzing a radiology report on the spine can analyze various radiology reports including X-rays, MRIs, and CTs, and can analyze rule-based radiology reports on the spine that are improved over systems in previous studies.

[0014] FIG. 1 is a diagram illustrating a method for generating a natural language processing model for analyzing a radiology report for the spine according to one embodiment.

[0015] FIG. 2 is a diagram illustrating an example of a step of preprocessing text, a step of mapping text, and a step of summarizing text according to one embodiment.

[0016] FIG. 3 is a graph showing the frequency of radiological findings in a training data set and a test data set according to one embodiment.

[0017] FIG. 4 is a schematic diagram illustrating an example of a computing device according to one embodiment.

[0018] One aspect provides a method for generating a natural language processing (NLP) model for analyzing radiology reports on the spine, the method comprising: preprocessing texts included in radiology reports included in a training data set selected from among radiology reports on the spine; mapping terms related to the spine extracted from the preprocessed texts to terms included in a pre-established term library; and summarizing the radiology reports by combining the mapped terms according to a pre-established format.

[0019] The terms used in the examples are selected from widely used, current terms, as much as possible. However, these terms may vary depending on the intentions of those skilled in the field, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, the applicant may arbitrarily select terms, in which case their meanings will be described in detail in the relevant description. Therefore, the terms used in the specification should be defined based on their intended meaning and the overall content of the specification, rather than simply their names.

[0020] When a part of the specification is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "unit" and "module" used in the specification mean a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.

[0021] Additionally, terms including ordinal numbers, such as “first” or “second,” used in the specification may be used to describe various components, but the components should not be limited by the terms. The terms may be used to distinguish one component from another.

[0022]

[0023] Below, embodiments are described in detail with reference to the attached drawings. However, the embodiments may be implemented in various different forms and are not limited to the examples described herein.

[0024] FIG. 1 is a diagram illustrating a method for generating a natural language processing model for analyzing a radiology report for the spine according to one embodiment.

[0025] Referring to FIG. 1, a method for generating a natural language processing (NLP) model for analyzing radiology reports on the spine may include a step of preprocessing texts included in radiology reports included in a training data set selected from among radiology reports on the spine (110); a step of mapping terms related to the spine extracted from the preprocessed texts to terms included in a pre-established term library (120); and a step of summarizing the radiology reports by combining the mapped terms according to a pre-established format (130).

[0026] In one specific example, the radiology report may be a radiology report for a spinal disease or low back pain, or a subject (e.g., a patient) suffering from a spinal disease or low back pain.

[0027] In one embodiment, the radiology report may be a report on any one image selected from the group consisting of an X-ray radiographic image, a magnetic resonance imaging (MRI) image, an ultrasound image, an endoscopic image, an elastography image, a thermal image, a positron emission tomography (PET) image, a single photon emission computed tomography (SPECT) image, an optical coherence tomography (OCT) image, and a computed tomography (CT) image.

[0028] In one specific example, the constructed term library may include words for spinal pathology, words for structural problems of the spine, and / or words for spinal diseases. More specifically, the above term library includes Listhesis, Scoliosis, Fracture, Spondylosis, Annular fissure, Disc bulge, Disc degeneration, Disc extrusion, Disc herniation, Disc protrusion, End plate edema or type 1 Modic change, Facet hypertrophy, Central stenosis, Foraminal stenosis, Nerve root displaced / compressed, Lateral recess stenosis, Spondylolysis, Schmorl's node, Osteophyte, Disc space narrowing. It may include at least one selected from the group consisting of: any stenosis, disc sequestration, intradiscal vacuum, and combinations thereof.More specifically, the above-mentioned pre-built term library may include a combination of Disc extrusion, End plate edema or type 1 Modic change, Central stenosis, Foraminal stenosis, Nerve root displaced / compressed, Lateral recess stenosis, and Any stenosis.

[0029]

[0030] Data set of this specification

[0031] A retrospective study of lumbar spine radiology reports was conducted on patients who visited CHA University Hospital from January 2011 to December 2020. A total of 18,640 reports were collected to form a dataset, from which 300 X-rays, 300 CT scans, and 300 MRI scans were randomly selected and extracted. The dataset was randomly split into training and test datasets in an 8:2 ratio. The training dataset was used to generate the NLP system described above, and the test dataset was used to validate the information extraction performance. The dataset (n = 900) was divided into a training dataset (n = 720) and a test dataset (n = 180). The median age, standard deviation, and proportion of male or female patients for each dataset are shown in Table 1. Research on the dataset in this specification was conducted with approval from the Institutional Review Board of CHA Medical Center (IRB No. 2021-04-212).

[0032]

[0033]

[0034] Natural Language Processing (NLP) System of this Specification

[0035] The Natural Language Processing (NLP) system described in this specification uses Python (3.8.10) as its NLP pipeline. Phrase and sentence segmentation of radiology reports were performed using regular expressions for negation detection and a rule-based approach. Regular expressions are specialized character sets for analyzing patterns in text, and the "re" module is one of the Python libraries used to implement regular expressions. In this specification, clinical information extraction rules were used to perform text analysis using "re."

[0036] Referring to FIG. 2, the natural language processing system of the present disclosure will be described in detail. Preprocessing texts contained in radiology reports included in a training data set selected from radiology reports on the spine may include tokenizing words and phrases within the reports. Specifically, the preprocessing step may include performing at least one of section segmentation, sentence segmentation, and normalization on the texts. The rule-based NLP system may refine unstructured raw text to remove unexpected spaces and punctuation marks and filter out sentences unrelated to the spine (or back pain).

[0037] The step of mapping terms related to the spine (or back pain) extracted from the above preprocessed texts to terms included in a pre-built terminology library may include mapping concepts related to the spine (or back pain) recorded in radiological reports to appropriate pre-built terms by referring to a dictionary and language rules of pre-built terms.

[0038] By summarizing the radiology reports by combining the mapped terms according to the above-described format, terms that are described or defined differently across radiology reports can be unified into a unified summary. This can assist experts (e.g., physicians) in making objective clinical judgments based on the summary results.

[0039]

[0040] Verification of the Natural Language Processing (NLP) System of this Specification

[0041]

[0042] The method of the present disclosure may further include a step of evaluating the results of the summary using a test data set selected from among the radiological reports on the spine. In one specific example, the evaluating step may be performed using at least one of recall, precision, accuracy, and F1 score.

[0043] Specifically, the extracted words were expressed in a matrix, and performance evaluations were conducted on the derived results. After initially building a rule-based NLP system, the entire process described above was repeated until the NLP system's performance improved. K-fold cross-validation was used to validate the NLP model, and the best-performing version of the NLP system was validated on the test dataset.

[0044] The chi-square test was used to compare proportions, and the t-test was used to compare quantitative variables. The performance of the NLP system was measured by recall, precision, accuracy, and F1 score, as shown below.

[0045] Recall = (Number of true positives) / (Number of positives + Number of false negatives)

[0046] Precision = (Number of positives) / (Number of positives) + (Number of false positives)

[0047] Accuracy = (Number of positives + Number of negatives) / (Number of positives + Number of negatives + Number of false positives + Number of false negatives)

[0048] F1 = (2 X Recall X Precision) / (Precision + Recall)

[0049] F1 was used to evaluate performance.

[0050] In the above equations, "positive" means that a specific term included in the established term library is detected as the same specific term in a radiology report through the natural language processing system. Furthermore, "negative" means that a specific term included in the established term library is not detected as the same specific term in a radiology report through the natural language processing system.

[0051] Meanwhile, a "false negative" refers to a case where a specific term included in a pre-established term library is not detected in a radiology report when the natural language processing system detects the same term. A "false positive" refers to a case where a specific term included in a pre-established term library is not detected in a radiology report when the natural language processing system detects the same term.

[0052]

[0053] Performance evaluation results of the natural language processing (NLP) system of this specification

[0054] The system or method of the present specification was evaluated using the above recall, precision, accuracy, and / or F1 score, and the results are shown in Table 2 and FIG. 3 below.

[0055]

[0056]

[0057] As a result, as shown in Fig. 3 and Table 2, in the test dataset (n = 120), all 23 radiological findings showed recall, precision, accuracy, and F1 scores of 0.9 or higher. The radiological findings with all four evaluation criteria equal to 1.0 were listhesis, annular fissure, disc bulge, disc extrusion, disc protrusion, endplate edema or type 1 Modic change, lateral recess stenosis, Schmorl's node, osteophyte, and any stenosis. The lowest F1 score was 0.9802 for facet hypertrophy. The recall was highest at 1.0 for 14 radiological findings (listhesis, fracture, annular fissure, disc bulge, disc extrusion, disc herniation, disc protrusion, endplate edema or type 1 Modic, facet hypertrophy, lateral recess stenosis, spondylolysis, Schmorl's node, osteophyte, any stenosis), and the lowest recall was 0.9840 for central stenosis. The precision was 1 for 16 radiological findings (listhesis, scoliosis, spondylosis, annular fissure, disc bulge, disc extrusion, disc protrusion, endplate edema or type 1 Modic, central stenosis, nerve root displaced / compressed, lateral recess stenosis, Schmorl's node, osteophyte, disc space narrowing, any stenosis, disc sequestration).The highest value was 0, and the lowest precision was 0.9611 for facet hypertrophy. The accuracy was 1.0 for 10 radiological findings (listhesis, annular fissure, disc bulge, disc extrusion, disc protrusion, endplate edema or type 1 Modic, lateral recess stenosis, Schmorl's node, osteophyte, any stenosis), and the lowest accuracy was 0.9611 for facet hypertrophy.

[0058] The F1 scores for the seven important radiographic findings mentioned above are as follows: 1.0 for disc extrusion, 1.0 for endplate edema or Type 1 Modic, 1.0 for any stenosis, 0.9919 for central stenosis, 0.9882 for foraminal stenosis, 0.9969 for nerve root displaced / compressed, and 1.0 for lateral recess stenosis.

[0059] As a result, it was found that the method of the present specification can effectively extract medical information by applying natural language processing technology to radiological reports of the spine, including lumbago.

[0060]

[0061] Computing device of this specification

[0062] The present invention provides a computing device comprising a memory storing at least one program; and a processor executing the at least one program to perform at least one operation, wherein the processor preprocesses texts included in radiology reports included in a training data set selected from among radiology reports on the spine, maps terms related to the spine extracted from the preprocessed texts to terms included in a pre-built terminology library, and summarizes the radiology reports by combining the mapped terms according to a preset format.

[0063] Referring to FIG. 4, the computing device may be a user terminal (200) or a server.

[0064] The user terminal (200) may be an electronic device including a display device and a device for receiving user input (e.g., a keyboard, a mouse, etc.), and including a memory (220) and a processor (210). In addition, the display device may be implemented as a touch screen and may perform a function for receiving user input. For example, the user terminal may be, but is not limited to, a notebook PC, a desktop PC, a laptop, a tablet computer, a smart phone, etc. The server may be a device that communicates with an external device (e.g., a user terminal, etc.). For example, the server may be a device that stores various data, including radiological information about the spine. Alternatively, the server may be an electronic device including a memory and a processor and having its own computing capabilities. For example, the server may be, but is not limited to, a cloud server.

[0065] The user terminal (200) includes a processor (210), memory (220), an input / output interface, and a communication module (230). For convenience of explanation, only components related to the present invention are illustrated in FIG. 4. Therefore, in addition to the components illustrated in FIG. 4, other general-purpose components may be further included in the user terminal (200). Furthermore, it will be apparent to those skilled in the art that the processor (210), memory (220), input / output interface, and communication module (230) illustrated in FIG. 4 may be implemented as independent devices.

[0066] The processor (210) can process computer program commands by performing basic arithmetic, logic, and input / output operations. Here, the commands can be provided from memory (220) or an external device (e.g., a server, etc.). In addition, the processor (210) can generally control the operations of other components included in the user terminal (200).

[0067] The processor (210) may be implemented as an array of a plurality of logic gates, or may be implemented as a combination of a general-purpose microprocessor and a memory storing a program that can be executed on the microprocessor. For example, the processor (210) may include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, the processor (210) may include an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. For example, the processor (210) may refer to a combination of processing devices, such as a combination of a digital signal processor (DSP) and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors coupled with a digital signal processor (DSP) core, or any other such combination of configurations.

[0068] The memory (220) may include any non-transitory computer-readable recording medium. As an example, the memory (220) may include a non-permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, etc. As another example, the non-permanent mass storage device such as a ROM, an SSD, a flash memory, a disk drive, etc. may be a separate permanent storage device distinct from the memory. In addition, the memory (220) may store an operating system (OS) and at least one program code (e.g., an NLP system of the present specification).

[0069] These software components may be loaded from a computer-readable recording medium separate from the memory (220). This separate computer-readable recording medium may be a recording medium that can be directly connected to the user terminal (200), and may include, for example, a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. Alternatively, the software components may be loaded into the memory (220) through a communication module (230) other than a computer-readable recording medium. For example, at least one program may be loaded into the memory (220) based on a computer program (e.g., the NLP system of the present specification) that is installed by files provided by developers or a file distribution system that distributes installation files of applications through the communication module (230).

[0070] An input / output interface (not shown) may be a means for interfacing with a device (e.g., a keyboard, a mouse, etc.) for input or output that may be connected to or included in a user terminal (200). The input / output interface (130) may be an element configured separately from the processor (210), but is not limited thereto, and the input / output interface may be configured to be included in the processor (210).

[0071] The communication module (230) may provide a configuration or function for the server and the user terminal (200) to communicate with each other via a network. In addition, the communication module (230) may provide a configuration or function for the user terminal (200) to communicate with other external devices. For example, control signals, commands, data, etc. provided under the control of the processor (210) may be transmitted to the server and / or external devices via the communication module (230) and the network.

[0072] Meanwhile, although not illustrated in FIG. 4, the user terminal (200) may further include a display device. Alternatively, the user terminal (200) may be connected to an independent display device via wired or wireless communication to transmit and receive data between the two. For example, radiological images or reports, medical information, and additional information based on medical information may be provided to the user via the display device.

Claims

1. A method for creating a natural language processing (NLP) model for analyzing a radiology report on the spine, A step of preprocessing texts contained in radiological reports included in a training data set selected from radiological reports on the spine; A step of mapping the spine-related terms extracted from the above preprocessed texts to terms included in a pre-built term library; and A method comprising the step of summarizing the radiology reports by combining the mapped terms according to a preset format.

2. In paragraph 1, The above preprocessing step is, A method for performing at least one of section segmentation, sentence segmentation, and normalization on the above texts.

3. In paragraph 1, A method further comprising the step of evaluating the results of the summary using a test data set selected from among the radiological reports for the spine.

4. In paragraph 3, The above evaluation steps are: A method for performing the evaluation using at least one of recall, precision, accuracy, and F1 score.

5. In paragraph 1, Radiographic reports on the above spine are: A method comprising a report on one or more images selected from the group consisting of X-ray radiographic images, magnetic resonance imaging (MRI) images, ultrasound images, endoscopic images, elastography images, thermal images, positron emission tomography (PET) images, single photon emission computed tomography (SPECT) images, optical coherence tomography (OCT) images, and computed tomography (CT) images.

6. In paragraph 1, The above-mentioned built terminology library is, A method that includes words for spinal pathology, words for structural problems of the spine, and words for spinal diseases.

7. In paragraph 1, The above-mentioned built terminology library is, Listhesis, Scoliosis, Fracture, Spondylosis, Annular fissure, Disc bulge, Disc degeneration, Disc extrusion, Disc herniation, Disc protrusion, End plate edema or type 1 Modic change, Facet hypertrophy, Central stenosis, Foraminal stenosis, Nerve root displaced / compressed, Lateral recess stenosis, Spondylolysis, Schmorl's node, Osteophyte, Disc space narrowing, Any A method comprising at least one selected from the group consisting of stenosis, disc sequestration, intradiscal vacuum, and combinations thereof.

8. In paragraph 6, The above-mentioned built terminology library is, A method comprising a combination of Disc extrusion, End plate edema or type 1 Modic change, Central stenosis, Foraminal stenosis, Nerve root displaced / compressed, Lateral recess stenosis, and Any stenosis.

9. In paragraph 1, The above-mentioned radiological report on the spine is a radiological report on a spinal disease or low back pain, or a subject suffering from a spinal disease or low back pain.

10. Memory in which at least one program is stored; and A processor comprising: a processor configured to execute at least one program to perform at least one operation; The above processor, A computing device that preprocesses texts included in radiology reports included in a training data set selected from radiology reports on the spine, maps terms related to the spine extracted from the preprocessed texts to terms included in a pre-built terminology library, and summarizes the radiology reports by combining the mapped terms according to a pre-established format.

Images