Diagnosis content extraction method and system, medium and product

Abstract

The invention provides a diagnosis content extraction method and system, a medium and a product. The method comprises the following steps: receiving a to-be-processed document; reading an extraction area divided by a user on the to-be-processed document; based on structural information of the diagnosis content in the extraction area, a corresponding information extraction mode is selected for information extraction, and the structural information comprises a text or a table. The efficiency of extracting the diagnosis content is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of information extraction, and in particular to a diagnostic content extraction method, system, medium and product. Background Art

[0002] When extracting patient information, the existing technology first requires manual designation of the extraction area, then extracting key information from the extraction area, and finally collating and outputting it.

[0003] The existing process has limitations, which are mainly reflected in the following four aspects: 1. The granularity of the extraction area is too small, and manual specification is time-consuming and laborious; 2. The location of the extraction area is fixed and cannot adapt to variable-length content; 3. Different files have different structures, and it is difficult to unify the processing flow; 4. The types of key information are diverse, among which key information may exist in the form of tables and texts, and the forms in different files are different, making it difficult to design the extraction mode. Summary of the invention

[0004] In view of this, the purpose of the embodiments of the present application is to provide a diagnostic content extraction method, system, medium and product, which can improve the problems of the prior art that the processing flow is difficult to unify and the extraction mode is difficult to design.

[0005] In order to achieve the above technical objectives, the technical solutions adopted in this application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for extracting diagnostic content, the method comprising:

[0007] Receive documents for processing;

[0008] Read the extraction area divided by the user on the document to be processed;

[0009] Based on the structural information of the diagnostic content in the extraction area, a corresponding information extraction mode is selected to perform information extraction, wherein the structural information includes text or table.

[0010] Furthermore, before selecting the corresponding information extraction mode to extract information, the method further includes:

[0011] When the diagnostic content is a table, adjacent characters with a spacing less than a set threshold are aggregated into one cell.

[0012] Furthermore, before selecting the corresponding information extraction mode to extract information, the method further includes:

[0013] When the diagnosis content is text, discrete single characters in the diagnosis content are aggregated into lines, and then all the lines are integrated.

[0014] Furthermore, before selecting the corresponding information extraction mode to extract information, the method further includes:

[0015] Remove all interfering characters;

[0016] The interference characters of the current page are the same characters that appeared at the same position on the previous page.

[0017] Further, the selecting a corresponding information extraction mode to extract information includes: selecting a table information extraction mode to extract information from the diagnosis content, and the table information extraction mode includes:

[0018] Pair two cells in the same row as key-value pairs to obtain tuples of all rows;

[0019] Based on the key value of the tuple, extract the value that matches the key value row by row.

[0020] Further, the selecting a corresponding information extraction mode to extract information includes: when the diagnosis content is text, selecting a regular expression to extract information from the diagnosis content.

[0021] Furthermore, the selecting a corresponding information extraction mode to extract information includes: when the diagnosis content is text, selecting a word grammatical analysis method to extract information from the sub-diagnosis content.

[0022] In a second aspect, the present application also provides a diagnostic content extraction system, including:

[0023] A receiving module, configured to receive a document to be processed;

[0024] A reading module, configured to read an extraction area divided by a user on a document to be processed;

[0025] The information extraction module is configured to select a corresponding information extraction mode to extract information based on the structural information of the diagnostic content in the extraction area, wherein the structural information includes text or table.

[0026] In a third aspect, an embodiment of the present application further proposes a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer executes the above method.

[0027] In a fourth aspect, an embodiment of the present application further proposes a computer program product, including a computer program, which implements the above method when executed by a processor.

[0028] The invention adopting the above technical solution has the following advantages:

[0029] After reading the extraction area of ​​the document to be processed, the application selects the corresponding information extraction mode based on the structural information of the diagnostic content in the extraction area to extract the diagnostic content. The above process forms a modular process, which is conducive to modular design and is applicable to files of all formats, thereby unifying the processing process. At the same time, corresponding information extraction modes are used for diagnostic contents with different structural information, which improves the efficiency of diagnostic content extraction.

[0030] Before selecting the corresponding information extraction mode for information extraction, the present application performs the steps of aggregating discrete single characters into rows, removing interfering characters, and aggregating adjacent characters with a spacing less than a set threshold into a cell, thereby avoiding human modification after information extraction and improving the efficiency of diagnostic content extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The present application may be further described by the non-limiting embodiments given in the accompanying drawings. It should be understood that the following drawings only illustrate certain embodiments of the present application and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings may be obtained based on these drawings without creative effort.

[0032] Figure 1 A flow chart of the method provided in an embodiment of the present application.

[0033] Figure 2 Schematic diagram of the extraction area division method.

[0034] Figure 3 This is a schematic diagram of table structure data.

[0035] Figure 4 This is a diagram of the patient information under the title.

[0036] Figure 5 This is a schematic diagram of the results obtained from lexical analysis and grammatical analysis.

[0037] Figure 6 Schematic diagram of the diagnostic content extraction system.

[0038] Icon: 1-receiving module; 2-reading module; 3-information extraction module. DETAILED DESCRIPTION

[0039] The present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that in the drawings or descriptions, similar or identical parts use the same figure numbers, and the implementation methods not shown or described in the drawings are forms known to ordinary technicians in the relevant technical field. In the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0040] Please refer to Figure 1 The present application provides a diagnostic content extraction method, wherein the diagnostic content extraction method may include the following steps:

[0041] Step 110: receiving a document to be processed.

[0042] Step 120: read the extraction area divided by the user on the document to be processed.

[0043] Step 130 , based on the structural information of the diagnostic content in the extraction area, select a corresponding information extraction mode to extract information, where the structural information includes text or table.

[0044] In step 120, the extraction region division essentially serves the purpose of content extraction. Therefore, in designing the division region, we need to start from the document semantics and put the parts containing the content to be extracted and with similar structures in one region to facilitate the subsequent regional content extraction, rather than randomly specifying a region.

[0045] In this embodiment, the division of the extraction area is manually specified in implementation, and each extraction area should be guaranteed to contain all the content of a certain type of information to be extracted and be as small as possible. The basis for the division of the extraction area is only the location of the content to be extracted. S120 proposed in this embodiment extracts the diagnostic content in the extraction area after manual specification.

[0046] Each extraction region in the extraction region division should be guaranteed to contain all the content of a certain type of diagnostic information to be extracted and be as small as possible. Therefore, before performing region division, first try to analyze the extraction region and put the content with similar structures in one region. From this perspective, some similar structures can be summarized from these types of documents, such as title, patient information, report time, reporter, etc., and then corresponding content extraction regions can be assigned to these similar structures.

[0047] Therefore, in this embodiment, only the paragraphs containing the diagnostic information to be extracted are divided into extraction areas, and part of the table containing the diagnostic information to be extracted is divided into extraction areas.

[0048] For example, parts of a table may indeed be divided into the same extraction area because the extraction area contains the content you want to extract. However, the entire table does not necessarily need to be divided into the same extraction area, because the definition of an extraction area is the smallest possible area that contains the content to be extracted, and the content to be extracted may not be distributed throughout the entire table, but may only be a certain area in it.

[0049] In this embodiment, the underlying logic of dividing the design area is: first, clarify the content to be extracted, and then manually frame an area so that the area can ensure that it contains all the content of a certain type of extracted information, while being as small as possible to avoid additional irrelevant information.

[0050] For example Figure 2 As shown, Figure 2 The title part of the document to be processed is the R1 area, the patient information below the title is the R2 area, and the footer is the R3 area.

[0051] In order to represent R1, R2 and R3, this embodiment uses a quadrilateral to represent them. Specifically, the upper left corner and the lower right corner of a quadrilateral are selected to represent a quadrilateral. Therefore, the extraction area can be represented by a four-tuple (x0, y0, x1, y1), where (x0, y0) represents the coordinates of the upper left corner of the quadrilateral, and (x1, y1) represents the coordinates of the lower right corner of the quadrilateral. In this way, the extraction area can be represented and stored, which is convenient for judging whether the content is within the extraction area in the subsequent process.

[0052] This part refers to the specific way to divide a structure into regions. First, the similar structures of the document are summarized in (1). In this part, specific regions are divided for each structure according to the structures summarized above. The basis for the division here is to specify the maximum region that each structure may occupy. For example, for the structure of patient information, we need to give a four-tuple (x0, y0, x1, y1) to represent the extraction region of patient information. At this time, when manually setting the extraction region, the maximum region that each structure can occupy is specified, thereby realizing the extraction of variable-length content.

[0053] In this embodiment, when performing extraction area division, since this embodiment summarizes the structure of the area in advance, for each divided extraction area, in addition to the four-tuple information, the structural information represented by it, such as table and text, can also be known. Therefore, the identification of whether the collected area is a table or a paragraph text is achieved by pre-specifying when designing the extraction area.

[0054] In this embodiment, in order to extract title information, patient information and conclusion information through this method, the four-tuple information can be manually set to set three extraction areas R1, R2 and R3.

[0055] In order to avoid the semantic meaning of the extracted information being different from that of the original document to be processed after information extraction and thus requiring manual modification of the extracted information, in this embodiment, after S120 and before S130, semantic reconstruction is performed on the diagnostic content within the extraction area. The significance of semantic reconstruction is as follows: Assume that after reading the document to be processed and dividing the extraction area, in the diagnostic content within the extraction area, the sub-item separation disappears, the paragraph structure is missing, etc. After information extraction, manual modification is required, resulting in a reduction in information extraction efficiency.

[0056] Therefore, semantic reconstruction is to reconstruct the text paragraphs and restore the table information according to the semantics, so that there is no need to manually correct the extracted content.

[0057] The process of semantic reconstruction includes:

[0058] When the structure information of the diagnostic content is a table, the semantics of the table are mainly composed of cells and row-column structures. Therefore, the process of table semantic reconstruction is as follows: First, the cell structure needs to be restored. Specifically, the characters adjacent (the distance between characters is less than a specific threshold) within the extraction area are aggregated into a single cell. Thus, all cells are restored. After that, two cells in the same row are paired into a key-value pair. Here, only the case where the table has two columns is considered because all the tables in the document have two columns, or the columns where the key information is located have two columns.

[0059] For PDF files, since the tables in PDF lack formatting information and the grid structure is complex, if the table structure is to be fully restored, it may be necessary to combine means such as image recognition to achieve a complete analysis. Therefore, after restoring the cell structure, in this embodiment, the table is simplified to a multi-row two-column structure. For each row of the table, a binary tuple is constructed to represent the table item and the corresponding value. After that, multiple rows are repeatedly extracted until the end of the table.

[0060] When the diagnostic content is text, the discrete single characters within the extraction area are integrated into lines. For example, (take)(material)(,)(smear)->(take material, smear), and then all the lines are aggregated into text paragraphs, such as:

[0061] (5. Lymphocytes..., mature lymph)(ocytes...; individual cytoplasm...)->(5. Lymphocytes..., individual cytoplasm is slightly blue)

[0062] The process of semantic reconstruction also includes removing all interfering characters and identifying non-text characters.

[0063] By observing the samples, we can find that the appearance of interference characters has a certain pattern. Specifically, interference characters are characters that exist in the previous page, and the position of the character is the same as that in the previous page, but different from the position of the character in the current page (for example, the line is not the same). Based on this pattern, we can filter out the characters with the same content and position as those in the previous page in the current page as interference characters and remove them.

[0064] The optical character recognition (OCR) solution is used to realize the recognition of non-text characters.

[0065] In S130, different information extraction modes are used for different diagnostic contents of structural information, thereby overcoming the defects of diverse key information types and difficult extraction mode design.

[0066] When the diagnostic content in the extraction area is structured data such as a table, since the cell recovery has been completed in the above semantic reconstruction process, when extracting the content of the table, two adjacent cells in each row are combined into a key-value pair. For example, for Figure 3 From the table shown, the following key-value pair representation can be obtained, and each key-value pair constitutes a tuple.

[0067] "Original RBC": "0.50",

[0068] "Proerythroblast": "0.50",

[0069] "Mesoblast": "1.00",

[0070] "Metaerythroblast": "7.00"

[0071] Each key-value pair includes a key value and a value corresponding to the key value. For example, "metablast": "7.00", "metablast" is the key value, and 7.00 is the value corresponding to the original red blood cell. When this pair of key-value pairs is extracted to the excel template, it is only necessary to determine whether "metablast" exists in the key item of the key-value pair list. If it exists, the corresponding value "7.00" can be directly obtained by indexing "metablast" as the key value.

[0072] When the diagnostic content in the extraction area is a text paragraph, the information is extracted by constructing a regular expression extraction pattern, such as for a paragraph of text:

[0073] Bone marrow proliferation was obviously active, with granulocytes accounting for 14.00% and erythroids accounting for 9.00%, and the granulocyte:erythroid ratio = 1.56.

[0074] You can design a regular expression: Particle system accounts for ([0-9]+\.[0-9]+%)

[0075] Extraction result: 14.00%

[0076] When the diagnostic content in the extraction area is the patient information under the document title, it is extracted by using lexical grammar analysis, for example:

[0077] like Figure 4 For the patient information shown, first, the tool performs lexical analysis, traversing the characters from left to right and from top to bottom, and gets "surname" for the first time and "name" for the second time. At this time, the lexical analysis determines that the item ("name") is a token. After that, the token is given to the syntax analysis module, which determines that the token "name" belongs to a keyword, and the corresponding syntax is "name": "***". Therefore, the syntax analysis module will ask the lexical analysis module for another token to meet the syntax requirements. At this time, the lexical analysis module traverses the characters, "Du", "DuX", "DuXX", and the following characters are blank separators, so "DuXX" is regarded as a token and handed over to the syntax analysis module. The syntax analysis module gets two tokens "name" and "DuXX", and determines that it conforms to the syntax "name": "**", so it constructs a key-value pair "name: DuXX".

[0078] After constructing the first key-value pair, repeat the above process. For the missing content such as the bed number, special processing is performed. After getting the token "bed number", a subsequent token should be required according to the grammar. At this time, the subsequent token is "clinical diagnosis", which is a keyword and does not meet the grammatical requirements, so the key-value pair corresponding to the bed number is constructed as "bed number: empty".

[0079] Finally, the lexical analysis and grammatical analysis results are as follows Figure 5 shown.

[0080] On this basis, since the output result of the syntax analysis is also a key-value pair, the processing method for subsequent queries is the same as the second key-value pair matching method.

[0081] This embodiment also proposes a diagnostic content extraction system, such as Figure 6 As shown, including:

[0082] A receiving module 1, configured to receive a document to be processed;

[0083] A reading module 2, configured to read the extraction area divided by the user on the document to be processed;

[0084] The information extraction module 3 is configured to select a corresponding information extraction mode to extract information based on the structural information of the diagnostic content in the extraction area, where the structural information includes text or table.

[0085] This embodiment further proposes a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer executes the above-mentioned diagnostic content extraction method.

[0086] This embodiment also provides a computer program product, including a computer program, which implements the above-mentioned diagnostic content extraction method when executed by a processor.

[0087] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A diagnostic content extraction method, characterized in that: The method comprises: Receive documents for processing; Read the extraction area divided by the user on the document to be processed; Based on the structural information of the diagnostic content in the extraction area, a corresponding information extraction mode is selected to perform information extraction, wherein the structural information includes text or table.

2. The method according to claim 1, characterized in that: Before selecting the corresponding information extraction mode to extract information, the method further includes: When the diagnosis content is a table, adjacent characters whose spacing is less than a set threshold are aggregated into one cell.

3. The method according to claim 1, characterized in that: Before selecting the corresponding information extraction mode to extract information, the method further includes: When the diagnosis content is text, discrete single characters in the diagnosis content are aggregated into lines, and then all the lines are integrated.

4. The method according to claim 1, characterized in that: Before selecting the corresponding information extraction mode to extract information, the method further includes: Remove all interfering characters; The interference characters of the current page are the same characters that appeared at the same position on the previous page.

5. The method according to claim 2, characterized in that: The selecting a corresponding information extraction mode to extract information includes: selecting a table information extraction mode to extract information from the diagnosis content; The table information extraction mode includes: Pair two cells in the same row as key-value pairs to obtain tuples of all rows; Based on the key value of the tuple, extract the value that matches the key value row by row.

6. The method according to claim 4, characterized in that: The selecting a corresponding information extraction mode to extract information includes: when the diagnosis content is text, selecting a regular expression to extract information from the diagnosis content.

7. The method according to claim 4, characterized in that: The selecting a corresponding information extraction mode to extract information includes: when the diagnosis content is text, selecting a word grammatical analysis method to extract information from the sub-diagnosis content.

8. A diagnostic content extraction system, characterized in that: include: A receiving module, configured to receive a document to be processed; A reading module, configured to read an extraction area divided by a user on a document to be processed; The information extraction module is configured to select a corresponding information extraction mode to extract information based on the structural information of the diagnostic content in the extraction area, wherein the structural information includes text or table.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images