Method and system for extracting text information in NXML format
Through intelligent node recognition and deep recursive extraction algorithms, the problems of low efficiency and poor flexibility in the existing technology of NXML document extraction are solved, efficient and accurate information extraction is achieved, and NXML documents of different formats and structures are adapted to NXML documents, which improves the data quality of biomedical research.
Patent Information
- Application Number
- CN202510103085.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-22
- Publication Date
- 2025-08-15
AI Technical Summary
When extracting NXML documents in Gene REVIEW database, the prior art is inefficient, poor flexibility and high error rate, and cannot effectively handle multi-layer nested structures, which affects the accuracy of information extraction and the reliability of research results.
Intelligent node recognition algorithm and deep recursive extraction algorithm are used to identify first-level titles of NXML documents, build structural frameworks, recursively process multi-level nested structures, and store information in preset formats.
Improves the efficiency and accuracy of information extraction, reduces manual intervention, ensures the integrity and accuracy of research data, and adapts to different versions and formats of NXML documents.
Smart Images
Figure CN120493864A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer language and document recognition, and in particular to a method and system for extracting NXML format text information. Background Art
[0002] The NXML (NLM XML) format is a standardized markup language specifically designed for biomedical information. It is widely used in the publication of scientific literature, clinical research, and genomic data. This format was developed to enhance the storability, exchangeability, and reusability of data, meeting the needs of modern data management and information sharing. For example, the Gene REVIEW database is an information platform specifically focused on genetic diseases, aiming to provide clinicians, researchers, and patients with high-quality medical information and genetic research results. This database compiles a vast amount of information on genetic diseases, clinical characteristics, genetic mechanisms, and gene therapy, making it a vital data resource in the biomedical field. Within the vast number of NXML documents, researchers often need to extract key information, such as abstracts, titles, references, and specific paragraphs, to support their research and decision-making. Although some tools and methods exist for information extraction, they lack operational convenience and adaptability, failing to meet the needs of modern biomedical research.
[0003] However, as the size of the database expands, how to efficiently and accurately extract relevant information from massive NXML format files has become a key challenge.
[0004] Existing techniques for extracting NXML information from the Gene REVIEW database primarily employ manual parsing, rule-based parsing methods, and basic XML parsing libraries. Many researchers rely on manually reading and organizing NXML documents from Gene REVIEW. While this process ensures accurate information extraction, it is inefficient for large datasets and prone to errors due to human error.
[0005] At the same time, existing solutions lack flexibility when dealing with the diverse formats and content of NXML documents within Gene REVIEW. Many tools are unable to handle structures containing multiple layers of nested data, resulting in the loss of important information. Manual operations or inaccurate matching rules can also easily lead to errors in information extraction, compromising the reliability of scientific analysis results. Developing an efficient and accurate method to extract NXML-formatted text information is an urgent need.
[0006] Therefore, the current technology for extracting NXML format text information still needs to be further improved. Summary of the Invention
[0007] In view of this, the present invention proposes a method and system for extracting NXML format text information, which solves the technical problems of low efficiency, poor flexibility and high error rate in the prior art when extracting the diverse formats and different contents of NXML documents in Gene REVIEW.
[0008] The technical solution of the present invention is achieved as follows:
[0009] In one aspect, the present invention provides a method for extracting text information in NXML format, comprising the following steps:
[0010] S1, input and read the NXML document, and format the text content according to the element type and indentation level of the NXML document;
[0011] S2, identifies the first-level headings in the NXML document, determines the structural framework of the NXML document, and extracts the text content of the relevant paragraphs;
[0012] S3, recursively processes the multi-level nested structure and extracts information of all levels of nested structures;
[0013] S4, storing the extracted information according to the structure and outputting it as a file in a preset format.
[0014] Based on this technical solution, it is further preferred that the element type includes a paragraph or a list,
[0015] For paragraph elements, extract the text content and add the corresponding indentation;
[0016] For list elements, extract the text content of the list item and add bullets and indentation before the text.
[0017] On the basis of this technical solution, further preferably, if the list contains a nested structure, the nested structure is recursively processed.
[0018] On the basis of this technical solution, further preferably, the identifying of the first-level title in the NXML document specifically includes the following steps:
[0019] Search for a chapter element with a specific identifier in the NXML document, find the element, and extract part of its content, including the title and text content;
[0020] Traverse all sub-chapter elements of the found part and extract the text content of their titles and sub-elements;
[0021] The result is output as a string containing the formatted text.
[0022] On the basis of this technical solution, it is further preferred, more specifically, to include the following steps:
[0023] First, search for a chapter element with a specific ID in the document. If found, get the chapter title. If the chapter has a title tag, use its text. Otherwise, use the specific part of the text in the tag as the default title and format it. At the same time, extract the main text of the first paragraph.
[0024] Then iterate over all sub-chapter elements in the chapter, extract the titles of each sub-chapter, find and process the paragraphs and lists in the sub-chapter elements, and incorporate the processed text into the text content;
[0025] The final result contains formatted chapter text. If the specified chapter does not exist, a prompt message is returned.
[0026] On the basis of this technical solution, it is further preferred that the recursive processing of the multi-level nested structure specifically includes the following steps:
[0027] Traverse all sub-section elements, recursively call itself for each sub-section, increase the indentation level each time the recursive call is processed, and extract the text content of all levels;
[0028] Returns a list containing the structured text content. If no text content is extracted, an empty list is returned.
[0029] On the basis of this technical solution, further preferably, the preset file format is csv.
[0030] On the basis of this technical solution, it is further preferred that UTF-8 encoding is used to process text for reading NXML documents in the database.
[0031] On the basis of this technical solution, further preferably, the fixed format includes csv, Excel, JSON or xml.
[0032] On the basis of this technical solution, further preferably, the parsing library includes BeautifulSoup or pandas library.
[0033] On the basis of this technical solution, further preferably, the database includes Gene REVIEW, PubMed or GeneCards database.
[0034] In a second aspect, the present invention provides a system for extracting NXML formatted text information, comprising:
[0035] The reading module is used to input and read NXML documents and format the text content according to the element type and indentation level of the NXML document;
[0036] Parsing module, used to identify the first-level headings in the NXML document, determine the structural framework of the NXML document, and extract the text content of the relevant paragraphs;
[0037] Recursive module, used to recursively process multi-level nested structures and extract information of all levels of nested structures;
[0038] The result module stores the extracted information in a structure and outputs it as a file in a preset format.
[0039] On the basis of this technical solution, further preferably, the reading module includes a paragraph unit, a list item unit and a nested processing unit.
[0040] On the basis of this technical solution, it is further preferred that the parsing module includes a search unit, a traversal unit and a return unit.
[0041] The search unit is used to search for a chapter element with a specific identifier in the NXML document, and extract part of the content, including the title and text content, after finding the element;
[0042] The traversal unit is used to traverse all sub-chapter elements of the found part and extract the text content of its title and sub-elements;
[0043] The return unit is used for result output, outputting a string containing formatted text.
[0044] On the basis of this technical solution, it is further preferred that the result module includes an organization unit and an output unit, the organization unit is used to store the extracted information in a structure, and the output unit is used to output the unit in CSV format.
[0045] The method and system for extracting NXML formatted text information described in the present invention have the following beneficial effects compared to the prior art:
[0046] The present invention uses an intelligent node recognition algorithm to analyze and extract multi-level, multi-structured information from NXML documents. This method differs from traditional manual parsing or simple regular expression matching, making the information extraction process more efficient and accurate. Compared to existing flat data extraction methods, the present invention introduces a deep recursive extraction algorithm that can effectively handle nested lists and complex structures. This algorithm design allows the system to demonstrate greater flexibility when dealing with NXML documents of varying formats and structures. Furthermore, the method possesses excellent adaptability and can handle NXML documents of varying versions and formats, making it more widely applicable in related fields such as the Gene REVIEW database. This approach can significantly increase the speed of information extraction, reduce the need for manual intervention, ensure the integrity and accuracy of research data, and thus promote research progress and clinical applications in the biomedical field.
[0047] The system of the present invention improves the accuracy and efficiency of information extraction by building a flexible parsing framework and combining deep recursive extraction and data integration capabilities. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0049] Figure 1 This is a flow chart of the method for extracting NXML format text information according to the present invention;
[0050] Figure 2 This is a schematic diagram of the method for extracting NXML formatted text information according to Example 2 of the present invention;
[0051] Figure 3 (a) is an excerpt of an analysis report on the method for extracting NXML format text information in LAMA2 muscular dystrophy according to the present invention;
[0052] Figure 3 (b) is a schematic diagram comparing the method of extracting NXML format text information of the present invention and the parsing of BeautifulSoup in LAMA2 muscular dystrophy. DETAILED DESCRIPTION
[0053] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0054] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "preferred implementation", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.
[0055] In a preferred embodiment, Figure 1 As shown, a method for extracting NXML format text information includes the following steps:
[0056] S1, input and read the NXML document, and format the text content according to the element type and indentation level of the NXML document;
[0057] S2, identifies the first-level headings in the NXML document, determines the structural framework of the NXML document, and extracts the text content of the relevant paragraphs;
[0058] S3, recursively processes the multi-level nested structure and extracts information of all levels of nested structures;
[0059] S4, storing the extracted information according to the structure and outputting it as a file in a preset format.
[0060] In a preferred embodiment, specifically, the element type includes a paragraph or a list.
[0061] For paragraph elements, extract the text content and add the corresponding indentation;
[0062] For list elements, extract the text content of the list item and add bullets and indentation before the text.
[0063] In a preferred embodiment, specifically, if the list contains a nested structure, the nested structure is processed recursively.
[0064] In a preferred embodiment, specifically, the identifying of the first-level title in the NXML document comprises the following steps:
[0065] Search for a chapter element with a specific identifier in the NXML document, find the element, and extract part of its content, including the title and text content;
[0066] Traverse all sub-chapter elements of the found part and extract the text content of their titles and sub-elements;
[0067] The result is output as a string containing the formatted text.
[0068] In a preferred embodiment, more specifically, the following steps are included:
[0069] First, search for a chapter element with a specific ID in the document. If found, get the chapter title. If the chapter has a title tag, use its text. Otherwise, use the specific part of the text in the tag as the default title and format it. At the same time, extract the main text of the first paragraph.
[0070] Then iterate over all sub-chapter elements in the chapter, extract the titles of each sub-chapter, find and process the paragraphs and lists in the sub-chapter elements, and incorporate the processed text into the text content;
[0071] The final result contains formatted chapter text. If the specified chapter does not exist, a prompt message is returned.
[0072] In a preferred embodiment, the recursive processing of a multi-level nested structure specifically includes the following steps:
[0073] Traverse all sub-section elements, recursively call itself for each sub-section, increase the indentation level each time the recursive call is processed, and extract the text content of all levels;
[0074] Returns a list containing the structured text content. If no text content is extracted, an empty list is returned.
[0075] In a preferred embodiment, more specifically, the preset file format is csv.
[0076] Example 1
[0077] A method for extracting text information in NXML format, comprising the following steps:
[0078] S1 first extracts the NXML document, which is mainly used to recursively extract the text content of elements and their nested lists, and extract text according to the current indentation level.
[0079] Specifically, if the element is a paragraph ( ), it gets the paragraph text, adds the corresponding indentation, and then stores it in the result list; if the element is a list ( <list>), which loops through each list item ( <list-item>), extracts the text of the paragraphs, adds bullet points and indents to the text, and checks whether there are nested lists in the list items. If so, it recursively calls itself to process; the entire process generates a list containing the text organized according to structure and indentation for analysis or display.
[0080] S2, then identify the first-level headings in the NXML document, first determine the text structure of the entire NXML document, that is, first determine the first-level headings that record information in the NXML document, and then extract the relevant paragraphs. The specific implementation steps are as follows:
[0081] First try to find the document with a specific ID <sec>tag, if found, it will start extracting the content of that section and get the title of this section, specifically, if the section has <title> If there is no tag, use the text in it; if not, use the text of the last paragraph in the ID as the default title (and format it); then, extract the first paragraph< / title> < / sec> < / list> The main text content.
[0082] Next, iterate through all the children in the section <sec> tag, for each subsection, extract its title, then find the paragraphs in the direct child elements< / sec> and list <list>, by processing these elements and adding the returned text to the text content.
[0083] Finally, a string containing the formatted text of the section is returned. If the specified section does not exist, "No Data Found" is returned.
[0084] The reason for this design is to allow for structured access to content in complex documents, especially when there are multiple levels and tags.
[0085] S3,recursively processes all subparts and their descendant contents,,which enables it to handle nested structures at any level and extract,text at all levels.
[0086] The specific implementation route is somewhat similar to step S2, but the difference is: traverse all sub-parts <sec>, for each subsection, it calls itself recursively while increasing the indentation level. This enables handling of more deeply nested structures.
[0087] Ultimately, a list containing structured text content is returned, reflecting the organization and hierarchy of the current section and all its subsections. If no content was extracted, an empty list is returned. In short, self-recursive calls are used to continuously drill down into the subsection hierarchy, extracting all nested content.
[0088] S4, all the extracted information is stored in a dictionary according to the structure, and then the output result is output in CSV format according to the set path and file name.
[0089] Example 2
[0090] like Figure 2 As shown, a method for extracting NXML format text information includes the following steps:
[0091] S1. NXML document loading: An efficient file loading mechanism is used to ensure fast loading of NXML documents stored in the GeneREVIEW database. UTF-8 encoding is used to process text, supporting the reading of files in different encoding formats and ensuring compatibility with diverse documents.
[0092] S2, data parsing and chapter identification: Use advanced parsing libraries, such as BeautifulSoup, to parse NXML document content. During the parsing process, the system builds a data tree based on the NXML structure, and intelligently identifies and marks chapters such as titles, paragraphs, lists, and sub-paragraphs to facilitate subsequent data management and information extraction.
[0093] S3, Information Extraction Algorithm: A deep recursive extraction algorithm is designed to extract information layer by layer from complex nested structures. This algorithm intelligently determines the type of the current node and determines the format and processing of the extracted content. For example, it extracts text from paragraph nodes and items from list nodes, ensuring efficient extraction of all information.
[0094] Specifically, deep recursive extraction is packaged into a function (extract_content). When encountering a list node, the function searches for list-items within it, processes each list-item, and further checks for nested lists. If nested lists exist, it calls itself to extract the nested content. Similarly, in addition to the list structure, this method also uses another function (extract_section_with_subsections) to extract the contents of a section and its subsections. It traverses each child node and recursively calls itself to extract the subsection contents. This approach ensures that all nested sections can be accessed, thereby extracting all the required information.
[0095] S4, Results Integration and Output: The extracted information is formatted into a structure that is easy to read and analyze, and stored in CSV or Excel format. This process makes subsequent data analysis and research more efficient. The output includes a summary of key information, making it easier for researchers to quickly access the data they need.
[0096] In this embodiment, more specifically, UTF-8 encoding is used to process text for reading NXML documents in the database.
[0097] In this embodiment, specifically, the storage format includes csv, Excel, JSON or xml.
[0098] In this embodiment, specifically, the parsing library used during parsing includes BeautifulSoup or pandas library.
[0099] In this embodiment, specifically, the database includes Gene REVIEW, PubMed or GeneCards database.
[0100] Example 3
[0101] A system for extracting NXML formatted text information, comprising:
[0102] The reading module is used to input and read NXML documents and format the text content according to the element type and indentation level of the NXML document;
[0103] Parsing module, used to identify the first-level headings in the NXML document, determine the structural framework of the NXML document, and extract the text content of the relevant paragraphs;
[0104] Recursive module, used to recursively process multi-level nested structures and extract information of all levels of nested structures;
[0105] The result module stores the extracted information in a structure and outputs it as a file in a preset format.
[0106] In this embodiment, specifically, the reading module includes a paragraph unit, a list item unit and a nesting processing unit.
[0107] In this embodiment, the parsing module specifically includes a search unit, a traversal unit and a return unit.
[0108] The search unit is used to search for a chapter element with a specific identifier in the NXML document, and extract part of the content, including the title and text content, after finding the element;
[0109] The traversal unit is used to traverse all sub-chapter elements of the found part and extract the text content of its title and sub-elements;
[0110] The return unit is used for result output, outputting a string containing formatted text.
[0111] In this embodiment, specifically, the result module includes an organization unit and an output unit. The organization unit is used to store the extracted information in a structure, and the output unit is used to output the unit in CSV format.
[0112] In a preferred embodiment, the method and system for extracting NXML formatted text information include, but are not limited to, using Python language as the core and utilizing BeautifulSoup and pandas libraries for NXML parsing and data processing.
[0113] In a preferred embodiment, the method and system for extracting NXML format text information can also be based on Java language and combined with DOM or SAX parsing library, and also have good stability and performance.
[0114] In a preferred embodiment, specifically, Figure 3 (a) shows the application of the method for extracting NXML format text information described in Example 2 in medical research data processing.
[0115] Get the NXML data downloaded from the Gene Review official website. Take LAMA2 muscular dystrophy as an example. The database records the genetic disease guidance related to muscular dystrophy in a complex and comprehensive manner. According to the basic parsing of the existing BeautifulSoup, the relevant content will be presented as a paragraph.
[0116] Specifically, Figure 3 (a) Describes the mode of inheritance of LAMA2 muscular dystrophy and lists the risk of inheritance within a family. For example, if both parents are carriers of the pathogenic mutation, what is the risk of a child developing the disease and being a carrier? In addition, if only one parent is a carrier of the pathogenic mutation, what is the risk of a child developing the disease and being a carrier?
[0117] At the same time, the comparison between the present invention and BeautifulSoup is as follows Figure 3 As shown in (b), it can be seen that the relationship between each paragraph and list is extremely clear, and each subheading and subheading and related content can be clearly seen.
[0118] In summary, the present invention provides a method and system for extracting text information in NXML format. The method has good adaptability and can handle NXML documents of different versions and formats, making it more widely used in related fields such as the Gene REVIEW database.
[0119] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.< / sec> < / list>
Claims
1. A method for extracting NXML formatted text information, characterized in that: The steps include: S1, input and read the NXML document, and format the text content according to the element type and indentation level of the NXML document; S2, identifies the first-level headings in the NXML document, determines the structural framework of the NXML document, and extracts the text content of the relevant paragraphs; S3, recursively processes the multi-level nested structure and extracts information of all levels of nested structures; S4, storing the extracted information according to the structure and outputting it as a file in a preset format.
2. The method for extracting NXML formatted text information according to claim 1, wherein: The element type includes paragraph or list, For paragraph elements, extract the text content and add the corresponding indentation; For list elements, extract the text content of the list item and add bullets and indentation before the text.
3. The method for extracting NXML formatted text information according to claim 2, wherein: If the list contains a nested structure, the nested structure is processed recursively.
4. The method for extracting NXML formatted text information according to claim 1, wherein: The method of identifying the first-level title in the NXML document specifically includes the following steps: Search for a chapter element with a specific identifier in the NXML document, find the element, and extract part of its content, including the title and text content; Traverse all sub-chapter elements of the found part and extract the text content of their titles and sub-elements; The result is output as a string containing the formatted text.
5. The method for extracting NXML formatted text information according to claim 4, wherein: More specifically, The following steps are involved: First, search for a chapter element with a specific ID in the document. If found, get the chapter title. If the chapter has a title tag, use its text. Otherwise, use the specific part of the text in the tag as the default title and format it. At the same time, extract the main text of the first paragraph. Then iterate over all sub-chapter elements in the chapter, extract the titles of each sub-chapter, find and process the paragraphs and lists in the sub-chapter elements, and incorporate the processed text into the text content; The final result contains formatted chapter text. If the specified chapter does not exist, a prompt message is returned.
6. The method for extracting NXML formatted text information according to claim 5, wherein: The recursive processing of the multi-level nested structure specifically includes the following steps: Traverse all sub-section elements, recursively call itself for each sub-section, increase the indentation level each time the recursive call is processed, and extract the text content of all levels; Returns a list containing the structured text content. If no text content is extracted, an empty list is returned.
7. The method for extracting NXML formatted text information according to claim 1, wherein: The preset file format is csv.
8. A system for extracting NXML formatted text information, characterized in that: include: The reading module is used to input and read NXML documents and format the text content according to the element type and indentation level of the NXML document; Parsing module, used to identify the first-level headings in the NXML document, determine the structural framework of the NXML document, and extract the text content of the relevant paragraphs; Recursive module, used to recursively process multi-level nested structures and extract information of all levels of nested structures; The result module stores the extracted information in a structure and outputs it as a file in a preset format.
9. The system for extracting NXML formatted text information according to claim 8, wherein: The reading module includes a paragraph unit, a list item unit and a nesting processing unit.
10. The system for extracting NXML formatted text information according to claim 8, wherein: The parsing module includes a search unit, a traversal unit, and a return unit. The search unit is used to search for a chapter element with a specific identifier in the NXML document, and extract part of the content, including the title and text content, after finding the element; The traversal unit is used to traverse all sub-chapter elements of the found part and extract the text content of its title and sub-elements; The return unit is used for result output, outputting a string containing formatted text.