Unstructured document data extraction, filling method and system

Abstract

The application discloses a structured document data extraction and filling method and system, and relates to the technical field of table content extraction. The method comprises the following steps: acquiring a to-be-processed document; the to-be-processed document is a structured document storing a table; performing format conversion on the to-be-processed document to obtain a first document in XML format; performing information extraction on the first document to obtain table-stored content; storing the position of a key in the table, the position of a value in the table and the extracted table-stored content to obtain a second document in JSON format; and storing the second document as a third document in CSV format or Excel format. The application can improve efficiency and ensure the accuracy of the result.

Description

Technical Field

[0001] This invention relates to the field of table content extraction technology, and in particular to a method and system for extracting and filling unstructured document data. Background Technology

[0002] In materials science research, many key parameters related to the chemical composition and properties of materials are stored in unstructured documents such as doc or docx files. Extracting this parameter information from these documents is crucial for building materials science databases and for conducting materials research. Traditional extraction methods include manual extraction or forcibly converting the data into structured documents. The former is more accurate but requires significant manpower, while the latter, although direct, can lead to data format corruption, such as garbled text and mismatched key-value pairs. To balance time efficiency and data quality, a standardized workflow can be defined to manage the entire data extraction and re-filling process, thus completing the data transition from unstructured to structured or semi-structured. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for extracting and filling unstructured document data, which can improve efficiency while ensuring the accuracy of the results.

[0004] To achieve the above objectives, the present invention provides the following solution:

[0005] A method for extracting unstructured document data, comprising:

[0006] Obtain the document to be processed; the document to be processed is an unstructured document containing tables.

[0007] The document to be processed is converted to obtain a first document in XML format;

[0008] Information is extracted from the first document to obtain the content stored in a table;

[0009] The positions of the keys and values ​​in the table, along with the extracted table content, are stored to obtain a second document in JSON format.

[0010] Store the second document as a third document in CSV or Excel format.

[0011] Optionally, after storing the second document as a third document in CSV or Excel format, the method further includes:

[0012] The third document is processed using the λ function.

[0013] Optionally, the step of extracting information from the first document to obtain the table storage content specifically includes:

[0014] The XPath language is used to extract information from the first document to obtain the content stored in a table.

[0015] Optionally, storing the second document as a third document in CSV or Excel format specifically includes:

[0016] The extracted table content is stored according to the positions of the keys and values ​​in the table to obtain a third document in CSV or Excel format.

[0017] An unstructured document data extraction system, comprising:

[0018] The acquisition module is used to acquire the document to be processed; the document to be processed is an unstructured document that stores tables.

[0019] The format conversion module is used to convert the document to be processed into a first document in XML format.

[0020] The extraction module is used to extract information from the first document to obtain the content stored in a table.

[0021] The first storage module is used to store the positions of the keys in the table, the positions of the values ​​in the table, and the extracted table content to obtain a second document in JSON format;

[0022] The second storage module is used to store the second document as a third document in CSV or Excel format.

[0023] Optionally, the unstructured document data extraction system further includes:

[0024] The data cleaning module is used to process the third document using the λ function.

[0025] Optionally, the extraction module specifically includes:

[0026] The extraction unit is used to extract information from the first document using the XPath language to obtain the content stored in a table.

[0027] Optionally, the second storage module specifically includes:

[0028] The second storage unit is used to store the extracted table content according to the position of the key and the position of the value in the table to obtain a third document in CSV or Excel format.

[0029] An unstructured document data filling method, applied to any of the unstructured document data extraction methods described above, the filling method comprising:

[0030] An empty data template in XML format is obtained based on the XSD specification;

[0031] The content stored in the third document obtained according to the unstructured document data extraction method described above is stored in the empty data template to obtain a filled file.

[0032] An unstructured document data population system, comprising:

[0033] The template building module is used to obtain empty data templates in XML format based on the XSD specification;

[0034] The filling module is used to store the content stored in the third document obtained according to the unstructured document data extraction method described above into the empty data template to obtain a filled file.

[0035] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0036] The document to be processed is first converted into an XML document. Then, information is extracted from the XML document to obtain the content stored in a table. The positions of the keys and values ​​in the table, as well as the extracted table content, are stored to obtain a JSON document. The JSON document is then stored as a third-party document in CSV or Excel format. No manual operation is required, and the entire process is free of garbled characters, thus improving efficiency while ensuring the accuracy of the results. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart for extracting unstructured document data provided in an embodiment of the present invention;

[0039] Figure 2 This is a flowchart of a method for extracting and refilling unstructured document data provided in an embodiment of the present invention;

[0040] Figure 3 A diagram illustrating tabular data stored in a JSON format file;

[0041] Figure 4 This is a diagram illustrating how tabular data exists in a JSON file.

[0042] Figure 5 This is a schematic diagram of the personalized key-value configuration information of the present invention;

[0043] Figure 6 This is a flowchart of the extraction process. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] like Figure 1 As shown, this embodiment of the invention provides a method for extracting unstructured document data, including:

[0047] Step 101: Obtain the document to be processed; the document to be processed is an unstructured document containing tables. Traditionally, unstructured documents refer to data without a fixed format, such as Word, PDF, PPT, and various formats of images and videos. Semi-structured data refers to data without a relational model but with a basic fixed structural pattern, such as log files, XML documents, and JSON documents. Structured data, on the other hand, refers to data with a relational model, managed in the form of relational database tables. For example, doc and docx files are unstructured, while XML and JSON files are semi-structured, and CSV and Excel files are considered structured documents.

[0048] Step 102: Convert the document to be processed to obtain a first document in XML format.

[0049] Step 103: Extract information from the first document to obtain the content stored in the table.

[0050] Step 104: Store the positions of the keys and values ​​in the table, as well as the extracted table content, to obtain a second document in JSON format.

[0051] Step 105: Save the second document as a third document in CSV or Excel format.

[0052] In practical applications, after storing the second document as a third document in CSV or Excel format, the following steps are also included:

[0053] The third document is processed using the λ function.

[0054] In practical applications, the step of extracting information from the first document to obtain the table storage content specifically includes:

[0055] The XPath language is used to extract information from the first document to obtain the content stored in a table.

[0056] In practical applications, storing the second document as a third document in CSV or Excel format specifically includes:

[0057] The extracted table content is stored according to the positions of the keys and values ​​in the table to obtain a third document in CSV or Excel format.

[0058] In practical applications, since .doc is the document format used by Microsoft Office Word 2003 and .docx is the document format used by Microsoft Office Word 2007, both are file formats saved by MS Office's Word tools. Taking the Python scripting language as an example, the win32com module mainly provides Python with the ability to call Windows underlying components to manipulate Word, Excel, PowerPoint, etc., so this tool can be used to manipulate .doc or .docx files. Because in actual testing, it was found that key-value data could not be extracted well from either .doc or .docx files, but after converting the .doc or .docx file to XML format, all the key-value data expected to be extracted is stored in... <w:tbl> ...< / w:tbl> The tag is used to convert the document to be processed into a first document in XML format. <w:tbl>These are tag symbols for XML files, where 'w' indicates a namespace and 'tbl' indicates tabular data. Since most of the key-value data extracted in this invention appears in irregular tables within doc or docx files, the focus here is primarily on extracting... <w:tbl>The data within the labels, and the key value data they subsequently refer to, will also be extracted from these irregular tables.

[0059] In practical applications, XML files can easily have all table information extracted using the XPath language. This has two main advantages: firstly, it facilitates identification and extraction; secondly, it avoids duplicate or garbled characters. Therefore, XPath is used here to extract information from the first document to obtain the table content. The actual content of a table in an XML file is based on... <w:tbl> 、 <w:tr> 、 <w:tc> 、 <w:p> 、 <w:r> 、 <w:t>The data is composed of five levels of tags, from largest to smallest. That is, `tbl` represents a complete table, `tr` represents a row within `tbl`, `tc` represents a cell within each `tr` row, `p` represents paragraph information within each `tc` cell, `r` represents line information within `p`, and `t` represents text information within `r`. Therefore, combining the text within all `t` tags of a `tc` represents the information in a single cell. By using XPath statements to focus on each `tr` within `tbl`, all information in `tbl` can be extracted and transformed into a two-dimensional row-column structured data.

[0060] In practical applications, based on the obtained "two-dimensional row and column structure data," an intermediate storage state needs to be selected to support further data extraction. JSON files offer more readable structured content, allowing for multi-level nested structures. Most importantly, JSON's unique key-value object data organization allows for additional configuration information to be written into the file. During parsing, it's only necessary to distinguish whether the required value is data or configuration information based on a specific key. Currently, all table information has been extracted, but the table arrangement is not uniform; some are arranged by column name by row, while others are by column. Furthermore, some data values ​​are not located in a specific row. Therefore, it's necessary to write certain configurations so that the program can perform data extraction operations at specific locations based on the configuration. Only by providing a JSON file format can specific configurations be written. Therefore, a JSON file format is chosen here, storing the positions of the keys and values ​​in the tables, along with the extracted table content, to obtain a second JSON document.

[0061] A detailed introduction to the configuration information is provided below: The configuration rules are a set of custom specifications. Specifying the position of key-value pairs provides users with a one-time solution for data extraction strategies under a certain template. The configuration information adopts a "key-value description" strategy. The "key" field specifies the relative position of the key in the table, while the "value" field specifies the relative position of the value. Keys and values ​​are matched one-to-one, and in principle, incorrect matching will not occur. To facilitate users in customizing personalized matching strategies, a set of relatively easy-to-understand and use configuration rules is provided below, as shown in Table 1:

[0062] Table 1

[0063]

[0064]

[0065]

[0066] In practical applications, personalized extraction configuration refers to a configuration description that enables data extraction based on the user's individual needs, while a standardized description is a crucial prerequisite for standardized data extraction. Based on the configuration information written in the JSON file, the program can automatically extract information from specified locations and obtain key-value pairs. These key-value pairs can be viewed as a one-dimensional result, with each key mapping to a corresponding value. Therefore, a structured document such as CSV or Excel can be chosen to store all the extracted data, and the second document can be stored as a third document in CSV or Excel format.

[0067] In practical applications, the most basic information includes some private information. To address the issue of desensitizing private information in documents, two approaches can be considered: one is regularized field filtering; the other is using sentiment analysis techniques from natural language processing to identify and filter sensitive words in the document. The first approach is more efficient but has insufficient coverage; the second approach can identify some special words and can serve as a supplement to regularized filtering, thus better achieving the initial desensitization and protection of material data. This represents a balance between the data extractor and the data recipient, and also considers and addresses potential issues such as copyright protection and data quality.

[0068] Data cleaning is necessary because, when it's impossible to ensure that every doc or docx file accurately describes the key-value data, it's essential to clean the extracted data to unify it under a single template. This process primarily uses regular expressions to achieve a rapid data cleaning workflow, providing initial assurance of data quality. Therefore, after obtaining the third-party documents, privacy information anonymization and data cleaning can be performed.

[0069] This invention allows users to extract location configuration information based on personalized data. During the data extraction process, the data undergoes multiple format conversions. In the personalized configuration-based extraction process, JSON file format serves as the intermediate form of the data and the channel for writing configuration information. Configuration information is formed based on the user-specified data location description, and standardized extraction is completed based on the configuration. To address field mismatches or information anonymization issues that arise during data extraction, necessary regularization methods are also part of the work in this invention.

[0070] In response to the above methods, embodiments of the present invention also provide an unstructured document data extraction system, comprising:

[0071] The acquisition module is used to acquire the document to be processed; the document to be processed is an unstructured document that stores tables.

[0072] The format conversion module is used to convert the document to be processed into a first document in XML format.

[0073] The extraction module is used to extract information from the first document to obtain the content stored in a table.

[0074] The first storage module is used to store the positions of the keys in the table, the positions of the values ​​in the table, and the extracted table content to obtain a second document in JSON format.

[0075] The second storage module is used to store the second document as a third document in CSV or Excel format.

[0076] In practical applications, unstructured document data extraction systems also include:

[0077] The data cleaning module is used to process the third document using the λ function.

[0078] In practical applications, the extraction module specifically includes:

[0079] The extraction unit is used to extract information from the first document using the XPath language to obtain the content stored in a table.

[0080] In practical applications, the second storage module specifically includes:

[0081] The second storage unit is used to store the extracted table content according to the position of the key and the position of the value in the table to obtain a third document in CSV or Excel format.

[0082] This invention also provides a method for filling unstructured document data, applied to the unstructured document data extraction method described in the above embodiments, the filling method comprising:

[0083] An empty data template in XML format is obtained based on the XSD specification;

[0084] The content stored in the third document obtained according to the unstructured document data extraction method described in the above embodiments is stored in the empty data template to obtain a populated file. Thanks to the data format constraints of the XSD specification for XML files, XSD can provide a second quality assurance for the data in the XML file after the data is populated. The data population follows the description based on the XSD (XML Schema Definition) specification. XSD defines the elements and structures that can appear in the document, which can be regarded as the validation and constraint of XML document data. In the process of data re-population, the empty XML template file obtained based on the XSD specification is the carrier of data population. After the data population is completed, the data is transformed from unstructured to semi-structured, improving the usability and application value of the data.

[0085] In practical applications, XSD (Extended Data Formatting) is a language for defining XML file formats. XSD-defined data templates support the definition of various data types and extensibility, and provide support for data validation, data protection, and data communication. The reason for needing XSD-defined data templates is that data in Word document formats lacks a standardized descriptive form, making it difficult to describe or use in a standardized way. Therefore, XSD is used to define XML data templates.

[0086] In practical applications, XML files can leverage the XSD specification to standardize data, ensuring data integrity and quality. XML data tags support customization and multi-level nesting, granting users greater autonomy. XML provides more readable data descriptions and eliminates data ambiguity (path uniqueness). Using a semi-structured XML data model is flexible and autonomous. The final data storage format can be flexibly determined for different research purposes and data usage scenarios. Therefore, this embodiment selects an XML template for population.

[0087] In practical applications, remote procedure calls implemented using the gRPC framework collect documents and store the populated files. gRPC is a high-performance, open-source RPC (Remote Procedure Call) framework, originating from Google and developed based on the ProtoBuf serialization protocol, supporting multiple languages ​​(Golang, Python, Java, etc.). Because of its support for the HTTP / 2 protocol, gRPC has promising prospects in the development of client-side backend services for Android, iOS, and other operating systems. gRPC provides a simple method for defining services, while clients can fully utilize the features of HTTP / 2 streams, thus helping to save bandwidth, reduce the number of TCP connections, and conserve CPU usage. In summary, the gRPC framework enables rapid remote document extraction and storage.

[0088] This embodiment has the following technical effects:

[0089] This embodiment generates a data template based on the XSD normalized description, extracts location information according to personalized key-value data, and performs data re-population based on data extracted from configured doc or docx files. The final data storage format is an empty XML data file obtained based on the XSD specification. If the extraction process is not performed locally, gRPC technology can be used for remote procedure calls to upload the obtained XML file to the database for centralized and unified data management. This approach is suitable for tasks that require extracting data streams from unstructured documents and converting them into structured or semi-structured data storage.

[0090] This invention also provides an unstructured document data population system, comprising:

[0091] The template building module is used to obtain empty data templates in XML format based on the XSD specification;

[0092] The filling module is used to store the content stored in the third document obtained by the unstructured document data extraction method according to the above embodiments into the empty data template to obtain a filled file.

[0093] like Figure 2 As shown, this embodiment of the invention provides a more specific method for extracting and re-filling unstructured document data, with the following specific steps:

[0094] Step 1: Obtain the files to be converted: Obtain the data file path, get all the doc or docx files to be converted, and specify them as $Files. The doc or docx files contain tables, as shown in Table 2.

[0095] Table 2

[0096]

[0097] Step 2: Standardize document format: Convert doc or docx files into XML file format to obtain $XMLs. For each $XMLs file, only the data storage format has been changed. This facilitates the subsequent data extraction process.

[0098] Specifically, for all files in $Files, use the T function (from the win32com library) to convert between .doc and .docx file formats under Windows. The operation method is as follows:

[0099]

[0100] Step 3: Perform preliminary extraction and intermediate storage of key-value data based on $XMLs. Since the data to be extracted is mainly stored in tables within doc or docx files, and table types have specific identifiers in XML files, the content of the tables in the converted XML files is extracted using XPath statements based on these identifiers. Then, JSON format is selected as the intermediate storage state for the data according to the specifications.

[0101] This step uses XPath technology to extract the table containing the main key-value data from the $XMLs obtained in step two. This is a preliminary extraction step, and JSON format is selected as the intermediate storage state for the data according to the specification. Specifically, it is operated according to the following formula:

[0102]

[0103] δ is used for special format processing of tables, mainly to convert the nested structure of XML into the unique nested structure of JSON. tbl represents each table in XML, i can be used as a key to describe the position of each table, and N represents the number of tables to be extracted in $XML.

[0104] The tabular data is stored in JSON, and its format must be... Figure 3 The format is as follows: values ​​are the converted table data, while tag, key, and value are reserved fields for writing personalized configuration information in step four.

[0105] Step four, the difficulty of the extraction process lies in determining the location of key-value information. When one key corresponds to one value, the location for key-value extraction can be specified through user-customized location configuration. This configuration process has a custom specification provided to the user, and this custom specification is implemented by mimicking the slicing method in high-level programming languages. This ensures that users can accurately determine the key-value extraction location with minimal programming knowledge and maximum efficiency. Based on the $JSONs obtained in Step three, in each JSON file, the user's personalized configuration is written into the three positions of tag, key, and value in the general composition format. This configuration information is obtained after the user writes the standardized representation of Table 1 based on the key-value positions in the table. This configuration information is synchronized with Step three and, along with the table information in the XML, is extracted and injected into the JSON. The result is as follows. Figure 4 and Figure 5 As shown, based on the user's actual personalized needs, the key-value information extraction location configuration information is written according to the given description rules.

[0106] Step 5: After obtaining the $JSONs containing configuration information after completing Step 4, we begin extracting data. Based on the configuration already written into the JSON, we directly extract the values ​​part according to the tag and value information in the JSON. The extraction results are shown in Table 3.

[0107] Table 3

[0108]

[0109] Based on the personalized configuration obtained in step four, data is extracted and temporarily stored or stored in a CSV structure. The data extraction is based on list slicing technology. Slicing is a feature of most programming languages, and the slicing specifications and descriptions of most programming languages ​​are quite similar, making it easy to learn. Figure 6 Flowchart of the extraction process:

[0110] cell 1，1 to cell m,n This is an abstract description of a table with m rows and n columns, representing it as an m-row, n-column matrix arrangement. Within this matrix arrangement, the positions are defined in the previously written key and value configuration information. Figure 6 The given reference indicates that the key is in column 1 and the value is in column n. Therefore, following the arrow's direction, we obtain one-to-one key-value pair groups, and the cell... 1，1 The value corresponding to the key is cell. 1,n And so on, cell m，1 The corresponding value is cell m,n Finally, the sequence of one-to-one key-value pairs is converted into a CSV structure for storage, with all keys serving as headers and the values ​​as rows of data in the CSV file.

[0111] Based on the configuration information provided by the key and value, the corresponding key-value pairs are located. This configuration information supports simultaneous location across multiple columns and rows. The resulting one-to-one key-value pairs are then converted into a CSV file structure for storage. All $JSONs are processed in this step to obtain $CSVs.

[0112] Step Six: Perform further data cleaning and quality checks on the $CSVs obtained in Step Five. Regularization matching is introduced to further filter, clean, and validate the data, thereby improving data quality. Since the data in the original files lacks standardization, its values ​​may contain sensitive or redundant information. Therefore, regularization and natural language processing are used to further filter, clean, and validate the data, thus improving its quality.

[0113] Data cleaning primarily considers both keys and values. Key processing mainly involves using the `lang` file provided in the template file conforming to the XSD specification to map the key names. The formula is as follows:

[0114]

[0115] L is a language description file based on the data template defined by XSD, used to convert tag names (keys) in XML into names that exist in the target language set (currently mainly involving Chinese and English). Further processing by the λ function ensures that each column name (key) in the CSV has at least one corresponding value in L; this is a surjective function.

[0116] Step 7: Generate an XML empty data template based on the XSD specification, and populate the template described by the XSD specification with data from multiple CSV files.

[0117] Users utilize XSD technology to create data templates to receive the incoming final data. This automatically generated data template can also be manually modified with restrictions, as XML supports multi-level nested structures, allowing for the categorization of key-value pairs or the addition of hierarchical structures. Based on the XSD specification, an XML empty data template, `$empty`, is generated. This empty data template is used for data filling; it needs to be predefined, specifying the exact filling position for each key-value pair.

[0118] Read each CSV file in $CSVs to obtain the key-value pairs, and then use the following method to find the corresponding fill position in $empty:

[0119]

[0120] The `find` function is a method provided by most tools that support XML technology to query the location of elements within XML. The `prepath` function is based on XPath technology to obtain the path statement, and can be used to populate the `$empty` file with data.

[0121] Step 8: Data Saving: Save the cleaned data extracted in Step 6 into the empty data template obtained in Step 7 according to the specifications.

[0122] The following example illustrates the filling process:

[0123] Assume the $empty file is structured as follows:

[0124]

[0125] The file format of a certain CSV file is:

[0126] value x y z

[0127] Therefore, the presentation state of the $empty file after filling should be as follows:

[0128]

[0129] This completes the most basic data filling process, which is also a data standardization process.

[0130] Step Nine: Remote Procedure Call Document Collection Based on the gRPC Framework. To facilitate the extraction of data files on different computer user systems, gRPC technology is introduced to implement remote procedure call document collection. After the aforementioned eight steps are completed locally by the user, the extraction and re-population process yields a series of XML files. If the data extraction and re-population process is completed remotely, resulting in a series of XML files, these XML files can be uploaded to a designated file database to achieve centralized and unified data management.

[0131] This embodiment has the following technical effects:

[0132] 1. This invention uses a standardized approach to extract, integrate, and re-populate key key-value data from material data documents. It extracts key key-value material data from several unstructured documents (such as doc or docx files) with consistent templates, performs data standardization based on XSD descriptions, and finally completes the data re-population process using an XML data template file derived from the XSD descriptions. This realizes the flow of data from unstructured documents to structured or semi-structured documents. It integrates technologies such as document format conversion, personalized extraction location configuration, XSD-defined data templates, regular expression matching and sentiment analysis, and gRPC (Google Remote Procedure Call framework). For a batch of doc or docx files with consistent templates, a standardized process completes the data flow, extracting intuitive but inconveniently standardized or usable data from the doc or docx files and backfilling it into an XML file template defined by XSD standardization, thereby enabling better description and use of the data.

[0133] 2. The data extraction process described in this invention is a unified operation for doc or docx files, and has uniformity and standardization.

[0134] 3. This invention grants users the right to personalize data extraction configurations, thereby making data extraction more targeted and purposeful.

[0135] 4. The data cleaning and verification of this invention further ensures the quality of the data, and the standardized data template defined by XSD is used to store and extract the data, so as to fully explore the practical application value of the data.

[0136] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0137] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.< / w:t> < / w:r> < / w:p> < / w:tc> < / w:tr> < / w:tbl> < / w:tbl> < / w:tbl>

Claims

1. A method for unstructured document data extraction, characterized by, include: Get the document to be processed; The document to be processed is an unstructured document containing tables; The document to be processed is converted to obtain a first document in XML format; Information is extracted from the first document to obtain the content stored in a table; The positions of the keys and values ​​in the table, along with the extracted table content, are stored to obtain a second document in JSON format. Save the second document as a third document in CSV or Excel format; The third-party documents undergo privacy information desensitization and data cleaning processes.

2. The unstructured document data extraction method of claim 1, wherein, After storing the second document as a third document in CSV or Excel format, the method further includes: The third document is processed using the λ function.

3. The method for extracting unstructured document data according to claim 1, characterized in that, The step of extracting information from the first document to obtain the table storage content specifically includes: The XPath language is used to extract information from the first document to obtain the content stored in a table.

4. The method for extracting unstructured document data according to claim 1, characterized in that, The step of storing the second document as a third document in CSV or Excel format specifically includes: The extracted table content is stored according to the positions of the keys and values ​​in the table to obtain a third document in CSV or Excel format.

5. A system for extracting unstructured document data, characterized in that, include: The acquisition module is used to acquire documents to be processed. The document to be processed is an unstructured document containing tables; The format conversion module is used to convert the document to be processed into a first document in XML format. The extraction module is used to extract information from the first document to obtain the content stored in a table. The first storage module is used to store the positions of the keys in the table, the positions of the values ​​in the table, and the extracted table content to obtain a second document in JSON format; The second storage module is used to store the second document as a third document in CSV or Excel format; and to perform privacy information desensitization and data cleaning on the third document.

6. The unstructured document data extraction system according to claim 5, characterized in that, Also includes: The data cleaning module is used to process the third document using the λ function.

7. The unstructured document data extraction system according to claim 5, characterized in that, The extraction module specifically includes: The extraction unit is used to extract information from the first document using the XPath language to obtain the content stored in a table.

8. The unstructured document data extraction system according to claim 5, characterized in that, The second storage module specifically includes: The second storage unit is used to store the extracted table content according to the position of the key and the position of the value in the table to obtain a third document in CSV or Excel format.

9. A method for filling unstructured document data, characterized in that, The method for extracting unstructured document data applied to any one of claims 1-4, wherein the filling method comprises: An empty data template in XML format is obtained based on the XSD specification; The content stored in the third document obtained by the unstructured document data extraction method according to any one of claims 1-4 is stored in the empty data template to obtain a filled file.

10. A system for filling unstructured document data, characterized in that, include: The template building module is used to obtain empty data templates in XML format based on the XSD specification; A filling module is used to store the content stored in the third document obtained by the unstructured document data extraction method according to any one of claims 1-4 into the empty data template to obtain a filled file.

Images