Method for extracting data from a document

Abstract

The application provides a method for extracting data in literature, which combines a literature segmentation tool with a large language model, establishes an end-to-end material literature data extraction workflow that can be quickly modified, solves the confusion and mismatch problem of different parameters associated with the same material, and independently performs text extraction and chart extraction in steps, and then performs matching and merging, effectively reducing the accuracy decline and video memory requirement of the large language model when processing long sentence sequences, and greatly reducing the time of model modification and data preprocessing annotation of users when performing various literature extraction, improving the efficiency of literature extraction and analysis.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a method for extracting data in literature. BACKGROUND

[0002] The literature data in the field of materials is an important reference for research and development such as material design and process optimization, but specific numerical, structural, and curve data are often scattered in literature, usually stored in unstructured formats such as pdf and xml, and various parameter data need to be extracted and organized into structured table formats such as xlsx and csv before batch analysis and operation of the data can be performed.

[0003] Manual reading of literature for information extraction has high accuracy and recall rate, but the cost of professional knowledge training and time consumption is high. Information extraction (IE) is a process of automatically extracting key information from a large amount of literature resources by relying on advanced technologies such as natural language processing, machine learning, and deep learning. This field has become increasingly important in recent years with the rapid development of artificial intelligence technology. Early information extraction mainly relied on rule matching, part-of-speech tagging, and other methods based on pre-defined rules and manual annotation to identify information in specific formats or patterns, but it was difficult to deal with complex and variable natural language texts.

[0004] Information extraction relying on artificial intelligence technologies such as convolutional neural networks and recurrent neural networks has improved the model's ability to capture and accuracy of key information. In recent years, large language models (LLM) developed by combining neural networks and attention mechanisms have the ability to perform unsupervised learning on large-scale texts and then fine-tune them for specific tasks, improving information extraction performance. They have shown superiority in understanding context and semantic representation and have been widely reported in academic circles.

[0005] However, there is still a lack of complete end-to-end information extraction tools driven by LLM. The main reasons include: existing work reports focus on the accuracy of IE tools on well-organized text datasets, ignoring the process of extracting from document formats to text sets; existing work reports rely on commercial non-open source LLMs (such as GPT4) or specialized LLMs trained for specific fields, which cannot be quickly migrated based on changing field information needs; existing work reports lack the ability to extract text and chart associations. SUMMARY

[0006] The technical problem to be solved by the present application is to provide a method for extracting data in literature, which can establish an end-to-end material literature data extraction workflow that can be quickly modified, solve the confusion and mismatch problem of different parameters associated with the same material, and improve the efficiency of literature extraction and analysis.

[0007] To solve the above problems, the present application provides a method for extracting data in literature, comprising: using a literature segmentation tool to segment the text and the chart of the literature to be extracted according to a preset segmentation mode, and extracting the chart to form a text structured table including the segmented text and literature information and a chart structured table including the chart or chart storage path, the text information of the chart and the literature information; using a large language model and a preset standard text prompt word to confirm whether the text structured table contains entity materials, if yes, according to the parameters in the material data template, using the large language model and the preset standard text prompt word to extract data related to the parameters from the text structured table to form a first text data set, the same group of data in the first text data set includes all extraction results related to the same parameter and the literature information corresponding to the extraction results, and the parameters of different groups of data in the first text data set are different; merging all data corresponding to different parameters of the same material name from the same literature in the first text data set to form a second text data set, the same group of data in the second text data set includes all data of different parameters of the same material from the same literature, and the materials or source literature of different groups of data in the second text data set are different; using a large language model and a preset standard chart prompt word to match the text information of the chart in the chart structured table with the data in the second text data set, and forming a final literature data set according to the matching result, the same group of data in the literature data set includes all text data and charts corresponding to the same material from the same literature.

[0008] In an embodiment, the step of using a literature segmentation tool to segment the text and the chart of the literature to be extracted according to a preset segmentation mode comprises: using a literature segmentation tool to segment the literature to be extracted into primary text and multiple independent charts according to a preset segmentation mode; using a literature segmentation tool to further segment the primary text into required text according to a preset segmentation mode; the required text is the segmented text in the text structured table, and the independent chart is the chart in the chart structured table.

[0009] In an embodiment, in the step of using a literature segmentation tool to segment the literature to be extracted into primary text according to a preset segmentation mode, the preset segmentation mode corresponds to the format of the literature to be extracted.

[0010] In an embodiment, in the step of using a literature segmentation tool to further segment the primary text into required text according to a preset segmentation mode, the preset segmentation mode is one of single sentence, multiple sentences, paragraph or chapter.

[0011] In an embodiment, the literature information includes a literature title.

[0012] In an embodiment, the textual information of the chart includes a title of the chart and characters included in the chart.

[0013] In an embodiment, the preset standard text prompt word includes: judging whether the segmented text exists an entity material; judging whether the segmented text exists a parameter in the material data template; judging whether the segmented text exists a value of the parameter; and judging whether the segmented text exists a unit of the parameter.

[0014] In an embodiment, the material data template includes a plurality of parameters of the material to be extracted, and the step of using the large language model and the preset standard text prompt word to extract data related to the parameters from the text structured table includes: using the large language model and the preset standard text prompt word to extract data related to each parameter from the text structured table respectively, and taking all extraction results related to the same parameter and the source literature of the extraction results as a group of data.

[0015] In an embodiment, the extraction result includes a material name, a value of the parameter, and a unit of the parameter.

[0016] In an embodiment, the preset standard chart prompt word includes: judging whether the textual information of the chart describes a material name in the second text data set.

[0017] In an embodiment, after the steps of forming the text structured table including the segmented text and the literature information, and forming the chart structured table including the chart or the chart storage path, the textual information of the chart, and the literature information, the method further includes the following steps: restoring special format content in the text structured table and the chart structured table, and replacing original content in the text structured table and the chart structured table.

[0018] The method for extracting data in literature provided by the embodiment of the present application combines the literature segmentation tool with the large language model, establishes an end-to-end material literature data extraction workflow that can be quickly modified and run, solves the confusion and mismatch problem of different parameters associated with the same material, and effectively reduces the accuracy decline and video memory requirement of the large language model when processing long sentence sequences by first performing text extraction and chart extraction independently and then performing matching and merging. Moreover, the time for model modification and data preprocessing annotation by the user during various literature extraction is greatly reduced, and the literature extraction and analysis efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description only show some embodiments of the present application, and for those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0020] Figure 1 is a schematic diagram of steps of the method for extracting data in a document provided by an embodiment of the present application;

[0021] Figure 2 is a text structured table and a chart structured table formed according to the document to be extracted, wherein the (a) part is the text structured table, and the (b) part is the chart structured table;

[0022] Figure 3 is a schematic diagram of a material data template of a polishing type material;

[0023] Figure 4 is a schematic diagram of a first text data set formed by the method for extracting data in a document provided by an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of a second text data set formed by the method for extracting data in a document provided by an embodiment of the present application;

[0025] Figure 6 is a schematic diagram of a final document data set formed by the method for extracting data in a document provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] The specific embodiments of the method for extracting data in a document provided by the present application will be described in detail below with reference to the drawings.

[0027] Figure 1 is a schematic diagram of steps of the method for extracting data in a document provided by an embodiment of the present application, please refer to Figure 1 , the method for extracting data in a document comprises the following steps:

[0028] In step S10, a document segmentation tool is used to segment the text and charts of the document to be extracted according to a preset segmentation mode, and the charts are extracted to form a text structured table including the segmented text and document information, and a chart structured table including the charts or chart storage paths, text information of the charts and the document information. This step is used to form the text structured table and the chart structured table from the text and charts of the document to be extracted respectively.

[0029] In some embodiments, before step S10 is performed, the original file of the literature to be extracted is stored in the folder of the specified path. The original file of the literature to be extracted can be a file in pdf format or a document in word, txt or other document format.

[0030] In some embodiments, the literature segmentation tool adopts a text segmentation tool combined with a graphic optical recognition tool, a multi-model integration tool, to realize the separate extraction and arrangement of text and charts. This type of tool includes but is not limited to scipdfparser, grobid, openparse, llama parser, unstructured, etc. programs, toolkits or software.

[0031] In some embodiments, the chart includes pictures and tables, the literature information includes the title of the literature, the chart structured table includes the chart or the storage path of the chart, and the text information of the chart includes the title of the chart and the characters included in the chart.

[0032] In some embodiments, the step of using the literature segmentation tool to segment the text and the chart of the literature to be extracted according to the preset segmentation mode includes:

[0033] The literature to be extracted is segmented into primary texts and multiple independent charts using the literature segmentation tool according to the preset segmentation mode. In this step, the preset segmentation mode corresponds to the format of the literature to be extracted. The independent chart is the chart in the chart structured table in step S10. For example, the format of the literature to be extracted includes title, abstract, main article section 1, main article section 2, reference, thanks, citation, Figure 1 、 Figure 2 , table 1, table 2, etc. In this step, the literature to be extracted is segmented according to the segmentation mode of title, abstract, main article section 1, main article section 2, reference, thanks, citation, Figure 1 、 Figure 2 , table 1, table 2, etc. Each of the text parts of title, abstract, main article section 1, main article section 2, reference, thanks, citation, etc. is a primary text, Figure 1 、 Figure 2 , and each of the chart parts of table 1, table 2, etc. is a chart.

[0034] The primary text is further segmented into required texts using a document segmentation tool according to a preset segmentation mode. In some embodiments, the preset segmentation mode is one of single sentence, multiple sentences, paragraphs, or chapters. The required texts are the segmented texts in the text structured table in step S10. In this step, the primary text is further segmented into smaller texts, and when the length of a single text is less than a predetermined value, the text is combined with the next segmented text, which is the segmented text in the text structured table in step S10, to further improve the accuracy of data extraction. For example, the primary text is segmented into required texts using a document segmentation tool according to a three-sentence mode.

[0035]

[0036] represents the i+1th primary text obtained only by the segmentation tool in each document, T i represents the i th final segmented text obtained by splicing and cutting the primary text in each document, T i+1 represents the i+1th final segmented text obtained by splicing and cutting the primary text in each document, len(T i represents the string length of the i th final segmented text, l tool represents the length threshold value that requires the next segmented text to be combined (changed according to the text segmentation tool and the number of documents, artificially predetermined).

[0037] In some embodiments, the text structured table includes the segmented text, the paragraph where the text is located, and document information; and the chart structured table includes a chart or a chart storage path, a title of the chart, characters in the chart, and the document information.

[0038] The method for extracting data in documents provided by the embodiments of the present application can segment the text according to the required length, thereby avoiding the problems of poor user experience and insufficient flexibility caused by fixed segmentation length, and the accuracy rate decline and memory requirement caused by the large language model when processing long sentence sequences in subsequent steps can also be effectively avoided. At the same time, the method for extracting data in documents provided by the embodiments of the present application uses a document segmentation tool to segment the documents, reduces the time for users to perform document segmentation, picture export, and other preprocessing, and greatly improves the efficiency of document extraction.

[0039] For example, Figure 2 is a text structured table and a chart structured table formed according to the document to be extracted, wherein the (a) part is the text structured table, and the (b) part is the chart structured table. Please refer to Figure 2The text structured table includes multiple groups of data, each group of data including to-be-extracted text (i.e., segmented text), a paragraph where the to-be-extracted text is located, and a document title. The chart structured table includes multiple groups of data, each group of data including a chart storage path, a chart title, and a document title. Figure 2 Text and pictures from two documents are listed illustratively.

[0040] Please continue to refer to Figure 1 At step S11, it is determined whether the text structured table contains entity materials using a large language model and preset standard text prompt words. If yes, data related to parameters in the material data template is extracted from the text structured table using the large language model and the preset standard text prompt words according to the parameters in the material data template, to form a first text data set. The same group of data in the first text data set includes all extraction results related to the same parameter and document information corresponding to the extraction results, and different groups of data in the first text data set have different parameters. This step is used to extract segmented text in the text structured table and form a first text data set.

[0041] The large language model (LLM) refers to a deep learning model trained using a large amount of text data, which can generate natural language text or understand the meaning of language text. The large language model can process various natural language tasks such as text classification, question answering, and dialogue, and is an important way to artificial intelligence. The large language model includes but is not limited to llama3, llama2, qwen1.5, Yi34, and other large models and large model architectures.

[0042] The standard text prompt word is a series of questions in the large language model, including questions about the text. In the method for extracting data in the document provided in an embodiment of the present application, the standard text prompt word can be set in advance according to the type of the document, the type of the material to be extracted, and the user's needs to meet individual needs. In some embodiments, the preset standard text prompt word includes: determining whether the segmented text contains entity materials; determining whether the segmented text contains parameters in the material data template; determining whether the segmented text contains a value of the parameter; and determining whether the segmented text contains a unit of the parameter.

[0043] The material data template refers to a data list containing various parameters of a certain type or certain types of materials. For example, if a user needs to obtain data of polishing materials, the material data template is a data list of various parameters of polishing materials, for example, the material data template is a set containing parameters such as viscosity and diffusion coefficient. In some embodiments, the material data template is designed by a domain engineer according to experience and knowledge in advance, and covers as many potential parameters as possible.

[0044] In some embodiments, the material data template includes multiple parameters of the material to be extracted. The step of extracting data related to the parameters from the text structured table using a large language model and preset standard text prompts includes: extracting data related to each parameter from the text structured table using a large language model and preset standard text prompts, and taking all extraction results related to the same parameter and the source documents of the extraction results as a set of data, and multiple sets of data constitute the first text dataset.

[0045] In some embodiments, the extraction results include the material name, the parameter value, and the unit of the parameter. In other embodiments, the preset standard text prompts can be set according to the user's needs to obtain different extraction results and meet personalized requirements.

[0046] In some embodiments, this step uses a large language model and preset standard text prompts for confirmation. Figure 2 Whether the text to be extracted from the structured text table shown contains entity material can be confirmed, for example, using a large language model and a preset standard text prompt "Does the original text in the following sentences contain entity material?" Figure 2 If the text to be extracted from the structured text table shown contains physical materials, then if so, the material names of all physical materials in the text are output and used as one of the extraction results, thus determining the material names; upon confirmation... Figure 2 After extracting entity material from the text structured table shown, the large language model and preset standard text prompts are used: "Does the original text in the following sentences contain parameter A?", "Does the original text in the following sentences contain a numerical value of parameter A?", and "Does the original text in the following sentences contain a unit of parameter A?". If so, the large language model continues to provide commands: "Use the original text in the following sentences to output parameter A", "Use the original text in the following sentences to output the numerical value of parameter A", and "Use the original text in the following sentences to output the unit of parameter A", thereby extracting data related to the parameter from the text structured table to form the first text dataset.

[0047] For example, Figure 3 This is a schematic diagram of a material data template for polished materials. Figure 4 This is a schematic diagram of the first text dataset formed by the method for extracting data from documents provided in this embodiment of the invention. Please refer to [link / reference]. Figure 3 and Figure 4 ,exist Figure 3The material data template of the medium polishing material only shows the viscosity and diffusion coefficient. In this step, the large language model and the preset standard text prompt word are used to confirm Figure 2 whether the entity material is contained in the text to be extracted from the text structured table shown, and if so, output the material name of all entity materials in the text to be extracted as one of the extraction results. Whether the viscosity exists in the original text in the following sentence, whether the numerical value of the viscosity exists in the original text in the following sentence, and whether the unit of the viscosity exists in the original text in the following sentence. If so, the large language model continues to provide the command “output the numerical value of the viscosity using the original text in the following sentence” and “output the unit of the viscosity using the original text in the following sentence”. A plurality of parameters are sequentially executed according to the above method, so as to extract data related to the parameters from the text structured table. In Figure 4 , the (a) part is a group of data, and the (b) part is another group of data, and the parameters corresponding to the two groups of data are different, wherein the (a) part is the data of viscosity, and the (b) part is the data of diffusion coefficient.

[0048] Please continue to refer to Figure 1 , in step S12, all data of different parameters corresponding to the same material from the same literature in the first text data set are merged to form a second text data set, and a same group of data of the second text data set includes all data of different parameters corresponding to the same material from the same literature, and materials or source literatures of different groups of data of the second text data set are different. For example, please refer to Figure 5 , Figure 5 is a schematic diagram of the second text data set formed by the method for extracting data in literature provided by the embodiment of the present application, and all data of different parameters corresponding to the same material from the same literature in Figure 4 are merged to form a second text data set. The method for extracting data in literature provided by the embodiment of the present application extracts each parameter respectively and then merges the data, which reduces the information interference and confusion among different parameters, and reduces the limitation of the length of literature segmentation and the consumption of display memory during program running.

[0049] The total display memory of the large language model inference is G 模型 + G KV Cache + G 中间激活值 (2)

[0050] wherein G 模型 is the display memory required for loading the large language model, and is related to the type of the selected large language model, G KVCache is the display memory required for operation during the inference process of the large language model, and is related to the length of the input text;

[0051] G 中间激活值The GPU memory required for storing the intermediate activation values in the large model inference process.

[0052] G KV Cache = B * (2 * l * h * d * (s + n) * sizeof(dtype)) (3)

[0053] Where s is the total text length of the input large language model, n is the text length of the large language model data output, B, l, h, d are fixed parameters related to the selection of the large language model, representing batch size, decoder layer number, number of attention heads per layer, and single head dimension, respectively, and sizeof(dtype) is the number of bytes of the storage data type.

[0054] s = s prompt + s 分割文本 (4)

[0055] Where s prompt is the total length of the prompt word, which is a constant value in the large language model running; s 分割文本 is the length of the segmented text, it can be seen that the embodiment of the application uses the method of extracting each parameter respectively to reduce s 分割文本 , thereby reducing G (KV Cache) , and finally reducing the total GPU memory occupied by the large language model inference, and reducing the running device requirements.

[0056] Step S13, using the large language model and the preset standard chart prompt word to match the text information of the chart in the chart structured table with the data in the second text data set, and forming a final literature data set according to the matching result, wherein the same group of data in the literature data set includes all text data and charts corresponding to the same material derived from the same literature. This step is used to combine the content of the chart structured table with the data in the second text data set to form the final literature data set. The literature data set can be cleaned, labeled and analyzed as needed. The literature data set can be an excel or csv format file, wherein the chart can be stored in the form of embedded table or in the form of an orderly stored folder.

[0057] The standard chart prompt word is a series of questions in a large language model, including questions about charts. In the method for extracting data in a literature provided in an embodiment of the present application, the standard chart prompt word can be set in advance according to the type of literature, the type of material to be extracted, and the needs of the user to meet individual needs. In some embodiments, the preset standard chart prompt word includes: judging whether the text information of the chart describes the material name in the second text data set. If the text information of the chart describes the material name in the second text data set, the chart and the second text data set derived from the same literature are combined to form a final literature data set.

[0058] For example, Figure 6 is a schematic diagram of the final literature data set formed by the method for extracting data in a literature provided in an embodiment of the present application. Please refer to Figure 6 , using a large language model and a preset standard chart prompt word to match the text information of the chart in the chart structure in Figure 1 with the material name in the second text data set in Figure 5 , and forming a final literature data set according to the matching result, the same group of data in the literature data set includes all text data and charts corresponding to the same material derived from the same literature.

[0059] The method for extracting data in a literature provided in an embodiment of the present application extracts the text and chart of the literature to be extracted respectively and then matches and combines them, effectively solving the problem of accuracy decline and video memory requirement of a large language model when processing long sentence sequences, and avoiding the precision decline caused by image recognition and labeling.

[0060] The method for extracting data in a literature provided in an embodiment of the present application reduces the time of users to perform literature segmentation, picture export and other pre-processing, reduces the time of users to design prompt words for material literature, reduces the time of manual intervention model modification and data pre-processing, and overall improves the efficiency of literature extraction and analysis. Moreover, the method for extracting data in a literature provided in an embodiment of the present application can extract material data parameters according to user needs, directly generate a structured data table from the original format of the literature, and improve the generality and migration ability of information extraction.

[0061] The method for extracting data in a literature provided in an embodiment of the present application establishes an end-to-end material literature data extraction workflow that can be quickly run and modified, solves the confusion and mismatch problem of different parameters associated with the same material, matches picture information with parameter information, so that the time of users for model modification and data pre-processing labeling is greatly reduced when extracting various literatures, and the efficiency of literature extraction and analysis is improved.

[0062] In some embodiments, the special formats such as material structural formula, formula, scientific notation, etc. will be distorted during extraction and cannot maintain the original format, then the extraction method of the present application also processes the special formats such as material structural formula, formula, scientific notation, etc. after segmentation to make the special formats in the text in the text structured table maintain the original format. Specifically, after the steps of forming the text structured table including the segmented text and the literature information and the chart structured table including the chart or the chart storage path, the text information of the chart and the literature information, the following steps are further included: restoring the special format content in the text structured table and the chart structured table and replacing the original content in the text structured table and the chart structured table. The restoration of the special format content in the text structured table and the chart structured table can be performed by existing methods, such as by relying on regular expressions, verification functions, etc. The special format processing tool is modified by comparing the literature extraction accuracy and recall rate indexes before and after the introduction of the processing tool.

[0063] P 准确率 = TP / (TP+FP) (5)

[0064] R 召回率 = TP / (TP+FN) (6)

[0065] wherein P 准确率 is the literature extraction accuracy, R 召回率 is the recall rate of literature extraction, the accuracy and the recall rate are two indexes for evaluating the literature extraction effect, TP is the number of parameters correctly extracted by the large language model, FP is the number of parameters extracted by the large language model but inconsistent with the original text, and FN is the number of parameters that should be extracted but not extracted by the large language model.

[0066] It should be noted that the terms "comprise" and "have" and their conjugates, as used in the specification and claims of the present application, are intended to encompass the presence of stated features, structures, components, elements, or objects, but do not preclude the presence or addition of one or more other features, structures, components, elements, or objects. The terms "first," "second," and the like, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another, unless otherwise indicated by context. The use of the terms "a" and "an" and "the" and "said" and "at least one" and "one or more" to refer to an element or object also is taken to cover "one or more" unless otherwise indicated by context. The term "based on" can be taken to mean "based at least in part on," that is, not necessarily based exclusively on the matter following the term. In addition, the embodiments and features discussed above can be combined with each other, as much as is physically possible. Furthermore, in the following description of various embodiments, numerous specific details are set forth in order to provide a thorough understanding of the embodiments. However, various embodiments can be practiced without these specific details. In other instances, well-known components have not been described in detail in order to avoid unnecessarily obscuring the embodiments. Unless otherwise noted, the use of the ordinal adjectives (i.e., "first," "second," etc.) to describe a common but distinct member of a generic set is to be understood as indicating a number of the features. The use of the terms "at least one" and "one or more" is to be taken as synonymous with each other.

[0067] The above description is merely of the preferred embodiments of the application and various modifications and improvements can be made by those skilled in the art without departing from the principles of the application. Such modifications and improvements are intended to fall within the scope of the application.

Claims

1. A method for extracting data from literature, characterized in that, include: The document segmentation tool is used to segment the text and charts of the document to be extracted according to a preset segmentation method, and the charts are extracted to form a text structured table including the segmented text and document information, and a chart structured table including the charts or chart storage path, the text information of the charts and the document information. The large language model and preset standard text prompt words are used to confirm whether the text structured table contains entity materials. If so, according to the parameters in the material data template, the large language model and preset standard text prompt words are used to extract data related to the parameters from the text structured table to form a first text dataset. The same set of data in the first text dataset includes all extraction results related to the same parameter and the literature information corresponding to the extraction results. The parameters of different sets of data in the first text dataset are different. The first text dataset is formed by merging all data of different parameters corresponding to the same material name from the same document in the first text dataset to form the second text dataset. The same set of data in the second text dataset includes all data of different parameters of the same material from the same document. The materials or source documents of different sets of data in the second text dataset are different. The text information of the charts in the chart structure table is matched with the data in the second text dataset using a large language model and preset standard chart prompts. The final literature dataset is formed based on the matching results. The same set of data in the literature dataset includes all text data and charts corresponding to the same material from the same document. The material data template includes multiple parameters of the material to be extracted. The steps of extracting data related to the parameters from the text structured table using a large language model and preset standard text prompts include: extracting data related to each parameter from the text structured table using a large language model and preset standard text prompts, and taking all extraction results related to the same parameter and the source literature of the extraction results as a set of data.

2. The method for extracting data from documents according to claim 1, characterized in that, The steps for using a document segmentation tool to segment the text and figures of the document to be extracted according to a preset segmentation method include: The document segmentation tool is used to segment the document to be extracted into primary text and multiple independent charts according to the preset segmentation method. Use a document segmentation tool to further segment the primary text into the required text according to the preset segmentation method; The required text is the segmented text in the text structured table, and the independent chart is the chart in the chart structured table.

3. The method for extracting data from documents according to claim 2, characterized in that, In the step of using a document segmentation tool to segment the document to be extracted into primary text according to a preset segmentation method, the preset segmentation method corresponds to the format of the document to be extracted.

4. The method for extracting data from documents according to claim 2, characterized in that, In the step of using a document segmentation tool to further segment the primary text into the desired text according to a preset segmentation method, the preset segmentation method is one of the following: single sentence, multiple sentences, paragraph, or chapter.

5. The method for extracting data from documents according to claim 1, characterized in that, The document information includes the document title.

6. The method for extracting data from documents according to claim 1, characterized in that, The text information of the chart includes the chart title and the characters included in the chart.

7. The method for extracting data from documents according to claim 1, characterized in that, The preset standard text prompts include: determining whether the segmented text contains physical materials; determining whether the segmented text contains parameters from the material data template; determining whether the segmented text contains the numerical value of the parameter; and determining whether the segmented text contains the unit of the parameter.

8. The method for extracting data from documents according to claim 1, characterized in that, The extraction results include the material name, the numerical value of the parameter, and the unit of the parameter.

9. The method for extracting data from documents according to claim 1, characterized in that, The preset standard chart prompts include: determining whether the text information of the chart describes the material name in the second text dataset.

10. The method for extracting data from documents according to claim 1, characterized in that, After forming a text structured table including segmented text and document information, and a chart structured table including charts or chart storage paths, text information of the charts, and document information, the method further includes the following steps: restoring the special format content in the text structured table and the chart structured table, and replacing the original content in the text structured table and the chart structured table.

Images