Data extraction method and device, electronic equipment and storage medium

Abstract

The invention provides a data extraction method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring a to-be-processed document; determining whether the format of the to-be-processed document is a preset format or not, wherein the preset format is any one of a word document, a PDF document, a PPT document and an excel document. If yes, data extraction schemes are determined according to the format of the to-be-processed document, and one preset format corresponds to one data extraction scheme. And based on the data extraction scheme, extracting valid data from the to-be-processed document. According to the method, a customized data extraction scheme can be adopted for documents in various formats, the documents can be segmented according to the structural characteristics of the documents, and the document segmentation efficiency during data extraction is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data extraction method, device, electronic device and storage medium. Background Art

[0002] With the development of computer technology, enterprises have accumulated a large amount of electronic document resources in the process of promoting digital office. These documents are usually used to record information on internal business, management and operations. By processing electronic documents, effective key information can be extracted, and the use of these key data is conducive to improving the core competitiveness, innovation ability and operational efficiency of enterprises.

[0003] The current electronic document data extraction technology segments the document content according to a certain character length or fixed rules. This processing method limits the segmentation length or rules, and may discard non-text content when processing non-pure text documents, resulting in unsatisfactory segmentation results. In addition, the segmented content will become semantically incoherent due to the lack of context. In view of this, how to improve the segmentation efficiency when extracting data from documents of different formats has become an urgent problem to be solved in the current document data extraction technology. Summary of the invention

[0004] The present application provides a data extraction method, device, electronic device and storage medium. The present application can adopt a customized data extraction solution for documents of various formats, can segment documents according to their structural characteristics, and improve the efficiency of document segmentation during data extraction.

[0005] In a first aspect, the present application provides a data extraction method, the method comprising: obtaining a document to be processed. Determine whether the format of the document to be processed is a preset format, the preset format being any one of the following: word document, PDF document, PPT document, excel document. If so, determine a data extraction scheme based on the format of the document to be processed, one preset format corresponds to one data extraction scheme, the data extraction scheme is used to identify key information in the document to be processed, to segment the document to be processed based on the key information, and to extract valid data from the segmented content, the key information is used to represent the structured features of the document to be processed. Based on the data extraction scheme, extract valid data from the document to be processed.

[0006] The present application provides a data extraction method, which can adopt corresponding data extraction schemes for documents of different formats by identifying the document format. The data extraction scheme can segment the document according to the structural characteristics of the document, and can extract valid data from the segmented content. Through the data extraction scheme, not only can different documents be effectively segmented, but also the limitations of the prior art of document segmentation according to character length or fixed rules are broken, so that the theme or structure of the segmented content is relatively complete, and the efficiency of document segmentation is improved.

[0007] A possible implementation method, when the document to be processed is a PPT document, extracting valid data from the document to be processed based on a data extraction scheme includes:

[0008] Perform layout analysis on the document to be processed and identify key areas, where the key areas include at least one of the following: text area, title area, table area, and picture area.

[0009] The document to be processed is segmented based on the key area to obtain at least one image.

[0010] For each image, valid data is extracted through optical character recognition.

[0011] Another possible implementation manner, when the document to be processed is a word document, extracting valid data from the document to be processed based on a data extraction scheme includes:

[0012] Determines whether the document to be processed contains a directory structure.

[0013] If so, the document to be processed is segmented based on the directory structure, and valid data is extracted from each segmented part.

[0014] If not, the document to be processed is segmented based on semantic analysis to obtain semantically related paragraphs, and valid data is extracted from the semantically related paragraphs.

[0015] Another possible implementation method, when the document to be processed is an Excel document, extracting valid data from the document to be processed based on a data extraction scheme includes:

[0016] Extract the header information and each row of data of the document to be processed.

[0017] Each row of data is split, and the length of each part of the data obtained after the split is less than or equal to the length of the input data of the large language model.

[0018] Configure corresponding header information for each part of the divided content to obtain valid data.

[0019] Another possible implementation manner, when the document to be processed is a PPT document, extracting valid data from the document to be processed based on a data extraction scheme includes:

[0020] Extract the PPT chapter directory in the document to be processed.

[0021] After segmenting the document to be processed based on the PPT chapter directory, valid data is extracted from each segmented part.

[0022] Another possible implementation method, after obtaining valid data, further includes: performing semantic analysis on the valid data, generating question-answer pairs and summaries, and using the question-answer pairs and summaries to train knowledge retrieval and question-answer models.

[0023] Another possible implementation manner, after obtaining the valid data, may further include: converting the format of the valid data into the format of training data, converting the training data into vectors, and storing the vectors in a vector database.

[0024] In a second aspect, the present application provides a data extraction device, which includes: an acquisition module, a determination module, an extraction module and a processing module.

[0025] The acquisition module is used to obtain documents to be processed.

[0026] The determination module is used to determine whether the format of the document to be processed is a preset format.

[0027] The determination module is also used to determine a data extraction plan according to the format of the document to be processed.

[0028] The extraction module is used to extract valid data from the document to be processed based on the data extraction scheme.

[0029] The processing module is used to generate question-answer pairs and summaries based on valid data.

[0030] The processing module is also used to convert the format of the valid data into the format of the training data, convert the training data into vectors, and store them in the vector database.

[0031] In a possible implementation, the extraction module is specifically used to extract valid data from the document to be processed based on the data extraction scheme when the document to be processed is a PDF document, including:

[0032] Perform layout analysis on the document to be processed and identify key areas, where the key areas include at least one of the following: text area, title area, table area, and picture area.

[0033] The document to be processed is segmented based on the key area to obtain at least one image.

[0034] For each image, valid data is extracted through optical character recognition.

[0035] In another possible implementation, the extraction module is specifically used to extract valid data from the document to be processed based on the data extraction scheme when the document to be processed is a word document, including:

[0036] Determines whether the document to be processed contains a directory structure.

[0037] If so, the document to be processed is segmented based on the directory structure, and valid data is extracted from each segmented part.

[0038] If not, the document to be processed is segmented based on semantic analysis to obtain semantically related paragraphs, and valid data is extracted from the semantically related paragraphs.

[0039] In another possible implementation, the extraction module is specifically used to extract valid data from the document to be processed based on the data extraction scheme when the document to be processed is an Excel document, including:

[0040] Extract the header information and each row of data of the document to be processed.

[0041] Each row of data is split, and the length of each part of the data obtained after the split is less than or equal to the length of the input data of the large language model.

[0042] Configure corresponding header information for each part of the divided content to obtain valid data.

[0043] In another possible implementation, the extraction module is specifically used to extract valid data from the document to be processed based on the data extraction scheme when the document to be processed is a PPT document, including:

[0044] Extract the PPT chapter directory in the document to be processed.

[0045] After segmenting the document to be processed based on the PPT chapter directory, valid data is extracted from each segmented part.

[0046] In another possible implementation, the processing module is further used to perform semantic analysis on valid data to generate question-answer pairs and summaries, and the question-answer pairs and summaries are used to train knowledge retrieval and question-answering models.

[0047] In another possible implementation, the processing module is further used to convert the format of the valid data into the format of the training data, convert the training data into vectors, and store them in a vector database.

[0048] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory; the memory stores instructions executable by the processor; when the processor is configured to execute the instructions, the electronic device implements the method of the first aspect above.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium, which includes: computer software instructions; when the computer software instructions are executed in an electronic device, the electronic device implements the method of the first aspect above.

[0050] In a fifth aspect, the present application provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the steps of the related method described in the first aspect above to implement the method of the first aspect above.

[0051] The beneficial effects of the second to fifth aspects mentioned above refer to the corresponding description of the first aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A system architecture diagram of PP-OCRv3 provided for this application;

[0053] Figure 2 A text detection model diagram in a PP-OCRv3 system provided in this application;

[0054] Figure 3 A structural diagram of a PAN with a large receptive field provided for this application;

[0055] Figure 4 A text recognition model diagram in a PP-OCRv3 system provided in this application;

[0056] Figure 5 An attention mechanism to guide CTC training strategy diagram provided for this application;

[0057] Figure 6 A flow chart of a data extraction method provided for this application;

[0058] Figure 7 A schematic diagram of a process for extracting data from a PPT format document provided in this application;

[0059] Figure 8 A schematic diagram of a process for extracting data from a word format document provided in this application;

[0060] Fig. 9 An example diagram of a method for extracting data from a word format document provided in this application;

[0061] Fig.10A schematic diagram of a process for extracting data from Excel format documents provided in this application;

[0062] Fig.11 A schematic diagram of a process for extracting data from a PPT format document provided for this application

[0063] Fig.12 An example diagram of a method for extracting data from a PPT format document provided in this application;

[0064] Fig.13 A flowchart of another data extraction method provided in this application;

[0065] Fig.14 A flowchart of another data extraction method provided by this application;

[0066] Fig.15 A diagram of the architecture of a knowledge generation system combined with a data extraction method provided for this application;

[0067] Fig.16 A flow chart of a knowledge generation system combined with a data extraction method provided for the present application;

[0068] Fig.17 A data extraction method and a large language model interaction diagram provided for this application;

[0069] Fig.18 A schematic diagram of the composition of a data extraction device provided in this application;

[0070] Fig.19 A schematic diagram of the composition of an electronic device provided in this application. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0072] It should be noted that, in the embodiments of the present application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0073] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical items or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order.

[0074] With the continuous advancement of information technology, enterprises and organizations have gradually accumulated a large amount of electronic document resources in the process of promoting digital transformation. These documents record various business, management and operational information within the enterprise or organization, and have become an indispensable part of enterprise operations. Therefore, making full use of these document resources can, to a certain extent, optimize enterprise management, improve operational decision-making efficiency, and help the development of enterprises.

[0075] As for how to make full use of document resources, it involves electronic document data extraction technology. Electronic document data extraction technology includes content segmentation and information extraction. The content segmentation part is responsible for segmenting the document according to a specific length or fixed rule, while the information extraction part is responsible for extracting valid data from the segmented content.

[0076] However, existing document segmentation methods are limited by fixed lengths or rules, which may cause the segmented content to be disconnected in context, resulting in semantic or structural incompleteness. In addition, when processing non-pure text documents, existing methods may also ignore non-text information in the document (such as pictures, tables, etc.), resulting in reduced document segmentation efficiency.

[0077] The present application provides a data extraction method, which can adopt corresponding data extraction schemes for documents of different formats by identifying the document format. The data extraction scheme can segment the document according to the structural characteristics of the document, and can extract valid data from the segmented content. Through the data extraction scheme, not only can different documents be effectively segmented, but also the limitations of the prior art of document segmentation according to character length or fixed rules are broken, so that the theme or structure of the segmented content is relatively complete, and the efficiency of document segmentation is improved.

[0078] A data extraction method provided in the present application can be applied to a server. The server can be a server cluster composed of multiple servers, or a single server, or a computer, or a processor or processing chip in a server or computer. The embodiments of the present application do not limit the specific device form of the computing device.

[0079] In order to better understand the solution of the present application, some terms or related technologies in the present application are now explained.

[0080] Office Open XML: OOXML for short, is an electronic document format based on extensible markup language (XML) and ZIP (ziv-lempel information packet, ZIP) compression technology. It is designed for the creation, editing, management and sharing of electronic documents, such as word processing documents, spreadsheets and presentations. As an open standard, OOXML was jointly approved by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) as the ISO / IEC 29500 standard in 2008.

[0081] Path augmentation network (PAN): PAN is a network structure that improves feature extraction and transfer capabilities through a specific path augmentation strategy. In PP-OCRv3, the PAN structure is further optimized to have a larger receptive field, thereby improving the model's ability to detect large fonts and text with extreme aspect ratios.

[0082] Feature pyramid network (FPN): A multi-scale feature fusion network structure that fuses feature maps of different scales through bottom-up and top-down paths, while introducing a residual attention mechanism to enhance the representation ability of feature maps and further improve the performance of the model in text detection tasks.

[0083] Connectionist temporal classification (CTC): CTC is a machine learning algorithm for processing sequence data, especially for situations where the length of the output sequence is inconsistent with the length of the input sequence, such as speech recognition and text recognition. It can directly learn the mapping relationship from the input sequence to the output sequence without aligning the sequences. In the text recognition module of PP-OCRv3, CTC is an important component. It is combined with other strategies (such as attention-guided training, data augmentation, etc.) to improve the accuracy and efficiency of text recognition.

[0084] PaddlePaddle Optical Character Recognition Version 3 (PP-OCRv3) system can extract key information from images. It is a two-stage OCR system, such as Figure 1 The figure shows a system architecture diagram of PP-OCRv3, which includes a text detection algorithm, a text recognition algorithm, and a detection frame corrector responsible for identifying the text direction. The text detection algorithm uses the differentiable binarization (DB) algorithm, and the text recognition algorithm uses the scene text recognition method based on a single visual model (SVTR).

[0085] Compared with PP-OCRv2, PP-OCRv3 has made nine improvements in the text detection module and text recognition module:

[0086] Text Detection Module:

[0087] 1. PAN structure with large receptive field (large kernel PAN, LK-PAN).

[0088] 2. Teacher model mutual learning strategy (deep mutual learning, DML).

[0089] 3. FPN structure of residual attention mechanism (residual and squeeze-and-excitation enhanced feature pyramid network, RSE-FPN).

[0090] Text recognition module:

[0091] 1. Lightweight text recognition network (SVTR lightweight convolutional network, SVTR_LCNet).

[0092] 2. Attention-guided CTC training strategy (guided training with CTC, GTC).

[0093] 3. Data augmentation strategy for mining text context information (text context augmentation, TextConAug).

[0094] 4. Self-supervised pre-trained model (text rotation network, TextRotNet).

[0095] 5. Unified deep mutual learning (UDML) strategy.

[0096] 6. Unlabeled image mining (UIM) solution.

[0097] In the text detection module, the PP-OCRv3 detection model is an upgrade of the collaborative mutual learning (CML) text detection distillation strategy in PP-OCRv2. Figure 2 As shown, a PP-OCRv3 text detection model provided by this application. The core idea of ​​CML combines the traditional standard distillation of teachers guiding students and DML mutual learning between student networks. The teacher network can provide guidance while the student networks learn from each other. PP-OCRv3 further optimizes the effects of the teacher model and the student model respectively. Among them, when optimizing the teacher model, LK-PAN is proposed and the DML strategy is introduced, and when optimizing the student model, the RSE-FPN structure is proposed.

[0098] like Figure 2 As shown, the input module receives image data and transmits it to student model 1 and student model 2, which output responses respectively. Figure 1 ,response Figure 2 ,The response graph represents the preliminary detection results of the image data.

[0099] Secondly, the response Figure 1 and response Figure 2 Enter the relative entropy module, which can evaluate the text detection performance of the student model. Figure 1 and response Figure 2 By comparing the true value labels (i.e. the annotated data corresponding to the image data), the true value difference loss is obtained. The student model can gradually improve the accuracy of text detection under supervised learning by reducing the true value difference loss.

[0100] In addition, the response Figure 1 and response Figure 2 It is also necessary to refer to the distillation loss and compare it with the teacher model. The distillation loss is used to measure the difference between the student model and the teacher model. During the distillation process, minimizing the distillation loss allows the student model to learn better features of the teacher model, allowing it to maintain a relatively streamlined structure while continuously approaching the high accuracy of the teacher model.

[0101] Finally, the post-processing module receives the response Figure 1 and response Figure 2 , refine the text area, correct the overlapping or misjudged areas, integrate the scattered areas, and output more accurate and reliable detection results.

[0102] like Figure 3As shown in the figure, a PAN structure with a large receptive field provided by the present application is provided. The core is to change the convolution kernel in the path augmentation of the PAN structure from 3*3 to 9*9. By increasing the convolution kernel, the receptive field covered by each position in the feature map is improved, making it easier to detect large font text and text with extreme aspect ratios.

[0103] In the text recognition module, PP-OCRv3 is optimized based on the text recognition algorithm SVTR. SVTR no longer uses the recurrent neural network (RNN) structure, but introduces a deep learning model structure based on the attention mechanism to more effectively mine the contextual information of text line images, thereby improving text recognition capabilities.

[0104] like Figure 4 As shown, a text recognition model in a PP-OCRv3 system provided by this application is shown.

[0105] in:

[0106] S401 and S405 are GTC, including attention mechanism and attention loss; the attention mechanism is used to capture the global context information in the image to help the model better understand the text content; the attention loss is used to measure the accuracy of the model when generating attention weights, thereby guiding the model to learn a more effective attention mechanism.

[0107] S402 and S404 are SVTR_LCNet, including PP-LCNet, SVTR generator (SVTR generation, SVTR_g), fully connected layer (fully connected layer, FC), CTC loss. PP-LCNet, as a feature extractor, converts the features of the input image into a series of vectors; SVTR_g is responsible for generating text recognition sequences based on the extracted features and attention weights; FC is used to convert the output of the generator into the final text recognition result. CTC Loss allows the length of the output sequence to be different from the length of the input sequence and does not require pre-alignment of the data. It solves the alignment problem by summing all possible alignments to calculate the loss.

[0108] S403 is UDML, which is a very effective strategy for improving the model effect for text recognition adopted in PP-OCRv2. In PP-OCRv3, for two different SVTR_LCNet and GTC structures, the feature map of PP-LCNet, the output of the SVTR module and the output of the attention module between them are supervised and trained simultaneously.

[0109] S406 is UIM, which is a very simple unlabeled data mining solution. The core idea is to use a high-precision text recognition large model to predict unlabeled data, obtain pseudo labels, and select samples with high prediction confidence as training data for training small models.

[0110] S407 is TextRotNet, which is a pre-trained model trained in a self-supervised manner using a large amount of unlabeled text line data. This model can initialize the initial weights of SVTR_LCNet, thereby helping the text recognition model converge to a better position.

[0111] S408 is TextConAug, which is a data augmentation strategy for mining text context information. It can enrich the context information of training data and improve the diversity of training data.

[0112] like Figure 5 As shown, a CTC training strategy diagram guided by an attention mechanism provided in this application, first, the system extracts features of the input image through PP-LCNet, and then further processes it through the SVTR_G module. The processed features are then sent to the attention module, which weights the features to emphasize the parts that are more important to the task. Subsequently, the output of the attention module is passed to the fully connected layer (FC) module, which maps the features to the output space. Finally, the entire process evaluates the model performance by calculating the CTC Loss, which is used to handle sequence-to-sequence mapping problems and considers all possible character alignments. The entire flowchart uses dotted lines to represent the connections and data flows between the modules, where the dotted lines in the figure are the gradient propagation directions and the solid lines are the calculation directions. The attention module is discarded during the inference phase.

[0113] Figure 6 A flow chart of a data extraction method provided in this application. Figure 6 As shown, a data extraction method provided by the present application can be implemented by the above computing device, and specifically includes the following steps:

[0114] S601: Obtain documents to be processed.

[0115] A processed document is an electronic document that can be created, edited, and updated in a computer system.

[0116] In some embodiments, after receiving the upload instruction from the user, the document to be processed can be obtained by accessing a local file resource manager, or the document to be processed can be obtained through a file path input by the user.

[0117] S602: Determine whether the format of the document to be processed is a preset format.

[0118] The default format is any of the following: word document, PDF document, PPT document, excel document.

[0119] It should be understood that by determining the format of the document to be processed, it is possible to distinguish whether the format of the document is one of the preset formats, so that a data extraction solution corresponding to the format can be matched subsequently.

[0120] In some embodiments, the format of the document to be processed can be obtained by querying the file type attribute of the document to be processed, or the format of the document to be processed can be determined by determining the file name suffix of the document to be processed.

[0121] S603: If yes, determine a data extraction plan based on the format of the document to be processed.

[0122] One preset format corresponds to one data extraction scheme. The data extraction scheme is designed based on the structural characteristics of the document in the preset format. If the format of the document to be processed belongs to one of the preset formats, the data extraction scheme of the document to be processed can be determined.

[0123] Specifically, the data extraction scheme is used to identify key information in the document to be processed, to segment the document to be processed based on the key information, and to extract valid data from the segmented content. The key information is used to represent the structural features of the document to be processed.

[0124] In some embodiments, the structural features represented by the key information may be the directory structure for word and PPT documents, the association structure between the table header and each row of data for an excel document, and the page layout for a PDF document.

[0125] It should be understood that the data extraction scheme can identify the key information of the document to be processed, and based on the key information, the subject or structure of each part of the document can be distinguished for segmentation, so that the segmented content can have a certain semantic coherence or structural integrity, which can improve the accuracy of document segmentation.

[0126] In some embodiments, the data extraction solution for PDF format documents can refer to Figure 7 , you can refer to the data extraction solution for word documents Figure 8 For data extraction solutions for Excel documents, please refer to Fig.10 For data extraction solutions for PPT documents, please refer to Fig.11 In other embodiments, if the document to be processed is not in a preset format, the document to be processed may be converted into a document in a preset format through format conversion.

[0127] Exemplarily, if the format of the document to be processed is a txt document, it can be converted into a PDF document, and then a corresponding data extraction solution is adopted.

[0128] S604: Extract valid data from the document to be processed based on the data extraction solution.

[0129] In some embodiments, the data extraction scheme is used to identify key information in the document to be processed, to segment the document to be processed based on the key information, and to extract valid data from the segmented content, wherein the key information is used to represent the structural features of the document to be processed.

[0130] In some embodiments, when extracting valid data from the segmented content, the semantic understanding capability of the large language model may be used to perform semantic understanding on the segmented content to extract valid data.

[0131] For example, when extracting data from the segmented content, valid data can be extracted from the segmented content through a variety of information recognition methods, such as recognizing image information through optical character recognition, recognizing table information through a table recognition model, and recognizing text parts through a large language model. After recognition, they are integrated into consistent knowledge, which can make the final generated valid data more complete and coherent.

[0132] Exemplarily, valid data may include date, time, and any text content that can be quantified and used for analysis.

[0133] The data extraction scheme for documents in different formats is described below in conjunction with specific embodiments and drawings of the specification.

[0134] In some embodiments, when the document to be processed is a PDF document, valid data is extracted from the document to be processed based on a data extraction scheme. Figure 7 The figure shows a flow chart of a PDF document data extraction process provided by the present application, and the specific method includes:

[0135] S604a1. Analyze the layout of the file to be processed and identify key areas.

[0136] The key area includes at least one of the following: a text area, a title area, a table area, and a picture area.

[0137] It should be understood that PDF can be converted into images through layout analysis, and layout analysis can also automatically identify and locate key areas such as text, titles, tables and pictures in the document, which can improve the comprehensiveness and depth of PDF document content recognition.

[0138] For example, the paddle detection model can be used to perform layout analysis on the PDF.

[0139] S604a2: Segment the document to be processed based on the key area to obtain at least one image.

[0140] S604a3. For each image, extract valid data through optical character recognition.

[0141] It should be understood that optical character recognition can more accurately identify multimodal information contained in an image and convert it into text information, which can improve the efficiency of data extraction to a certain extent.

[0142] Exemplarily, optical character recognition may be implemented using the PP-OVRv3 system.

[0143] In some embodiments, when the document to be processed is a word document, valid data is extracted from the document to be processed based on the data extraction scheme. Figure 8 The figure shows a flow chart of a word document data extraction process provided by the present application, and the specific method includes:

[0144] S604b1. Determine whether the document to be processed contains a directory structure.

[0145] S604b2: If yes, after segmenting the document to be processed based on the directory structure, extract valid data from each portion of data obtained after segmentation.

[0146] It should be noted that for documents with a directory structure, the content of each chapter is relatively complete to a certain extent. Segmentation based on the directory structure can make the segmented content have a certain semantic coherence. Furthermore, the semantic understanding ability of the large language model can be used to perform semantic analysis on the segmented content, which can realize the fusion of structured information and semantic information and generate more knowledgeable knowledge.

[0147] Exemplarily, the directory structure of the word document can be extracted through the Office Open Xml specification.

[0148] For example, Fig. 9 As shown, this application provides an example diagram of a method for extracting word document data.

[0149] 1. Directory structure extraction: extract the directory structure in the document to be processed.

[0150] 2. Chapter content extraction is to segment the document content according to the directory structure extracted in step 1.

[0151] 3. Extract data and use the semantic understanding ability of the large language model to extract effective data from the segmented content.

[0152] S604b3: If not, segment the document to be processed based on semantic analysis to obtain semantically related paragraphs, and extract valid data from the semantically related paragraphs.

[0153] It should be noted that if there is no directory structure in the word document, it means that the content theme of the document is relatively compact, and the document can be segmented based on the semantic understanding ability of the large language model.

[0154] In some embodiments, when the document to be processed is an Excel document, valid data is extracted from the document to be processed based on the data extraction scheme. Fig.10 As shown, the specific methods include:

[0155] S604c1. Extract the header information and each row of data of the document to be processed.

[0156] It should be noted that the first row of a table usually contains column names or headers, which describe the meaning of each column of data. Extracting the header information can accurately understand and process each column of data in the table. Each row of data is equivalent to a data object, and the attribute information of the data object can be understood through the header information.

[0157] S604c2. Split each row of data, and the length of each part of the data obtained after the split is less than or equal to the length of the input data of the large language model.

[0158] It should be noted that in the solution of the present application, the semantic understanding ability of the large language model can be used to extract effective data from the segmented content, so each line of data is segmented according to the character length acceptable to the large language model.

[0159] In some embodiments, the number of text unit tokens in each line of content can be calculated, and each line of content can be split according to the character length that can be processed by the large language model.

[0160] S604c3. Configure corresponding header information for each part of the divided content to obtain valid data.

[0161] It should be noted that when splitting a table, the header information needs to be retained to ensure that the split data still maintains the structural consistency of the original table.

[0162] In some embodiments, when the document to be processed is a PPT document, valid data is extracted from the document to be processed based on the data extraction scheme. Fig.11As shown, it is a schematic diagram of a process of extracting PPT document data provided by this application, and the specific method includes:

[0163] S604d1. Extract the PPT chapter directory in the document to be processed.

[0164] Exemplarily, the chapter directory structure in the PPT document can be extracted using the Office Open Xml specification.

[0165] S604d2. After segmenting the document to be processed based on the PPT chapter directory, extract valid data from each segmented part.

[0166] Exemplarily, the directory structure and chapter structure of the PPT document can be extracted through the Office Open Xml specification.

[0167] For example, Fig.12 As shown in FIG. 1 , an example diagram of a method for extracting data from a PPT document provided by the present application is shown. In which:

[0168] 1. Extract the slide catalog, extract the slide chapter catalog of the PPT document.

[0169] 2. Chapter content segmentation: segment the document content according to the slide chapter directory extracted in step 1. For table content, a large language model can be used for segmentation and recognition. For image content, PP-OCRv3 technology can be used for text recognition.

[0170] 3. Extract data and use the semantic understanding ability of the large language model to extract effective data from the segmented content.

[0171] In some embodiments, after step S604, step S605 is further included. Fig.13 As shown:

[0172] S605: Perform semantic analysis on valid data to generate question-answer pairs and summaries.

[0173] Among them, question-answer pairs and summaries are used to train knowledge retrieval and question-answering models.

[0174] It should be understood that the question-answer pairs and summaries obtained through semantic analysis of valid data are summaries of the logical levels and data features in the valid data, and can support subsequent training of knowledge retrieval and question-answering systems.

[0175] Exemplarily, a large language model may be used to perform semantic analysis on valid data to generate question-answer pairs and summaries.

[0176] In some embodiments, after step S604, step S606 may also be included. Fig.14 As shown:

[0177] S606: Convert the format of the valid data into the format of the training data. Convert the training data into vectors and store them in a vector database.

[0178] It should be understood that converting valid data into vectors can remove redundancy and noise in the valid data, making the data more concise, and helping to reduce the amount of computing and storage resource costs occupied by storing in a vector database. In addition, storing in a vector database is conducive to the long-term use of data.

[0179] Exemplarily, the format of the training data may be text, CSV, or JSON.

[0180] Exemplarily, a bidirectional encoder representations from transformers (BERT) or GPT (generative pre-trained transformer, GPT) can be used to convert training data into vectors.

[0181] Exemplarily, the data stored in the vector library can be used to train the knowledge retrieval and question answering system in step S604.

[0182] The present application provides a data extraction method. First, the method can adopt corresponding data extraction schemes for documents of different formats by identifying the format of the document to be processed. The data extraction scheme can segment the document according to the structural characteristics of the document, breaking away from the limitation of the prior art of segmenting the document according to the character length or fixed rules, making the subject or structure of the segmented content relatively complete, and improving the efficiency of document segmentation.

[0183] In addition, this method can also use semantic analysis to convert valid data into question-answer pairs and summaries, which can support subsequent training of retrieval models and question-answering systems, and achieve full utilization of document resources.

[0184] Finally, this method uses vector technology to vectorize and store valid data, and can also achieve persistent storage of document resources, providing a basis for subsequent use of document resources.

[0185] like Fig.15 As shown, the knowledge generation system architecture diagram provided by the present application in combination with a data extraction method includes a document segmentation module, a data extraction module, a knowledge enhancement module, and a knowledge generation module.

[0186] like Fig.16 As shown, it is a flow chart of the knowledge generation system combined with a data extraction method provided by the present application.

[0187] Combine the following Fig.15 and Fig.16 The document processing flow of the system is described, including:

[0188] The document segmentation module can adopt the corresponding data extraction method according to the document format, generate key information by identifying the structural features of the document, and segment the document according to the key information.

[0189] The data extraction module can use the semantic understanding ability of the large language model to perform intelligent analysis on the segmented content and extract valid data. At the same time, it integrates machine learning technology to automatically identify key content in the document through training models.

[0190] The knowledge enhancement module further analyzes the valid data through a large language model to generate question-answer pairs and summaries. The generated question-answer pairs and summaries can be used for knowledge retrieval and question-answering system training.

[0191] The knowledge generation module converts the processed document content into training data format and uses vectorization technology to write the training data into a vector database to facilitate subsequent use of the data.

[0192] Fig.17 A data extraction method and a large language model interaction diagram provided in this application. Fig.17 As shown in the figure, during the whole processing, the large language model is called to perform semantic understanding on the segmented content and analyze its contextual relationship to extract valid data. In addition, in document segmentation, the large language model can also perform semantic understanding on Excel documents or word documents without directory structure to segment documents.

[0193] Finally, the large language model can further perform semantic analysis on valid data to summarize its question-answer pairs and summaries, providing support for subsequent training of knowledge retrieval and question-answering systems.

[0194] For example, Fig.18 A schematic diagram of the composition of a data extraction device provided in this application. Fig.18 As shown, the data extraction device includes: an acquisition module 181, a determination module 182, an extraction module 183 and a processing module 184.

[0195] The acquisition module 181 is used to acquire the document to be processed.

[0196] The determination module 182 is used to determine whether the format of the document to be processed is a preset format.

[0197] The determination module 182 is further used to determine a data extraction scheme according to the format of the document to be processed.

[0198] The extraction module 183 is used to extract valid data from the document to be processed based on the data extraction scheme.

[0199] The processing module 184 is used to further process the valid data into question-answer pairs and summaries or convert them into vectors and store them in a vector repository.

[0200] In a possible implementation, the extraction module is specifically used to extract valid data from the document to be processed based on the data extraction scheme when the document to be processed is a PDF document, including:

[0201] Perform layout analysis on the document to be processed and identify key areas, where the key areas include at least one of the following: text area, title area, table area, and picture area.

[0202] The document to be processed is segmented based on the key area to obtain at least one image.

[0203] For each image, valid data is extracted through optical character recognition.

[0204] In another possible implementation, the extraction module is specifically used to extract valid data from the document to be processed based on the data extraction scheme when the document to be processed is a word document, including:

[0205] Determines whether the document to be processed contains a directory structure.

[0206] If so, the document to be processed is segmented based on the directory structure, and valid data is extracted from each segmented part.

[0207] If not, the document to be processed is segmented based on semantic analysis to obtain semantically related paragraphs, and valid data is extracted from the semantically related paragraphs.

[0208] In another possible implementation, the extraction module is specifically used to extract valid data from the document to be processed based on the data extraction scheme when the document to be processed is an Excel document, including:

[0209] Extract the header information and each row of data of the document to be processed.

[0210] Each row of data is split, and the length of each part of the data obtained after the split is less than or equal to the length of the input data of the large language model.

[0211] Configure corresponding header information for each part of the divided content to obtain valid data.

[0212] In another possible implementation, the extraction module is specifically used to extract valid data from the document to be processed based on the data extraction scheme when the document to be processed is a PPT document, including:

[0213] Extract the PPT chapter directory in the document to be processed.

[0214] After segmenting the document to be processed based on the PPT chapter directory, valid data is extracted from each segmented part.

[0215] In another possible implementation, the processing module is further used to perform semantic analysis on valid data to generate question-answer pairs and summaries, and the question-answer pairs and summaries are used to train knowledge retrieval and question-answering models.

[0216] In another possible implementation, the processing module is further used to convert the format of the valid data into the format of the training data, convert the training data into vectors, and store them in a vector database.

[0217] In an exemplary embodiment, the present application also provides an electronic device, which may be the wireless coverage assessment device in the above method embodiment. Fig.19 The following is a schematic diagram of the composition of an electronic device provided in this application. Fig.19 As shown, the electronic device may include: a processor 191 and a memory 192; the memory 192 stores instructions executable by the processor 191; when the processor 191 is configured to execute the instructions, the electronic device implements the method described in the aforementioned method embodiment.

[0218] In an exemplary embodiment, the present application also provides a computer-readable storage medium on which computer program instructions are stored; when the computer program instructions are executed by a computer, the computer implements the method described in the aforementioned embodiment. The computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, a non-temporary computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device.

[0219] In an exemplary embodiment, the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes the above-mentioned related method steps to implement the data extraction method in the above-mentioned embodiment.

[0220] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data extraction method, characterized in that: The method comprises: Get the documents to be processed; Determine whether the format of the document to be processed is a preset format; the preset format is any one of the following: word document, PDF document, PPT document, excel document; If yes, determine a data extraction scheme according to the format of the document to be processed; one preset format corresponds to one data extraction scheme; the data extraction scheme is used to identify key information in the document to be processed, so as to segment the document to be processed based on the key information, and extract the valid data from the segmented content; the key information is used to represent the structural features of the document to be processed; Based on the data extraction scheme, valid data is extracted from the document to be processed.

2. The method according to claim 1, characterized in that In the case that the document to be processed is a PDF document, extracting valid data from the document to be processed based on the data extraction scheme includes: Performing layout analysis on the file to be processed to identify key areas; the key areas include at least one of the following: a text area, a title area, a table area, and a picture area; Segmenting the document to be processed based on the key area to obtain at least one image; For each image, the valid data is extracted by optical character recognition.

3. The method according to claim 1, characterized in that In the case where the document to be processed is a Word document, extracting valid data from the document to be processed based on the data extraction scheme includes: Determining whether the document to be processed contains a directory structure; If yes, the document to be processed is segmented based on the directory structure, and valid data is extracted from each segmented part; If not, the document to be processed is segmented based on semantic analysis to obtain semantically related paragraphs, and valid data is extracted from the semantically related paragraphs.

4. The method according to claim 1, characterized in that In the case where the document to be processed is an Excel document, extracting valid data from the document to be processed based on the data extraction scheme includes: Extracting header information and each row of data of the document to be processed; Split each row of data so that the length of each part obtained after segmentation is less than or equal to the length of the input data of the large language model; The corresponding header information is configured for each part of the divided content to obtain the valid data.

5. The method according to claim 1, characterized in that In the case that the document to be processed is a PPT document, extracting valid data from the document to be processed based on the data extraction scheme includes: Extract the PPT chapter directory in the document to be processed; After the document to be processed is segmented based on the PPT chapter directory, valid data is extracted from each segmented part.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: The valid data is semantically analyzed to generate question-answer pairs and summaries; the question-answer pairs and summaries are used to train knowledge retrieval and question-answering models.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Converting the format of the valid data into the format of training data; The training data is converted into vectors and stored in a vector database.

8. A data extraction device, characterized in that: The device comprises: an acquisition module, a determination module, and an extraction module; The acquisition module is used to obtain the documents to be processed; A determination module, used to determine whether the format of the document to be processed is a preset format; The determination module is further used to determine a data extraction scheme according to the format of the document to be processed; The extraction module is used to extract valid data from the document to be processed based on the data extraction scheme.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the electronic device implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes: computer software instructions; When the computer software instructions are executed in an electronic device, the electronic device implements the method according to any one of claims 1 to 7.

Images