Text extraction method, device, equipment and storage medium

Abstract

The application discloses a text extraction method and device, equipment and a storage medium, and belongs to the technical field of machine learning. The application obtains a text to be extracted, confirms corresponding extraction conditions based on the text to be extracted, generates corresponding entity label data according to the extraction conditions, inputs the text to be extracted and the entity label data into a pre-trained global head-tail neural network model, and extracts target text through the global head-tail neural network model. In the text extraction process in the embodiment of the application, reading comprehension technology is adopted, the extraction conditions are determined according to different texts to be extracted, entity annotation data with category description information is matched after the extraction conditions are obtained, and then extraction is performed through the pre-trained global head-tail neural network model, so that the problem of entity nesting in information extraction in the financial field is solved, and the recognition accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to text extraction methods, apparatus, devices, and storage media. Background Technology

[0002] With the rapid development of artificial intelligence and the increasing maturity of natural language processing technology, some tasks that previously required manual processing are gradually being transferred to machines. A typical example is document parsing and automatic data entry. This involves parsing documents, extracting different content based on different scenarios, and then automatically entering the data. Traditional manual parsing and data entry is tedious, complex, and labor-intensive, while traditional text extraction is ineffective for handling long, complex, and nested entities in the financial field.

[0003] Currently used AI extraction methods often suffer from entity nesting issues, especially with long sentences that may contain multiple types of entities. This can lead to errors and omissions during extraction, making it crucial to address how to improve the accuracy of text extraction. Summary of the Invention

[0004] The main objective of this invention is to provide a text extraction method, apparatus, device, and storage medium, aiming to solve the problem of low accuracy in text extraction.

[0005] To achieve the above objectives, the present invention provides a text extraction method, which includes the following steps:

[0006] Obtain the text to be extracted, and confirm the corresponding extraction conditions based on the text to be extracted;

[0007] Generate entity label data corresponding to the text to be extracted based on the extraction conditions;

[0008] The text to be extracted and the entity label data are input into a pre-trained global head-tail neural network model to extract the target text.

[0009] Optionally, before the step of obtaining the text to be extracted and confirming the corresponding extraction conditions based on the text to be extracted, the method includes:

[0010] Creating the global head-and-tail neural network model specifically includes:

[0011] Based on different types of documents, construct corresponding extraction questions, obtain all categories of the entities to be extracted, and generate entity category description text;

[0012] Based on the entity category description text, entity labels are assigned to the text in the document to generate training set data;

[0013] The training set data is input into the pre-trained model, and the text feature vector is obtained by parsing.

[0014] The text feature vector is input into the global first and last pointer layer, and the global pointer matrix is ​​output.

[0015] Based on the global pointer matrix and the training set data, the loss value of the model is calculated using the cross-entropy loss function. When the loss value meets a preset value, the global head-tail neural network model is trained.

[0016] Optionally, the step of inputting the text feature vector into the global first and last pointer layer and outputting the global pointer matrix includes:

[0017] The text feature vector is multiplied by the first and second fully connected layers pre-constructed in the global start and end pointer layers to obtain a start pointer matrix and an end pointer matrix.

[0018] The starting pointer matrix is ​​multiplied by the double affine matrix in the global first and last pointer layer to obtain a new starting pointer matrix;

[0019] The global pointer matrix is ​​obtained by multiplying the ending pointer matrix and the new starting pointer matrix.

[0020] Optionally, the step of assigning entity labels to the text in the document based on the entity category description text to generate training set data includes:

[0021] Based on the entity category description text, entity tags are assigned to the text in the document, and the category of the entity tag and the position information of the category in the original text are recorded;

[0022] Based on the location information and the category of the entity label, a category matrix of the data with entity labels is output using global pointer labels;

[0023] The category matrix is ​​populated with values ​​based on the entity labels to generate training set data.

[0024] Optionally, the step of obtaining the text to be extracted and confirming the corresponding extraction conditions based on the text to be extracted includes:

[0025] Obtain the document to be extracted and confirm the category of the document to be extracted;

[0026] The category of the text to be extracted is matched with the categories of the different types of documents to obtain the category matching result;

[0027] Based on the category matching results, corresponding extraction conditions are generated.

[0028] Optionally, the step of generating entity label data corresponding to the text to be extracted based on the extraction conditions includes:

[0029] Based on the extraction conditions, obtain the entity category description text corresponding to the document of the different types;

[0030] Entity label data corresponding to the text to be extracted is generated based on the entity category description text.

[0031] Optionally, the step of obtaining the text to be extracted and confirming the corresponding extraction conditions based on the text to be extracted includes:

[0032] Obtain application scenario requirements;

[0033] Based on the application scenario requirements and the extraction conditions corresponding to the text to be extracted.

[0034] Furthermore, to achieve the above objectives, the present invention also provides a text extraction device, the device comprising:

[0035] The acquisition module is used to acquire the text to be extracted and to confirm the corresponding extraction conditions based on the text to be extracted.

[0036] The generation module is used to generate entity label data corresponding to the text to be extracted according to the extraction conditions;

[0037] The extraction module is used to input the text to be extracted and the entity label data into a pre-trained global head-tail neural network model to extract the target text.

[0038] Optionally, the text extraction device further includes:

[0039] The model creation module is used to create the global head-and-tail neural network model, specifically including:

[0040] Based on different types of documents, construct corresponding extraction questions, obtain all categories of the entities to be extracted, and generate entity category description text;

[0041] Based on the entity category description text, entity labels are assigned to the text in the document to generate training set data;

[0042] The training set data is input into the pre-trained model, and the text feature vector is obtained by parsing.

[0043] The text feature vector is input into the global first and last pointer layer, and the global pointer matrix is ​​output.

[0044] Based on the global pointer matrix and the training set data, the loss value of the model is calculated using the cross-entropy loss function. When the loss value meets a preset value, the global head-tail neural network model is trained.

[0045] Optionally, the model creation module is further configured to:

[0046] The text feature vector is multiplied by the first and second fully connected layers pre-constructed in the global start and end pointer layers to obtain a start pointer matrix and an end pointer matrix.

[0047] The starting pointer matrix is ​​multiplied by the double affine matrix in the global first and last pointer layer to obtain a new starting pointer matrix;

[0048] The global pointer matrix is ​​obtained by multiplying the ending pointer matrix and the new starting pointer matrix.

[0049] Optionally, the model creation module is further configured to:

[0050] Based on the entity category description text, entity tags are assigned to the text in the document, and the category of the entity tag and the position information of the category in the original text are recorded;

[0051] Based on the location information and the category of the entity label, a category matrix of the data with entity labels is output using global pointer labels;

[0052] The category matrix is ​​populated with values ​​based on the entity labels to generate training set data.

[0053] Optionally, the acquisition module is further configured to:

[0054] Obtain the document to be extracted and confirm the category of the document to be extracted;

[0055] The category of the text to be extracted is matched with the categories of the different types of documents to obtain the category matching result;

[0056] Based on the category matching results, corresponding extraction conditions are generated.

[0057] Optionally, the generation module is further configured to:

[0058] Based on the extraction conditions, obtain the entity category description text corresponding to the document of the different types;

[0059] Entity label data corresponding to the text to be extracted is generated based on the entity category description text.

[0060] Optionally, the acquisition module is further configured to:

[0061] Obtain application scenario requirements;

[0062] Based on the application scenario requirements and the extraction conditions corresponding to the text to be extracted.

[0063] In addition, to achieve the above objectives, the present invention also provides an apparatus comprising: a memory, a processor, and a text extraction program stored in the memory and executable on the processor, the text extraction program being configured to implement the steps of the text extraction method as described above.

[0064] In addition, to achieve the above objectives, the present invention also provides a storage medium storing a text extraction program, which, when executed by a processor, implements the steps of the text extraction method as described above.

[0065] The text extraction method, apparatus, device, and storage medium proposed in this invention involve acquiring the text to be extracted, determining the corresponding extraction conditions based on the text, generating corresponding entity label data according to the extraction conditions, and then inputting the text to be extracted and the entity label data into a pre-trained global first-and-last neural network model to extract the target text. In the text extraction process of this invention, reading comprehension technology is employed to determine the extraction conditions based on different texts to be extracted. After obtaining the extraction conditions, entity label data with category description information is matched, and then extraction is performed using the pre-trained global first-and-last neural network model. Therefore, this method is suitable for scenarios in the financial field where the same entity fragment has multiple nested categories and for recognizing entities of varying lengths and complexities in the financial field, with high recognition accuracy. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the structure of a text extraction device in the hardware operating environment involved in the embodiments of the present invention;

[0067] Figure 2 This is a flowchart illustrating the first embodiment of the text extraction method of the present invention;

[0068] Figure 3 This is a flowchart illustrating the process of constructing a global head-tail neural network model in one embodiment of the text extraction method of the present invention;

[0069] Figure 4 This is a schematic diagram of a global beginning and end neural network model in an embodiment of the text extraction method of the present invention;

[0070] Figure 5 This is a schematic diagram of the global pointer layer in an embodiment of the text extraction method of the present invention;

[0071] Figure 6 This is an overall flowchart of an embodiment of the text extraction method of the present invention;

[0072] Figure 7 This is a schematic diagram of the functional modules of an embodiment of the text extraction device of the present invention.

[0073] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0074] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0075] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a text extraction device in the hardware operating environment involved in the embodiments of the present invention.

[0076] like Figure 1 As shown, the text extraction device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0077] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the text extraction device and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0078] like Figure 1 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and a text extraction program.

[0079] exist Figure 1In the text extraction device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the text extraction device of the present invention can be set in the text extraction device, and the text extraction device calls the text extraction program stored in the memory 1005 through the processor 1001 and executes the text extraction method provided in the embodiment of the present invention.

[0080] This invention provides a text extraction method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the text extraction method of the present invention.

[0081] In this embodiment, the text extraction method includes:

[0082] Step S10: Obtain the text to be extracted, and confirm the corresponding extraction conditions based on the text to be extracted.

[0083] Step S20: Generate entity label data corresponding to the text to be extracted according to the extraction conditions;

[0084] Step S30: Input the text to be extracted and the entity label data into the pre-trained global head-tail neural network model to extract the target text.

[0085] The text extraction method in this embodiment is mainly used for information extraction tasks in the financial field. Information extraction tasks are often called named entity recognition tasks in machine learning. Named entity recognition is a token-level multi-class classification problem. Existing solutions typically use CNNs / RNNs / BERT+CRF to handle this type of problem. Compared to SoftMax, CRF incorporates label constraints. However, because this sequence labeling uses the BILOU annotation framework, each token can only belong to one type, which cannot solve the problem of overlapping entities, i.e., entity nesting. Specifically, this refers to the situation where, in a sentence of text, a shorter entity is completely contained within another longer entity. Therefore, the text extraction method in this embodiment is proposed to address the above problem.

[0086] The following will provide a detailed explanation of each step:

[0087] Step S10: Obtain the text to be extracted, and confirm the corresponding extraction conditions based on the text to be extracted.

[0088] In one embodiment, the text to be extracted is first obtained, and the corresponding extraction conditions are determined based on the text. The text to be extracted can be various types of documents in the financial field, such as: agency contracts, legal clauses, insurance clauses, fund contracts, etc. Optionally, the text to be extracted can be obtained from a database or downloaded from a corresponding financial management system; the specific acquisition method is not limited. Specifically, different types of text require different extraction content. For example, insurance clauses require the extraction of information such as: insurance period, insurance premium, insured, policyholder, etc.; fund contracts require the extraction of information such as: fund redemption fee rate, fixed investment period, rate of return, purchase amount, holding information, etc. Therefore, different extraction conditions need to be obtained for the text to be extracted, and the required valid information is extracted from the text according to the extraction conditions. The extraction conditions can be manually set, or the text to be extracted can be classified into different types of documents by content recognition, and the extraction conditions for that type of document can be matched from existing extraction rules.

[0089] Step S20: Generate entity label data corresponding to the text to be extracted according to the extraction conditions;

[0090] In one embodiment, entity label data corresponding to the text to be extracted is generated based on extraction conditions. The text to be extracted can be preprocessed, for example, by dividing the complete text into multiple sentences, which can be long or short sentences. For example, the division can be based on punctuation marks to segment the text. The preprocessed text is used as input for subsequent steps. Entity label data is the description needed to extract the target entity. For example, a query question is constructed to represent the type of entity to be extracted. The target entity to be extracted is described using reading comprehension methods, allowing machine learning to identify the features of the question and answer, thereby completing the extraction task. In some feasible implementations, when performing entity recognition, a native textual interpretation of the entity is selected as the descriptive text. For example, for an entity location, the descriptive text is used: find the geographical location, and then, based on the number N entity categories, N data entries are generated from one original text.

[0091] Therefore, in this embodiment, the corresponding entity label data is obtained according to the extraction conditions, so as to perform entity extraction on the text to be extracted based on the entity label data.

[0092] Step S30: Input the text to be extracted and the entity label data into the pre-trained global head-tail neural network model to extract the target text.

[0093] In one embodiment, the text to be extracted and its corresponding entity label data are input into a pre-trained global first-and-last neural network model. The global first-and-last neural network model predicts and outputs the corresponding target text, that is, the target entity corresponding to the text to be extracted. The global first-and-last neural network model is trained based on a global pointer network, which treats the first and last positions as a whole. This means the model is trained and evaluated using entity sequences as the basic unit, ensuring consistency between training and prediction objectives. After training, the model can perform sequence labeling on the original text based on the corresponding entity label data, thereby finding the required text segment and the target text within the segment. Therefore, this embodiment, using a global first-and-last neural network model for named entity recognition, solves the problem of inconsistency and lack of continuity in the extraction of the first and last positions of traditional entities, thus improving the accuracy of text extraction.

[0094] This embodiment acquires the text to be extracted, determines the corresponding extraction conditions based on the text, generates corresponding entity label data according to the extraction conditions, and then inputs the text to be extracted and the entity label data into a pre-trained global first-and-last neural network model to extract the target text. In the text extraction process of this embodiment, reading comprehension technology is used to determine the extraction conditions based on different texts to be extracted. After obtaining the extraction conditions, entity label data with category description information is matched, and then extracted using a pre-trained global first-and-last neural network model. Therefore, this method is suitable for scenarios in the financial field where the same entity fragment has multiple nested categories, and is also suitable for the recognition of entities of varying lengths and complexities in the financial field, with high recognition accuracy.

[0095] Furthermore, based on the first embodiment of the text extraction method of the present invention, a second embodiment of the text extraction method of the present invention is proposed.

[0096] Reference Figure 3 , Figure 3 This is a flowchart illustrating the construction of a global head-and-tail neural network model in one embodiment of the text extraction method of the present invention. In the second embodiment, before the steps of obtaining the text to be extracted and confirming the corresponding extraction conditions based on the text to be extracted, the method includes:

[0097] Step A, creating the global head-and-tail neural network model, specifically includes:

[0098] Step A1: Construct corresponding extraction questions based on different types of documents, obtain all categories of the entities to be extracted, and generate entity category description text;

[0099] Step A2: Label the text in the document with entity tags based on the entity category description text to generate training set data;

[0100] Step A3: Input the training set data into the pre-trained model and parse it to obtain the text feature vector;

[0101] Step A4: Input the text feature vector into the global first and last pointer layer and output the global pointer matrix;

[0102] Step A5: Based on the global pointer matrix and the training set data, calculate the loss value of the model using the cross-entropy loss function. When the loss value meets a preset value, the global head-tail neural network model is trained.

[0103] In this embodiment, the global head-to-tail neural network model needs to be trained before use. Specifically, it is trained based on different types of document data. It is understandable that different types of text have different content, and therefore their corresponding label data will also be different. The conditions required for extracting from different types of documents are also different. Therefore, in this embodiment, different types of documents are classified and labeled accordingly.

[0104] Reference Figure 4 , Figure 4 This is a schematic diagram of a global head-and-tail neural network model in an embodiment of the text extraction method of the present invention. Figure 4 As shown, the structure of the entity recognition model based on reading comprehension is a pre-trained model similar to BERT (Bidirectional Encoder Representation from Transformer) connected to a global first and last pointer neural network layer.

[0105] The following example uses the extraction of insurance policy documents. The extraction of common insurance policy documents mainly involves extracting important financial information such as insurance name, insurance code, insurance period, insurance premium rate, and insurance type.

[0106] The training steps for this model are as follows:

[0107] (1) Organize all categories of entities to be extracted and generate description text for each category. For example, for insurance name correspondence: find the name of the insurance; for insurance rate correspondence: find the cost ratio of the insurance.

[0108] (2) Label the insurance text information in batches with entity tags and record the tag category and its text position information;

[0109] (3) Generate training set data. Each data entry consists of: a category description text, the original text, and the category's position information within the original text. If a data entry has multiple entities, generate multiple data entries. For example: This insurance policy has a three-year term and a premium rate of 5.0%. This data entry has two entity types: insurance period (three years, positions 5, 6) and premium rate (5.0%, positions 11, 14). Generate two data entries:

[0110] [Find the insurance term] [This insurance is for three years, with a premium rate of 5.0%.] [5, 6, three years]

[0111] [Find the insurance premium rate] [This insurance is for three years, with a premium rate of 5.0%.] [11, 14, 5.0%]

[0112] (4) Preprocessing the data with entity labels. Further, in one embodiment, the step of assigning entity labels to the text in the document based on the entity category description text to generate training set data includes:

[0113] Step A21: Tag the text in the document with entity tags based on the entity category description text, and record the category of the entity tag and the position information of the category in the original text;

[0114] Step A22: Based on the location information and the category of the entity label, use global pointer labels to output the category matrix of the data with entity labels;

[0115] Step A23: Fill the category matrix with values ​​according to the entity labels to generate training set data.

[0116] In one embodiment, global pointer labels are used in data preprocessing. The actual output labels are a matrix with dimensions (sequence length, sequence length), with default values ​​all being 0, and then the values ​​are filled according to the labels. Continuing with the above example, if the position information of the text "three years" in the text is 5,6, then the element [5,6] of the matrix is ​​1.

[0117] (5) Model construction: The sequence labeling model structure adopted is a BERT-like pre-trained model followed by a global pointer layer. The detailed construction is as follows: The text feature vector obtained after parsing by the pre-trained model represents the semantic information of each input sequence in various dimensions. At this time, the text feature vector is input into the global pointer layer to obtain the global pointer matrix.

[0118] Optionally, in one embodiment, the step of inputting the text feature vector into the global beginning and end pointer layer and outputting the global pointer matrix includes:

[0119] Step A41: Multiply the text feature vector with the first fully connected layer and the second fully connected layer pre-constructed in the global start and end pointer layer to obtain a start pointer matrix and an end pointer matrix.

[0120] Step A42: Perform a product transformation between the starting pointer matrix and the double affine matrix in the global first and last pointer layer to obtain a new starting pointer matrix;

[0121] Step A43: Multiply the ending pointer matrix and the new starting pointer matrix to obtain the global pointer matrix.

[0122] Reference Figure 5 , Figure 5 This is a schematic diagram of the global pointer layer in an embodiment of the text extraction method of the present invention. Specifically, two independent matrices are constructed, namely the NN contained in the fully connected layer in the diagram. start and NN end The feature vectors [batch_size, seq_len, hidden_size] of the text output from the last hidden layer of the BERT model are multiplied to establish the association between text features, resulting in a start pointer matrix (i.e., the sentence beginning matrix in the figure) and an end pointer matrix (i.e., the sentence ending matrix in the figure) [batch_size, seq_len, num_cate, head]. Then, an independent attention matrix, i.e., the biaffine matrix [hidden_size, num_cate, head_size], is constructed and multiplied with the start pointer matrix according to the Einstein summation convention to obtain a new start pointer matrix (omitted in the figure). Finally, the new start pointer matrix and the sentence ending matrix are multiplied to output a global pointer matrix with dimensions [batch_size, num_cate, seq_len, seq_len]. This ensures that the start and end matrices of the label categories are no longer unrelated; each label category has a matrix with dimensions (sequence length, sequence length) representing the starting position of the label category. Since we are extracting entities based on reading comprehension, each sample requires a category description as a constraint in the input. Therefore, the final output entities are the descriptions of their corresponding entity categories, resulting in a final matrix with only 1 category. Here, `seq_len` refers to the text sequence length, `hidden_size` refers to the hidden layer size, `batch_size` refers to the number of samples in the batch, `num_cate` refers to the number of entity categories to be identified, and `head` refers to the head size of the attention matrix.

[0123] (6) Train and fit the model using labeled data. Construct the model's loss function using cross-entropy, train the model using this loss function, and set a preset value. Training is complete when the calculated value of the loss function meets the preset value. During the training process, a partial validation set is used to verify the model's performance.

[0124] (7) After the model training is completed, save the overall structure and weights of the model, load the saved model, and make predictions based on the input text information and different category description texts. The model can then predict and extract the corresponding entity fragments in the financial text information.

[0125] Optionally, in step A1, when constructing corresponding extraction questions based on different document types, the specific application scenario of the document is considered to obtain the actual application requirements, and extraction questions are constructed according to different scenario requirements. Further, corresponding to this approach, in one embodiment, the step of obtaining the text to be extracted and confirming the corresponding extraction conditions based on the text to be extracted includes:

[0126] Step S11: Obtain application scenario requirements;

[0127] Step S12: Based on the application scenario requirements and the text to be extracted, confirm the corresponding extraction conditions.

[0128] In one embodiment, the application scenario requirements are obtained, and the corresponding extraction conditions are determined based on these requirements and the text to be extracted. The application scenario requirements refer to the intended use of the extracted target text. For example, given an insurance contract, if the application requirement is to build a database, it needs to include detailed data such as the conditions, results, and limitations of insurance liability payment, policyholder / insured information, and signing time. However, if the application requirement is to statistically analyze business data, only data related to the business aspects of the insurance contract, such as the insured amount, insurance period, and insurance type, need to be extracted; other clause-related content does not need to be extracted. Therefore, when obtaining the extraction conditions, the application scenario requirements of the text to be extracted must also be considered.

[0129] In this embodiment, extraction problems are constructed based on different document types, and all categories of the entities to be extracted are obtained. Entity category description text is generated, and then entity labels are added to the text in the document according to the entity category description text. Training set data is generated and input into a pre-trained model. The text feature vectors are parsed and input into a global first-and-last pointer layer, which outputs a global pointer matrix. Based on the global pointer matrix and the training set data, the loss value of the model is calculated using the cross-entropy loss function. When the loss value meets a preset value, the global first-and-last neural network model is trained. In this embodiment, the input text and entity category description text for training the global first-and-last neural network model are obtained based on reading comprehension. Therefore, when performing named entity recognition, the problem of entity nesting in traditional methods is solved. At the same time, named entity recognition based on the global pointer network establishes the connection between the first and last entities, solving the problem of inconsistency and lack of continuity in the extraction of the first and last entities in traditional methods.

[0130] Furthermore, based on the previous embodiments of the text extraction method of the present invention, a third embodiment of the text extraction method of the present invention is proposed.

[0131] In the third embodiment, the step of obtaining the text to be extracted and confirming the corresponding extraction conditions based on the text to be extracted includes:

[0132] Step S11: Obtain the text to be extracted and confirm the category of the text to be extracted;

[0133] Step S12: Match the category of the text to be extracted with the categories of the different types of documents, and obtain the category matching result;

[0134] Step S13: Generate corresponding extraction conditions based on the category matching results.

[0135] In one embodiment, after obtaining the text to be extracted, it is classified, and extraction conditions are matched according to the category of each text. Understandably, during training, different types of documents are acquired and labeled with corresponding category description text. Based on the categories of different document types used during training, the category description text is classified to generate extraction conditions. When the text to be extracted is obtained, its category is confirmed. For example, if the text to be extracted is an insurance document, the corresponding insurance category extraction conditions are matched, thereby obtaining the category description data of the content to be extracted. Specifically, the text to be extracted can be classified as: insurance documents, legal clauses, contracts, and fund documents. (Refer to...) Figure 6 , Figure 6This is an overall flowchart of an embodiment of the text extraction method of the present invention, which summarizes the implementation process of the present invention. First, the input text, that is, the text to be extracted, is processed into N original texts (input text in the figure) and N corresponding entity annotation data (entity category description 1 in the figure). The original texts and entity annotation data are input into a global head-tail neural network model. The model obtains the head-tail score matrix of each entity category, and finally obtains all entities of the input text, that is, the target text.

[0136] Further, in one embodiment, the step of generating entity label data corresponding to the text to be extracted based on the extraction conditions includes:

[0137] Step S21: Obtain entity category description text corresponding to the document of the different types according to the extraction conditions;

[0138] Step S22: Generate entity label data corresponding to the text to be extracted based on the entity category description text.

[0139] In one embodiment, a large amount of document data is acquired during model training, and corresponding category description texts are set. Based on the target documents and category description texts in the training data, corresponding extraction conditions are generated. When it is necessary to identify the text to be extracted, the corresponding category description text is selected directly based on the existing training data, and entity label data is obtained based on the category description text. The text to be extracted is then processed to obtain the original text. The original text and entity label data are then input together into the global head-tail neural network model for text extraction.

[0140] This embodiment obtains the text to be extracted and confirms its category. The confirmed category is then matched with the categories of different document types. Based on the category matching results, corresponding extraction conditions are generated. These extraction conditions are generated based on the documents and corresponding category description text used during training. This embodiment achieves rapid generation of extraction conditions, thereby improving the overall text extraction speed.

[0141] The present invention also provides a text extraction device. For example... Figure 7 As shown, Figure 7 This is a schematic diagram of the functional modules of an embodiment of the text extraction device of the present invention.

[0142] The text extraction device of the present invention includes:

[0143] The acquisition module is used to acquire the text to be extracted and to confirm the corresponding extraction conditions based on the text to be extracted.

[0144] The generation module is used to generate entity label data corresponding to the text to be extracted according to the extraction conditions;

[0145] The extraction module is used to input the text to be extracted and the entity label data into a pre-trained global head-tail neural network model to extract the target text.

[0146] Optionally, the text extraction device further includes:

[0147] The model creation module is used to create the global head-and-tail neural network model, specifically including:

[0148] Based on different types of documents, construct corresponding extraction questions, obtain all categories of the entities to be extracted, and generate entity category description text;

[0149] Based on the entity category description text, entity labels are assigned to the text in the document to generate training set data;

[0150] The training set data is input into the pre-trained model, and the text feature vector is obtained by parsing.

[0151] The text feature vector is input into the global first and last pointer layer, and the global pointer matrix is ​​output.

[0152] Based on the global pointer matrix and the training set data, the loss value of the model is calculated using the cross-entropy loss function. When the loss value meets a preset value, the global head-tail neural network model is trained.

[0153] Optionally, the model creation module is further configured to:

[0154] The text feature vector is multiplied by the first and second fully connected layers pre-constructed in the global start and end pointer layers to obtain a start pointer matrix and an end pointer matrix.

[0155] The starting pointer matrix is ​​multiplied by the double affine matrix in the global first and last pointer layer to obtain a new starting pointer matrix;

[0156] The global pointer matrix is ​​obtained by multiplying the ending pointer matrix and the new starting pointer matrix.

[0157] Optionally, the model creation module is further configured to:

[0158] Based on the entity category description text, entity tags are assigned to the text in the document, and the category of the entity tag and the position information of the category in the original text are recorded;

[0159] Based on the location information and the category of the entity label, a category matrix of the data with entity labels is output using global pointer labels;

[0160] The category matrix is ​​populated with values ​​based on the entity labels to generate training set data.

[0161] Optionally, the acquisition module is further configured to:

[0162] Obtain the document to be extracted and confirm the category of the document to be extracted;

[0163] The category of the text to be extracted is matched with the categories of the different types of documents to obtain the category matching result;

[0164] Based on the category matching results, corresponding extraction conditions are generated.

[0165] Optionally, the generation module is further configured to:

[0166] Based on the extraction conditions, obtain the entity category description text corresponding to the document of the different types;

[0167] Entity label data corresponding to the text to be extracted is generated based on the entity category description text.

[0168] Optionally, the acquisition module is further configured to:

[0169] Obtain application scenario requirements;

[0170] Based on the application scenario requirements and the extraction conditions corresponding to the text to be extracted.

[0171] The present invention also provides a storage medium.

[0172] The present invention stores a text extraction program on a storage medium, which, when executed by a processor, implements the steps of the text extraction method described above.

[0173] The method implemented when the management program running on the processor is executed can be referred to in various embodiments of the management method of the present invention, and will not be repeated here.

[0174] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0175] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0176] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0177] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A text extraction method, characterized by, The text extraction method comprises the following steps: A global start-end neural network model is created, specifically comprising: Based on different types of documents, corresponding extraction problems are constructed, and all categories of entities that need to be extracted are obtained, and entity category description texts are generated; Based on the entity category description text, entity labels are given to the text in the document, and training set data is generated; The training set data is input into a pre-trained model, and the text feature vector is obtained by parsing; The text feature vector is input into the global start-end pointer layer to output the global pointer matrix; this step specifically comprises: the text feature vector is multiplied with the pre-constructed first full connection layer and the second full connection layer in the global start-end pointer layer to obtain an initial pointer matrix and an end pointer matrix; the initial pointer matrix is multiplied with the bi-affine matrix in the global start-end pointer layer to obtain a new initial pointer matrix; the end pointer matrix and the new initial pointer matrix are multiplied to obtain the global pointer matrix; Based on the global pointer matrix and the training set data, the loss value of the model is calculated by the cross-entropy loss function, and when the loss value meets the preset value, the global start-end neural network model is trained; Obtain the text to be extracted, and confirm the corresponding extraction condition based on the text to be extracted; According to the extraction condition, the entity label data corresponding to the text to be extracted is generated; The text to be extracted and the entity label data are input into the pre-trained global start-end neural network model to extract the target text.

2. The text extraction method of claim 1, wherein, The step of giving entity labels to the text in the document based on the entity category description text to generate training set data comprises: Based on the entity category description text, entity labels are given to the text in the document, and the category and position information of the entity labels in the original text are recorded; Based on the position information and the category of the entity labels, a global pointer label is used to output a category matrix of data with entity labels; According to the entity labels, the value of the category matrix is filled to generate training set data.

3. The text extraction method of claim 1, wherein, The step of obtaining the text to be extracted and confirming the corresponding extraction condition based on the text to be extracted comprises: Obtain the document to be extracted, and confirm the category of the document to be extracted; Match the category of the text to be extracted with the categories of the different types of documents to obtain a category matching result; Based on the category matching result, the corresponding extraction condition is generated.

4. The text extraction method of claim 3, wherein, The step of generating the entity label data corresponding to the text to be extracted based on the extraction condition comprises: According to the extraction condition, the entity category description text corresponding to the different types of documents is obtained; Based on the entity category description text, the entity label data corresponding to the text to be extracted is generated.

5. The text extraction method of claim 1, wherein, The step of obtaining the text to be extracted and confirming the corresponding extraction condition based on the text to be extracted comprises: Obtain the application scenario requirement; Based on the application scenario requirement and the text to be extracted, the corresponding extraction condition is confirmed.

6. A text extraction apparatus characterized by comprising: The text extraction device comprises: The model creating module is further configured to construct an extraction question corresponding to different types of documents, obtain all categories of entities to be extracted corresponding to the extraction question, generate entity category description text, label text in the document with an entity tag based on the entity category description text, and generate training set data; input the training set data into a pre-trained model, parse to obtain a text feature vector; input the text feature vector into a global start-end pointer layer, and output a global pointer matrix; calculate a loss value of the model based on the global pointer matrix and the training set data through a cross-entropy loss function, and when the loss value meets a preset value, train to obtain a global start-end neural network model; the model creating module is further configured to perform product transformation on the text feature vector and a first fully connected layer and a second fully connected layer pre-constructed in the global start-end pointer layer, to obtain a start pointer matrix and an end pointer matrix; perform product transformation on the start pointer matrix and a bi-affine matrix in the global start-end pointer layer, to obtain a new start pointer matrix; perform product transformation on the end pointer matrix and the new start pointer matrix to obtain the global pointer matrix. The obtaining module is configured to obtain a text to be extracted, and confirm an extraction condition corresponding to the text to be extracted based on the text to be extracted. The generating module is configured to generate entity tag data corresponding to the text to be extracted according to the extraction condition. The extracting module is configured to input the text to be extracted and the entity tag data into a pre-trained global start-end neural network model, and extract a target text.

7. A text extraction device, characterized by, The device comprises a memory, a processor, and a text extraction program stored on the memory and executable on the processor, and the text extraction program is configured to implement the steps of the text extraction method according to any one of claims 1 to 5.

8. A storage medium, characterized by The storage medium stores a text extraction program, and the text extraction program is executed by the processor to implement the steps of the text extraction method according to any one of claims 1 to 5.

Images