Method and device for realizing contract element extraction analysis in financial field based on determiner, processor and computer readable storage medium thereof
By using an LSTM network based on indicator words and a dynamic programming algorithm based on conditional random fields, the problem of low efficiency in contract element extraction in the financial field is solved, and efficient and accurate contract element extraction is achieved.
Patent Information
- Application Number
- CN202210560338.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-23
- Publication Date
- 2025-12-16
- Estimated Expiration
- 2042-05-23
AI Technical Summary
In the field of natural language processing in finance, the lack of datasets leads to traditional methods being inefficient and time-consuming in extracting contract elements, and human operation is prone to errors.
We adopt a contract element extraction method based on indicator words in the financial field. We learn contextual semantic information through LSTM network layers, and combine attention mechanism and conditional random field with dynamic programming algorithm to obtain the optimal path, thereby improving the efficiency of element extraction.
While ensuring accuracy, it significantly reduced the time for extracting contract elements, improved efficiency, and lowered labor costs.
Smart Images
Figure CN114943232B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of natural language processing, in particular to the field of identifying contract element entities, and specifically to a method and device for implementing financial contract element extraction analysis based on demonstratives, a processor and a computer readable storage medium thereof. BACKGROUND
[0002] Natural language processing in the financial field is still in the exploratory stage. Finance itself is a highly professional field, and many words will have special meanings in the financial context. All sub-problems will have a unique understanding method, and the way of measuring the processing result in the financial field is also different from other fields. Therefore, NLP in the financial field needs to prepare special training data sets, and all NLP methods are currently based on a large number of data sets. The lack of data sets is one of the biggest problems faced by NLP in the financial field, which is also caused by the high professionalism and depth of the financial field.
[0003] At present, the financial field is full of a large number of financial professional texts, and the text formats of different business scenarios are not the same. In the daily work of auditing, managing, etc. of such financial texts, it is necessary to extract key information from a large number of texts and enter the business system. Traditional manual operation not only consumes a large amount of labor cost, but also is prone to operational risks. Traditional BILSTM+CRF sequence labeling method can effectively solve the automation of contract element extraction. Based on the traditional BILSTM+CRF and other traditional information extraction methods, considering some prior knowledge and attention mechanism, a contract element extraction analysis method based on demonstratives in the financial field is proposed. Compared with the early BILSTM+CRF method, the efficiency of element extraction in the contract can be greatly improved while ensuring the accuracy of element extraction, and the fund contract data set in our company has been verified. SUMMARY
[0004] The purpose of the present application is to overcome the shortcomings of the prior art, and to provide a method, device, processor and computer readable storage medium based on demonstratives for implementing financial contract element extraction analysis, which satisfies high efficiency, high accuracy and greatly reduced time consumption.
[0005] In order to achieve the above purpose, the method, device, processor and computer readable storage medium based on demonstratives for implementing financial contract element extraction analysis according to the present application are as follows:
[0006] The method for implementing financial contract element extraction analysis based on demonstratives, which mainly comprises the following steps:
[0007] (1) Sentence the document and perform text parsing;
[0008] (2) obtaining an initial vector representation of text characters, aggregating representation information of the demonstrative in the LSTM network layer, calculating the similarity between the demonstrative and other character vectors in the context, and weighted sum;
[0009] (3) converting the weighted information into a probability distribution of each entity label, obtaining the optimal path by dynamic programming algorithm, and obtaining the element entity information of the text.
[0010] Preferably, the step (2) specifically comprises the following steps:
[0011] (2.1) obtaining an initial vector representation for each character of the text through a global word vector;
[0012] (2.2) learning the semantic information of the context through a single-layer or multi-layer bidirectional LSTM network layer;
[0013] (2.3) obtaining the representation information of the demonstrative in the LSTM network layer according to the position coding of the demonstrative, and aggregating the representation information of the demonstrative in the LSTM network layer;
[0014] (2.4) calculating the similarity between the demonstrative and other character vectors in the context based on the attention mechanism through vector inner product;
[0015] (2.5) weighted sum of the calculated similarity and the vector information of each character in the LSTM network layer, and strengthening the representation information of each character.
[0016] Preferably, the step (3) specifically comprises the following steps:
[0017] (3.1) inputting the weighted information into a linear layer, and converting it into a probability distribution of each entity label through a fully connected network;
[0018] (3.2) inputting the label probability distribution of each character into a conditional random field network, and learning the parameters of the transition matrix in the conditional random field network by maximizing the likelihood function of the real path probability distribution;
[0019] (3.3) obtaining the optimal path by dynamic programming algorithm.
[0020] More preferably, the optimal path in the step (3.3) is specifically a path with the maximum sum of the label generation probability and the transition probability of each character in the whole sentence.
[0021] Preferably, the method further comprises a step of training initial parameters of the model, specifically comprising the following processing process:
[0022] The optimal value of the training parameter of the linear layer is obtained by minimizing the loss function through an Adam optimization algorithm.
[0023] The device for implementing the contract element extraction analysis in the financial field based on the demonstrative word, which mainly comprises:
[0024] The processor is configured to execute computer executable instructions.
[0025] The memory stores one or more computer executable instructions, which, when executed by the processor, implement the steps of the method for implementing the contract element extraction analysis in the financial field based on the demonstrative word.
[0026] The processor for implementing the contract element extraction analysis in the financial field based on the demonstrative word, which mainly comprises:
[0027] The computer readable storage medium, which mainly comprises:
[0028] The method, device, processor and computer readable storage medium for implementing the contract element extraction analysis in the financial field based on the demonstrative word, compared with the early BILSTM+CRF method in the prior art, ensure the accuracy of element extraction, the whole time consumption is far less than that of the traditional method, and the efficiency of element extraction in the contract can be greatly improved. BRIEF DESCRIPTION OF DRAWINGS
[0029] Figure 1 The source data format of the method for implementing the contract element extraction analysis in the financial field based on the demonstrative word.
[0030] Figure 2 The information extraction flowchart of the embodiment of the method for implementing the contract element extraction analysis in the financial field based on the demonstrative word. DETAILED DESCRIPTION
[0031] In order to more clearly describe the technical content of the present application, the following further describes in combination with specific embodiments.
[0032] The method for implementing the contract element extraction analysis in the financial field based on the demonstrative word, which comprises the following steps:
[0033] (1) sentence the document, and perform text analysis;
[0034] (2) obtaining an initial vector representation of a text character, aggregating representation information of a demonstrative in an LSTM network layer, calculating similarity of the demonstrative and other character vectors in a context, and weighted summation;
[0035] (3) converting the weighted information into a probability distribution of each entity label, obtaining an optimal path by a dynamic programming algorithm, and obtaining element entity information of the text.
[0036] As a preferred embodiment of the present application, the step (2) specifically comprises the following steps:
[0037] (2.1) obtaining an initial vector representation of each character of the text through a global word vector;
[0038] (2.2) learning semantic information of a context through a single-layer or multi-layer bidirectional LSTM network layer;
[0039] (2.3) obtaining representation information of a demonstrative in an LSTM network layer according to position coding of the demonstrative, and aggregating representation information of the demonstrative in the LSTM network layer;
[0040] (2.4) calculating similarity of the demonstrative and other character vectors in a context based on an attention mechanism through vector inner product;
[0041] (2.5) performing weighted summation on the calculated similarity and vector information of each character in the LSTM network layer, and strengthening representation information of each character.
[0042] As a preferred embodiment of the present application, the step (3) specifically comprises the following steps:
[0043] (3.1) inputting the weighted information into a linear layer, and converting into a probability distribution of each entity label through a fully connected network;
[0044] (3.2) inputting the label probability distribution of each character into a conditional random field network, and learning parameters of a transition matrix in the conditional random field network by maximizing a likelihood function of a real path probability distribution;
[0045] (3.3) obtaining an optimal path by a dynamic programming algorithm.
[0046] As a preferred embodiment of the present application, the optimal path in the step (3.3) is specifically a path with maximum sum of label generation probability and transition probability of each character in the whole text.
[0047] As a preferred embodiment of the present application, the method further comprises a step of training initial parameters of a model, specifically comprising the following processing process:
[0048] The optimal value of the training parameter of the linear layer is obtained by minimizing the loss function through an Adam optimization algorithm.
[0049] The device for implementing the contract element extraction analysis in the financial field based on the demonstrative word, wherein the device comprises:
[0050] A processor configured to execute computer executable instructions;
[0051] A memory storing one or more computer executable instructions, which, when executed by the processor, implement the steps of the method for implementing the contract element extraction analysis in the financial field based on the demonstrative word.
[0052] The processor for implementing the contract element extraction analysis in the financial field based on the demonstrative word, wherein the processor is configured to execute computer executable instructions, which, when executed by the processor, implement the steps of the method for implementing the contract element extraction analysis in the financial field based on the demonstrative word.
[0053] The computer readable storage medium having a computer program stored thereon, which can be executed by a processor to implement the steps of the method for implementing the contract element extraction analysis in the financial field based on the demonstrative word.
[0054] The element entities extracted from the fund contract mainly include "performance bonus minimum interval", "duration expression", "trustee name", "trust fee rate", etc., and there are about 70 elements to be extracted in each contract. According to the traditional BILSTM+CRF method, it takes about 30s to extract one element from one contract, and if multiple models are used for parallel extraction, the processing time of one contract is about 3-5 minutes.
[0055] In order to ensure the accuracy of the element extraction and greatly improve the efficiency of the element extraction in the contract, the present application proposes a contract element extraction method based on the demonstrative word, and the "performance bonus minimum interval" is taken as an example in the specification. The specific steps of the present application are as follows:
[0056] Step s1: The fund contract is mainly in the format of word / pdf, and the entire contract document needs to be parsed, uniformly converted to UTF-8 encoding, etc. The current sentence division method mainly uses special punctuation marks such as period, exclamation mark, etc. The source data format is shown in the following table: Figure 1 wherein "performance reward" is the demonstrative word information, and "6 months" is the "performance bonus minimum interval" entity to be extracted.
[0057] Step s2: The contract element extraction framework based on the demonstrative word is shown in the following table: Figure 2
[0058] Step s2.1: As shown, first get the initial vector representation of each character in each sentence through Glove word vector (global word vector). Figure 2
[0059] Step s2.2: Learn the semantic information of the context through a single-layer or multi-layer bidirectional LSTM (Long short-term memory) network layer.
[0060] Step s2.3: According to the position coding of the demonstrative, obtain the representation information of the demonstrative in the LSTM network, which has integrated the semantic information of the context.
[0061] Step s2.4: After aggregating the representation information of the demonstrative in the LSTM network, calculate the similarity between it and the context other word vectors.
[0062] Step s2.5: Weighted sum the above calculated similarity and the LSTM network vector information of each word in the entire sentence.
[0063] Step s3: The weighted information obtained in the above steps is first sent to a linear layer for dimension reduction conversion to the probability distribution of each entity label; secondly, it is sent to a conditional random field network to learn the parameters of the transition matrix in the conditional random field by maximizing the likelihood function of the true path probability distribution; finally, in the inference process, the Viterbi algorithm, a dynamic programming algorithm, is used to parse the optimal path from all possible marking paths, i.e. the sum of the marking probability and transition probability of each character in the entire sentence text is maximum, to obtain the required text element information.
[0064] The Viterbi algorithm uses the idea of dynamic programming to recursively calculate the most likely (locally optimal) path in the current state path using back pointers.
[0065] Step s4: Train the initial parameters of the model, and the initial parameters are shown in Table 1.
[0066] Table 1 Initial parameters for model training
[0067]
[0068]
[0069] Currently, the setting of the initial parameters is mainly based on empirical values, and the grid search method is used in subsequent optimization and adjustment. The Adam optimization algorithm is used to minimize the loss function to obtain the optimal values of the training parameters such as the weights and biases of the linear layer.
[0070] In the specific embodiments of the present application, the method of the present application is compared with the method of the prior art in terms of model effect results, and the comparison results are shown in Table 2.
[0071] Table 2 Comparison results of model results
[0072]
[0073] In order to more comprehensively evaluate the effect of the model, some indicators and reasoning time are listed as evaluation indicators. The indicators include accuracy, recall rate and F1 score.
[0074] The calculation formula of the accuracy is as follows:
[0075]
[0076] The calculation formula of the recall rate is as follows:
[0077]
[0078] The calculation formula of the F1 score is as follows:
[0079]
[0080] Wherein TP is True Positive, the number of actual positive samples in the predicted positive samples; FP is False Positive, the number of actual negative samples in the predicted positive samples; FN is False Negative, the number of actual positive samples in the predicted negative samples. The reasoning time is the time difference from text parsing to final model data result for the same contract and the same field, which is verified as much as possible in the same environment.
[0081] From the comparison results in Table 2, it can be seen that the information extraction method based on the demonstrative word only differs by 0.0084 in F1-score from the traditional method. However, the reasoning time is 1 / 80 of the traditional method. Especially in the actual scene, a contract often has 50-100 elements to be extracted, so the time consumption of the whole article will be much less than that of the traditional method.
[0082] The specific implementation scheme of the present embodiment can be referred to the related description in the above embodiments, which will not be repeated here.
[0083] It can be understood that the same or similar parts in the above embodiments can be mutually referred to, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.
[0084] It should be noted that, in the description of the present application, the terms "first", "second" and the like are used only for descriptive purposes, and cannot be construed as indicating or implying relative importance. In addition, in the description of the present application, the meaning of "a plurality of" is at least two, unless otherwise specified.
[0085] Any process or method descriptions or descriptions of the flow diagrams described herein or otherwise described in the present application can be understood as representing the steps of the code modules, segments or portions of the respective codes, and the scope of the preferred embodiments of the present application includes additional implementation in which the functions performed by the code modules, segments or portions are carried out in a different order, in substantially simultaneous fashion, or in reverse order, and the like, as will be understood by those skilled in the art.
[0086] It should be understood that parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented in hardware, and as in another embodiment, it can be implemented by any one or a combination of the following technologies known in the art: discrete logic circuit with logic gate circuit for implementing logic functions on data signals, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) and the like.
[0087] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiments can be completed by programs instructing relevant hardware, and the corresponding programs can be stored in a computer readable storage medium, which includes one or a combination of the steps of the method embodiments when executed.
[0088] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can exist physically, or two or more units can be integrated in one module. The above integrated module can be realized in the form of hardware or in the form of software functional module. The integrated module, if realized in the form of software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium.
[0089] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.
[0090] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or more embodiments or examples.
[0091] The method, device, processor and computer readable storage medium for realizing contract element extraction analysis in the financial field based on the demonstrative word of the present application, compared with the early BILSTM+CRF method in the prior art, guarantees the element extraction accuracy, the whole time consumption is far less than the traditional method, and the efficiency of element extraction in the contract can be greatly improved.
[0092] In this specification, the present application has been described with reference to its specific embodiments. However, it is obvious that various modifications and changes can be made without departing from the spirit and scope of the present application. Therefore, the specification and drawings should be considered as illustrative rather than limiting.
Claims
1. A method for extracting and analyzing contract elements in the financial field based on indicator words, characterized in that, The method includes the following steps: (1) Segment the document and parse the text; (2) Obtain the initial vector representation of the text characters, aggregate the representation information of the indicator words in the LSTM network layer, calculate the similarity between the indicator words and other character vectors in the context, and sum them by weight; (3) The weighted information is converted into the probability distribution of each entity label, and the optimal path is obtained by dynamic programming algorithm to obtain the element entity information of the text; Step (2) specifically includes the following steps: (2.1) Obtain an initial vector representation for each character of the text using global word vectors; (2.2) Learn the semantic information of the context through single-layer or multi-layer bidirectional LSTM network layers; (2.3) Based on the position encoding of the indicator words, obtain the representation information of the indicator words in the LSTM network layer, and aggregate the representation information of the indicator words in the LSTM network layer; (2.4) Based on the attention mechanism, the similarity between the indicator word and other character vectors in the context is calculated through the vector inner product; (2.5) The calculated similarity and the vector information of the LSTM network layer of each character are weighted and summed to strengthen the representation information of each character; Step (3) specifically includes the following steps: (3.1) Input the weighted information into the linear layer and transform it into the probability distribution of each entity label through a fully connected network for dimensionality reduction; (3.2) Input the probability distribution of each character into the conditional random field network, and finally maximize the likelihood function of the true path probability distribution to learn the transition matrix in the conditional random field network. (3.3) The optimal path is obtained by analyzing the Viterbi algorithm.
2. The method for extracting and analyzing contract elements in the financial field based on indicator words according to claim 1, characterized in that, The optimal path in step (3.3) is specifically the path that maximizes the sum of the tag generation probability and transition probability of each character in the entire text.
3. The method for extracting and analyzing contract elements in the financial field based on indicator words according to claim 1, characterized in that, The method also includes a step of training the initial parameters of the model, specifically including the following processing steps: The optimal values of the training parameters of the linear layer are obtained by minimizing the loss function using the Adam optimization algorithm.
4. An apparatus for performing indicator-based analysis of contract elements in the financial field, characterized in that, The device includes: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the method for extracting and analyzing contract elements in the financial field based on indicator words, as described in any one of claims 1 to 3.
5. A processor for implementing indicator-based contract element extraction and analysis in the financial field, characterized in that, The processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the method for extracting and analyzing contract elements in the financial field based on indicator words as described in any one of claims 1 to 3.
6. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method for extracting and analyzing contract elements in the financial field based on indicator words, as described in any one of claims 1 to 3.