Method, system and equipment for structuring fund contracts in the financial field

Abstract

The present invention belongs to the field of text processing, and specifically relates to a method, system and device for structuring fund contracts in the financial field, aiming to solve the problems of the existing structured text recognition methods that have limited vertical field coverage, low accuracy and incomplete information. The present invention includes: converting the document to be processed into a hypertext format to obtain a hypertext to be processed; based on the hypertext to be processed, obtaining element positioning; based on the element positioning, extracting element schemes; based on the element schemes, outputting business content according to a business logic rule library; based on the business content, combining into structured results. The present invention adds a dictionary knowledge base and a business logic rule library, and introduces a newer deep learning model, namely the MRC model, which greatly improves the accuracy and efficiency of the model, and builds a more complete framework system for structuring fund contracts in the financial field, which can be plugged in and out, has strong versatility, and can be quickly copied to different scenarios.

Description

Background Art

[0002] With the development of the digital age, technology + artificial intelligence technology is increasingly favored by business personnel in the financial field. The text formats in the financial field are diverse, and the unified format type has many different categories. The text is lengthy, up to hundreds of pages, with more than 200 structured elements. At the same time, traditional technology is difficult to achieve multi-format, multi-type, and multi-element structuring.

[0003] In order to improve the accuracy of structuring, cover multiple types of texts, better serve business personnel, and reduce costs and increase efficiency, our device builds a professional knowledge base in the financial field and a business post-processing logic rule base on the traditional system, adds a specific format text parser, and first parses the fund contract text file to be structured using the text parser, and then converts it into a standard file format to obtain the processed text, and then classifies the text through the classifier, passes through the dictionary knowledge base, and further outputs the results through the model, combined with the business logic rule base, outputs the results required by the business, and finally obtains the structured Key-Value pairs through post-processing, and proposes a method and device for structuring fund contracts in the financial field. At the same time, this device can be applied to multiple operating devices such as servers, PCs, tablets, and mobile phones.

[0004] The method and device can speed up the work efficiency of business personnel, improve the accuracy of structuring, and improve the operating platform in the financial field. Summary of the invention

[0005] In order to solve the above problems in the prior art, namely, the problems that the existing structured text recognition methods have limited vertical field coverage, low accuracy, and incomplete information, the present invention provides a method for structuring fund contracts in the financial field, the method comprising:

[0006] Step S100, converting the document to be processed into a hypertext format to obtain a hypertext to be processed;

[0007] Step S200, obtaining element location based on the hypertext to be processed;

[0008] Step S300, extracting element content based on the keyword knowledge base, extraction scheme and element positioning;

[0009] Step S400, based on the element content, outputting the business content according to the business logic rule library;

[0010] Step S500: Combining into structured results based on the business content; the data type of the structured results is a key-value pair.

[0011] In some preferred implementations, converting the document to be processed into a hypertext format to obtain the hypertext to be processed is specifically:

[0012] The schema standard of the fund contract in the financial field is configured according to the logical knowledge base; the schema standard includes the possible chapters of the target element, the possible expression mode of the target element, the title, the unified and standardized title, the chapter number, the unified and standardized chapter number, the title number and the unified and standardized title number; wherein the unified and standardized title is the title of the chapter where the target element is located, which is obtained by mapping according to the chapter-standard chapter mapping relationship;

[0013] Convert entities or objects that meet the schema standard in the PDF document to be processed into hypertext format through PDFparser;

[0014] Convert entities or objects that meet schema standards in the DOC, DOCX or XML document to be processed into hypertext format through Parser;

[0015] Through OCR, entities or objects that meet the schema standard in the scanned document or image to be processed are converted into hypertext format.

[0016] In some preferred implementations, obtaining the element location based on the hypertext to be processed is specifically:

[0017] Based on the preset element knowledge base, regular matching and similarity matching methods are used to determine whether the current sentence of the hypertext to be processed is the sentence where the target element is located. If it is the sentence where the target element is located, the target element location is obtained and the next sentence is set as the current sentence until all sentences of the hypertext to be processed are traversed.

[0018] In some preferred implementations, extracting element content based on a keyword knowledge base, an extraction scheme, and element positioning is specifically as follows:

[0019] Recording the preset extraction scheme of each target element through a JSON file; the extraction scheme includes a digital extraction scheme, a text extraction scheme, a sentence-level extraction scheme, and a paragraph-level extraction scheme;

[0020] Based on the keyword knowledge base, extraction scheme and element positioning, the element content is extracted through regular matching and entity recognition model based on deep learning; specifically:

[0021] Extracting element contents respectively through the first entity recognition model, the second entity recognition model and the third entity recognition model to obtain first element contents, second element contents and third element contents respectively;

[0022] The two elements with the highest similarity are selected as the final elements by voting.

[0023] In some preferred embodiments, after outputting service content according to the service logic rule library based on the element solution, it further includes a process of post-processing symbols and noises in the service content.

[0024] In some preferred embodiments, the service content is combined into a structured result based on the service content, specifically, the service content is combined into a structured result through a service logic knowledge base.

[0025] In some preferred embodiments, the service logic knowledge base is extracted by a trained relation extraction model, and the training method of the trained relation extraction model includes:

[0026] Step A100: Obtain a financial scenario document with manual annotation as a training set;

[0027] Step A200: Extract the training set text logic rules from the training set through the relation extraction model; the relation extraction model is an MRC model;

[0028] Step A300: Calculate the performance of the extraction result through the training set text logic rules, and adjust the model parameters by the stochastic gradient method until the performance of the extraction result reaches a set threshold and the loss function converges, so as to obtain a trained relation extraction model;

[0029] The loss function adopts a conditional random field function:

[0030] Loss=-logP(y1,y2,...,y n 丨X)

[0031] where X represents the input sample, y1 represents the output of the first path of the relation extraction model, y2 represents the output of the second path of the relation extraction model, y n represents the output of the nth path of the relation extraction model, n represents the total number of paths of the relation extraction model, and P represents the maximum probability.

[0032] In the second aspect of the present invention, a fund contract structuring system in the financial field is disclosed. The system includes: a format conversion module, an element positioning module, a solution extraction module, a service content output module, and an event combination module;

[0033] The format conversion module is configured to convert the document to be processed into a hypertext format to obtain the hypertext to be processed;

[0034] The element positioning module is configured to obtain element positioning based on the hypertext to be processed;

[0035] The solution extraction module is configured to extract an element solution based on the element positioning;

[0036] The business content output module is configured to output business content based on the element scheme and according to the business logic rule base;

[0037] The event combination module is configured to combine into a structured result based on the business content; the data type of the structured result is a key-value pair.

[0038] The third aspect of the present invention proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned financial field fund contract structuring method.

[0039] In a fourth aspect of the present invention, a computer-readable storage medium is proposed, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned method for structuring fund contracts in the financial field.

[0040] Beneficial effects of the present invention:

[0041] (1) The present invention provides a method for structuring fund contracts in the financial field, adds a dictionary knowledge base and a business logic rule base, and introduces a relatively new deep learning model, namely the MRC (reading comprehension) model, which greatly improves the accuracy and efficiency of the model;

[0042] (2) A more complete framework system for the structure of fund contracts in the financial field has been built, which can be plugged in and out. This system framework can be used to extract elements such as private equity funds, public equity funds, and futures contracts. It has strong versatility and can be quickly copied to different scenarios;

[0043] (3) According to fund contracts in different formats, corresponding intelligent parsing tasks are adapted and can be quickly converted into hypertext;

[0044] (4) For text sentences, a variety of location positioning and extraction schemes have been developed to quickly filter out useless text and improve the efficiency of structuring;

[0045] (5) The texts solved are longer, up to more than 100 pages, with more structured elements, achieving standard structured output of more than 200 elements. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0047] Figure 1 It is a flowchart of a method for structuring a fund contract in the financial field according to an embodiment of the present invention;

[0048] Figure 2 It is a schematic diagram of the principle of the method for structuring a fund contract in the financial field in an embodiment of the present invention; DETAILED DESCRIPTION

[0049] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0050] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0051] The present invention provides a method for structuring fund contracts in the financial field, adds a dictionary knowledge base and a business logic rule base, introduces a newer deep learning model, namely the MRC (reading comprehension) model, which greatly improves the accuracy and efficiency of the model; and builds a more complete framework system for structuring fund contracts in the financial field, which can be plugged in and out, has strong versatility, and can be quickly copied to different scenarios.

[0052] A method for evaluating the security effectiveness of cultural relics based on system dynamics of the present invention comprises:

[0053] Step S100, converting the document to be processed into a hypertext format to obtain a hypertext to be processed;

[0054] Step S200, obtaining element location based on the hypertext to be processed;

[0055] Step S300, extracting element content based on the keyword knowledge base, extraction scheme and element positioning;

[0056] Step S400, based on the element content, outputting the business content according to the business logic rule library;

[0057] Step S500: Combining into structured results based on the business content; the data type of the structured results is a key-value pair.

[0058] In order to more clearly illustrate the system of the present invention, Figure 1 and Figure 2 Each step in the embodiment of the present invention is described in detail.

[0059] The present invention parses fund contract texts of different formats, classifies different fund contracts based on a classification model, obtains hypertext, and then obtains a positioning structure in combination with a positioning scheme. The extraction results are obtained based on a keyword knowledge base and an extraction scheme (deep learning model). Based on the extraction results, the business logic rule library + post-processing is used to output a standard structured result. Not only is the business logic rule library of the fund contract constructed, but the original framework is also greatly improved, which improves the accuracy and stability of the structure. It also has good compatibility and portability, and can be nested, developed and maintained on different platforms.

[0060] The first embodiment of the present invention provides a method for structuring a fund contract in the financial field, comprising:

[0061] Step S100, converting the document to be processed into a hypertext format to obtain a hypertext to be processed;

[0062] In this embodiment, the process of converting the document to be processed into a hypertext format to obtain the hypertext to be processed is as follows:

[0063] The schema standard of the fund contract in the financial field is configured according to the logical knowledge base; the schema standard includes the possible chapters of the target element, the possible expression mode of the target element, the title, the unified and standardized title, the chapter number, the unified and standardized chapter number, the title number and the unified and standardized title number; wherein the unified and standardized title is the title of the chapter where the target element is located, which is obtained by mapping according to the chapter-standard chapter mapping relationship; the schema standard example is: extracting the value of "whether it is an installment product", the extraction result is {"whether it is an installment product": "yes"}, {"Is it an installment product": "No"} or {"Is it an installment product": "-"}. The schema standard example also includes a general expression and logic rule base: ["(This fund.*? Duration.*? Open subscription to establish fund units.*?.)","(This fund.*? Establish the .*?[0-9]+.*? Period fund units)","(Fund.*? Issued in installments)","(The [0-9](?![Point])+.*? Fund units)","(The [0-9 One Two Three Four Five Six Seven Eight Nine Ten] Period.*? Shares)"]; Among them, .*? means matching any character;

[0064] Convert entities or objects that meet the schema standard in the PDF document to be processed into hypertext format through PDFparser;

[0065] Convert entities or objects that meet schema standards in the DOC, DOCX or XML document to be processed into hypertext format through Parser;

[0066] Through OCR, entities or objects that meet the schema standard in the scanned document or image to be processed are converted into hypertext format.

[0067] When performing hypertext conversion, this embodiment only retains the chapters where the target elements exist, and filters out the chapters where the target elements do not exist, thereby maximizing the effectiveness of the converted hypertext and further improving the timeliness of the contract structure.

[0068] Step S200, obtaining element location based on the hypertext to be processed;

[0069] In this embodiment, the element location is obtained based on the hypertext to be processed, specifically:

[0070] Based on the preset element knowledge base, regular matching and similarity matching methods are used to determine whether the current sentence of the hypertext to be processed is the sentence where the target element is located. If it is the sentence where the target element is located, the target element location is obtained and the next sentence is set as the current sentence until all sentences of the hypertext to be processed are traversed.

[0071] The sentence-level positioning scheme includes: elements, chapter number, subheading number, sentence number, and the first character number of the sentence level; the image-level positioning scheme includes: elements, chapter number, and pixel value; the table-level positioning scheme includes: elements, chapter number, table coordinates, and the coordinate values ​​of the upper left corner and lower right corner.

[0072] Extraction: After accurately locating the elements to be extracted, use the extraction model to extract them.

[0073] Step S300, extracting element content based on the keyword knowledge base, extraction scheme and element positioning;

[0074] In this embodiment, the step S300 is specifically as follows:

[0075] The extraction scheme of each preset target element is recorded through a JSON file; the extraction scheme includes a digital extraction scheme, a text extraction scheme, a sentence-level extraction scheme and a paragraph-level extraction scheme; the extraction schemes are all stored in a Class class to form multiple extraction schemes; describing all extraction schemes in a Class class is conducive to the rapid duplication and expansion of extraction schemes, such as adding extraction schemes of pictures or tables;

[0076] Based on the keyword knowledge base, extraction scheme and element positioning, the element content is extracted through regular matching and entity recognition model based on deep learning; specifically:

[0077] Extracting element contents respectively through the first entity recognition model, the second entity recognition model and the third entity recognition model to obtain first element contents, second element contents and third element contents respectively;

[0078] Select the two element contents with the highest similarity by voting as the final element contents. Specifically, if the first element content and the second element content have a high similarity, but the first element content and the second element content have a low similarity with the third element content respectively, select the first element content and the second element content as the final element contents.

[0079] Step S400, based on the element content, obtain service content according to the service logic rule library;

[0080] Since the schema is configured in advance when performing hypertext conversion, the acquisition of service content according to the logic knowledge library can also be added quickly.

[0081] In this embodiment, after step S400, it further includes a process of post-processing symbols and noises in the service content.

[0082] Step S500, combine them into a structured result based on the service content; the data type of the structured result is a key-value pair.

[0083] In this embodiment, step S500 is specifically to combine service content into a result through the service logic knowledge library.

[0084] In this embodiment, the service logic knowledge library is extracted by a trained relation extraction model, and the training method of the trained relation extraction model includes:

[0085] Step A100, obtain financial domain documents with manual annotations as the training set;

[0086] Step A200, extract the training set text logic rules from the training set through the relation extraction model; the relation extraction model is an MRC model;

[0087] Step A300, calculate the extraction result performance through the training set text logic, and adjust the model parameters by the stochastic gradient method until the extraction result performance reaches a set threshold and the loss function converges to obtain a trained relation extraction model;

[0088] The loss function adopts a conditional random field function:

[0089] Loss=-logP(y1,y2,......,y n 丨X)

[0090] Among them, X represents the input sample, y1 represents the output of the first path of the relation extraction model, y2 represents the output of the second path of the relation extraction model, y nRepresents the output of the nth path of the relation extraction model, n represents the total number of paths of the relation extraction model, and P represents the maximum probability.

[0091] The method of the second embodiment of the present invention comprises: a format conversion module, an element positioning module, a scheme extraction module, a business content output module and an event combination module;

[0092] The format conversion module is configured to convert the document to be processed into a hypertext format to obtain a hypertext to be processed;

[0093] The element positioning module is configured to obtain element positioning based on the hypertext to be processed;

[0094] The scheme extraction module extracts element content based on the keyword knowledge base, extraction scheme and element positioning;

[0095] The business content output module is configured to output business content based on the element content and in accordance with a business logic rule base;

[0096] The event combination module is configured to combine into a structured result based on the business content; the data type of the structured result is a key-value pair.

[0097] An electronic device according to the third embodiment of the present invention comprises: at least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned method for structuring fund contracts in the financial field.

[0098] A fourth embodiment of the present invention is a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned method for structuring fund contracts in the financial field.

[0099] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related instructions of the storage device and processing device described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0100] The terms "first", "second", etc. are used to distinguish similar objects rather than to describe or indicate a particular order or sequence.

[0101] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus / device.

[0102] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A method for structuring fund contracts in the financial field, characterized in that: The method comprises: Converting the document to be processed into a hypertext format to obtain a hypertext to be processed; Based on the hypertext to be processed, obtaining element location; Extract element content based on keyword knowledge base, extraction scheme and element positioning; Based on the element content, output business content according to the business logic rule library; Based on the business content, the data is combined into a structured result; the data type of the structured result is a key-value pair; The combining into a structured result based on the business content specifically comprises combining the business content into a structured result through a business logic knowledge base; The business logic knowledge base is extracted through a trained relationship extraction model, and the training method of the trained relationship extraction model includes: Obtain manually annotated financial documents as training sets; Extracting training set text logic rules from the training set through the relation extraction model; the relation extraction model is an MRC model; The performance of the extraction result is calculated by the text logic rules of the training set, and the model parameters are adjusted by the stochastic gradient method until the performance of the extraction result reaches the set threshold and the loss function converges, thereby obtaining a trained relationship extraction model; The loss function uses a conditional random field function: Loss=-logP(y1,y2,……,y n 丨X) Among them, X represents the input sample, y1 represents the output of the first path of the relationship extraction model, y2 represents the output of the second path of the relationship extraction model, and y n Represents the output of the nth path of the relation extraction model, n represents the total number of paths of the relation extraction model, and P represents the maximum probability.

2. The method for structuring fund contracts in the financial field according to claim 1, characterized in that: The step of converting the document to be processed into a hypertext format to obtain the hypertext to be processed is specifically as follows: The schema standard of the fund contract in the financial field is configured according to the logical knowledge base; the schema standard includes the possible chapters of the target element, the possible expression mode of the target element, the title, the unified and standardized title, the chapter number, the unified and standardized chapter number, the title number and the unified and standardized title number; wherein the unified and standardized title is the title of the chapter where the target element is located, which is obtained by mapping according to the chapter-standard chapter mapping relationship; Convert entities or objects that meet schema standards in the DOC, DOCX or XML document to be processed into hypertext format through Parser; Through OCR, entities or objects that meet the schema standard in the scanned document or image to be processed are converted into hypertext format.

3. The method for structuring fund contracts in the financial field according to claim 1, characterized in that: Based on the hypertext to be processed, the element location is obtained, specifically: Based on the preset element knowledge base, regular matching and similarity matching methods are used to determine whether the current sentence of the hypertext to be processed is the sentence where the target element is located. If it is the sentence where the target element is located, the target element location is obtained and the next sentence is set as the current sentence until all sentences of the hypertext to be processed are traversed.

4. The method for structuring fund contracts in the financial field according to claim 1, characterized in that: Based on the keyword knowledge base, extraction scheme and element positioning, extract element content, specifically: Recording the preset extraction scheme of each target element through a JSON file; the extraction scheme includes a digital extraction scheme, a text extraction scheme, a sentence-level extraction scheme, and a paragraph-level extraction scheme; Based on the keyword knowledge base, extraction scheme and element positioning, the element content is extracted through regular matching and deep learning-based entity recognition model; Specifically: Extracting element contents respectively through the first entity recognition model, the second entity recognition model and the third entity recognition model to obtain first element contents, second element contents and third element contents respectively; The two elements with the highest similarity are selected as the final elements by voting.

5. The method for structuring fund contracts in the financial field according to claim 1, characterized in that: After outputting the business content according to the business logic rule base based on the element scheme, the process also includes post-processing the symbols and noise in the business content.

6. A system for structuring fund contracts in the financial field, based on the method for structuring fund contracts in the financial field as described in any one of claims 1 to 5, characterized in that: The system includes: a format conversion module, an element positioning module, a solution extraction module, a business content output module and an event combination module; The format conversion module is configured to convert the document to be processed into a hypertext format to obtain a hypertext to be processed; The element positioning module is configured to obtain element positioning based on the hypertext to be processed; The scheme extraction module is configured to extract element content based on the keyword knowledge base, the extraction scheme and the element location; The business content output module is configured to output business content based on the element content and according to the business logic rule library; The event combination module is configured to combine the business content into a result; the data type of the structured result is a key-value pair.

7. An electronic device comprising: at least one processor; And a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the method for structuring fund contracts in the financial field as described in any one of claims 1-5.

8. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to be executed by the computer to implement the method for structuring fund contracts in the financial field as described in any one of claims 1 to 5.

Images