Financial activity information mining system and method based on multiple base large language models

Abstract

The invention provides a financial activity information mining system and method based on a plurality of base large language models, and relates to the technical field of big data processing, and the method comprises the steps: employing a plurality of base large language models to carry out entity recognition and mining on an input financial information text, and obtaining a plurality of financial information entities; performing entity relationship identification and mining on the financial information text and the financial information entities to obtain entity relationships among the financial information entities; and arranging the financial information text, the financial information entities corresponding to the financial information text and the corresponding entity relationships into a chart according to the corresponding relationships, and displaying the chart. The method has the advantages that multiple base large language models are adopted for entity recognition and mining and relation extraction between entities, the recognition efficiency can be improved, the financial activity service accuracy requirement is met, and the problem that in the prior art, analysis of financial activity entities and incidence relations often depends on a manual mode, and the analysis efficiency is poor is solved. The efficiency is low; and important information is easy to miss.

Description

Technical Field

[0001] The present invention relates to the field of big data processing technology, and in particular to a financial activity information mining system and method based on multiple base large language models. Background Art

[0002] With the rapid development of my country's financial markets and information technology, various new financial activities are gradually increasing, placing higher demands on financial participants and regulatory agencies. Traditional methods for mining and analyzing financial activity information often rely on manual analysis, which is inefficient and prone to missing important information. Therefore, it is particularly important to develop a method that can improve the efficiency of identifying entities and relationships in financial activities while meeting the accuracy requirements of financial activity operations. Summary of the Invention

[0003] In response to the problems existing in the prior art, the present invention provides a financial activity information mining system based on multiple base large language models, comprising:

[0004] The entity recognition module is used to use multiple base large language models to perform entity recognition and mining on the input financial information text to obtain multiple financial information entities;

[0005] a relationship extraction module, connected to the entity recognition module, for identifying and mining entity relationships between the financial information text and the financial information entities, and obtaining entity relationships between the financial information entities;

[0006] The result output module is connected to the relationship extraction module and the entity recognition module, and is used to organize the financial information text and its corresponding financial information entities, as well as the corresponding entity relationships, into a chart for display according to the corresponding relationship.

[0007] Preferably, a text preprocessing module is further included, connected to the entity recognition module, including:

[0008] The text segmentation unit is used to segment the input original text into multiple segments according to semantic paragraphs or fixed lengths;

[0009] Text preprocessing, connected to the text segmentation unit, is used to perform denoising, cleaning, word segmentation and part-of-speech tagging preprocessing on each of the segmented texts, and then input the original text and each of the preprocessed segmented texts as the financial information text into the entity recognition module.

[0010] Preferably, the entity recognition module includes:

[0011] A first storage unit is used to store an entity prompt word engineering template, wherein the entity prompt word engineering template stores a plurality of template entity names;

[0012] The entity recognition unit is connected to the first storage unit and is used to input the financial information text and the entity prompt word engineering template into each of the base large language models to obtain multiple financial information entities corresponding to the template entity name.

[0013] Preferably, the entity recognition module further includes:

[0014] a first cross-validation unit, connected to the entity recognition unit, configured to cross-validate all the financial information entities obtained by the base large language models, retain the same financial information entities output by multiple base large language models, and repeatedly recognize the financial information texts corresponding to different financial information entities output by multiple base large language models before cross-validating again;

[0015] A first comprehensive scoring unit is connected to the first cross-validation unit and the first storage unit, and is used to score each of the financial information entities based on the results of the cross-validation, and then request manual verification and supplementation for each of the financial information entities whose score is below a preset threshold, and for the financial information text for which the corresponding financial information entity is not identified, and save the received supplementary content into the entity prompt word engineering template.

[0016] Preferably, the relationship extraction module includes:

[0017] A second storage unit is used to store a relationship prompt word engineering template, wherein the relationship prompt word engineering template stores a plurality of template entity relationships;

[0018] A relationship identification unit, connected to the second storage unit, is used to input the financial information text, the financial information entity, and the relationship prompt word engineering template into each of the base large language models to obtain an entity relationship between each of the financial information entities corresponding to the template entity relationship.

[0019] Preferably, the relationship identification module further includes:

[0020] a second cross-validation unit, connected to the relationship identification unit, configured to cross-validate the entity relationships of all the financial information entities obtained by the base large language models, retain the entity relationships of the same financial information entities output by multiple base large language models, and repeatedly identify the financial information texts and the financial information entities corresponding to the entity relationships of different financial information entities output by multiple base large language models, and then cross-validate again;

[0021] A second comprehensive scoring unit is connected to the second cross-validation unit and the second storage unit, and is used to score each of the financial information entities based on the results of the cross-validation, and then manually verify and supplement the entity relationships of each of the financial information entities that are below a preset scoring threshold, as well as the financial information texts and the financial information entities for which the corresponding entity relationships are not identified, and save the received supplementary content into the relationship prompt word engineering template.

[0022] Preferably, the result output module includes:

[0023] A visualization unit is used to correspond the financial information text with the financial information entity, correspond the financial information entity and the entity relationship between the financial information entity, and then sequentially associate all the financial information texts, the financial information entities and the entity relationship between the financial information entities according to the corresponding relationship and generate a visualization chart in a predetermined format.

[0024] Preferably, the predetermined format is one of a resource description framework, an extensible markup language, a JavaScript object notation, a character separated value, a graph database, and a relational database.

[0025] The present invention also provides a financial activity information mining method based on multiple base large language models, which is applied to the above-mentioned financial activity information mining system, comprising:

[0026] Step S1: The financial activity information mining system uses multiple base large language models to perform entity recognition and mining on the input financial information text to obtain multiple financial information entities;

[0027] Step S2: the financial activity information mining system identifies and mines entity relationships between the financial information text and the financial information entities to obtain entity relationships between the financial information entities;

[0028] In step S3, the financial activity information mining system organizes the financial information text and the corresponding financial information entities and the corresponding entity relationships into a chart for display according to the corresponding relationships.

[0029] Preferably, before executing step S1, a text preprocessing process is also included, including:

[0030] The financial activity information mining system divides the input original text into multiple segmented texts according to semantic paragraphs or fixed lengths, and then performs denoising, cleaning, word segmentation and part-of-speech tagging preprocessing on each segmented text, and then inputs the original text and each preprocessed segmented text as the financial information text into the entity recognition module.

[0031] The above technical solution has the following advantages or beneficial effects: it uses multiple base large language models to perform entity recognition and mining on the input financial information text and extract the relationship between entities, which can improve the efficiency of identifying entities and related relationships in financial activities, and meet the accuracy requirements of financial activity business, avoiding the problem that the analysis of financial activity entities and related relationships in traditional technologies often relies on manual methods, which is inefficient and easy to miss important information. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a schematic diagram of the results of a financial activity information mining system based on multiple base large language models in a preferred embodiment of the present invention;

[0033] Figure 2 The figure is a flow chart of a method for mining financial activity information based on multiple base large language models in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0034] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. The present invention is not limited to this embodiment, and other embodiments may also fall within the scope of the present invention as long as they conform to the gist of the present invention.

[0035] In a preferred embodiment of the present invention, based on the above problems existing in the prior art, a financial activity information mining system based on multiple base large language models is provided. Figure 1 Shown, including:

[0036] Entity recognition module 1, used to use multiple base large language models to perform entity recognition and mining on the input financial information text to obtain multiple financial information entities;

[0037] Relationship extraction module 2, connected to entity recognition module 2, is used to identify and mine entity relationships between financial information text and financial information entities, and obtain entity relationships between various financial information entities;

[0038] The result output module 3, the connection relationship extraction module 2 and the entity recognition module 1 are used to organize the financial information text and its corresponding financial information entities, as well as the corresponding entity relationships into charts for display according to the corresponding relationships.

[0039] Specifically, Large Language Model (LLM, such as GPT-4, PaLM, Claude, etc.) is an artificial intelligence technology based on deep learning, mainly used for natural language processing (NLP) tasks.

[0040] In the present invention, multiple base large language models are used to perform entity recognition and mining on the input financial information text and to extract the relationships between entities. This can improve the efficiency of identifying entities and related relationships in financial activities, and meet the accuracy requirements of financial activity business. It avoids the problem that the analysis of financial activity entities and related relationships in traditional technologies often relies on manual methods, which is inefficient and prone to missing important information.

[0041] In a preferred embodiment of the present invention, a text preprocessing module 4 is further included, connected to the entity recognition module 1, including:

[0042] The text segmentation unit 41 is used to segment the input original text into multiple segmented texts according to semantic paragraphs or fixed lengths;

[0043] The text preprocessing unit 42 is connected to the text segmentation unit 41 and is used to perform denoising, cleaning, word segmentation and part-of-speech tagging preprocessing on each segmented text, and then input the original text and each preprocessed segmented text as financial information text into the entity recognition module.

[0044] Specifically, in this embodiment, the text segmentation unit segments the text according to semantic paragraphs or fixed lengths to ensure that the data blocks of the input model meet its context window limit (such as 4096 tokens).

[0045] In the text preprocessing unit, denoising and cleaning uses regular expressions and a rule engine to remove irrelevant characters (such as HTML tags and special symbols), retaining the pure text content. Word segmentation and part-of-speech tagging use financial-specific dictionaries (such as the FinBERT vocabulary) for word segmentation, combined with part-of-speech tagging (such as "noun - institution" and "verb - transaction") to assist in subsequent entity classification.

[0046] The beneficial effects of the text preprocessing module are as follows:

[0047] 1. Input standardization: Segmentation and cleaning reduce model parsing errors caused by format confusion.

[0048] 2. Domain adaptability: Financial-specific word segmentation improves the accuracy of entity recognition (such as distinguishing "Apple Inc." from "Apple futures").

[0049] In a preferred embodiment of the present invention, the entity recognition module 1 is as follows Figure 1 Shown, including:

[0050] The first storage unit 11 is used to store an entity prompt word engineering template, wherein the entity prompt word engineering template stores a plurality of template entity names;

[0051] The entity recognition unit 12 is connected to the first storage unit 11 and is used to input the financial information text and the entity prompt word engineering template into each base large language model to obtain multiple financial information entities corresponding to the template entity name.

[0052] In a preferred embodiment of the present invention, the entity recognition module 1 is as follows Figure 1 As shown, it also includes:

[0053] A first cross-validation unit 13, connected to the entity recognition unit 12, is configured to cross-validate all financial information entities obtained by each base large language model, retain the same financial information entities output by multiple base large language models, and repeatedly recognize the financial information texts corresponding to different financial information entities output by multiple base large language models before cross-validating again;

[0054] The first comprehensive scoring unit 14 is connected to the first cross-validation unit 13 and the first storage unit 11, and is used to score each financial information entity based on the cross-validation results, and then request manual verification and supplementation for each financial information entity whose score is below a preset threshold, as well as for the financial information text for which the corresponding financial information entity is not identified, and save the received supplementary content into the entity prompt word engineering template.

[0055] Specifically, the Large Language Model (LLM) is a deep learning-based AI technology primarily used for Natural Language Processing (NLP) tasks. While LLMs have made significant progress in NLP, they generally employ black-box models that do not fully represent the intermediate computational steps involved in the output. Consequently, they can lead to problems such as machine hallucinations due to training data bias, logical errors, and fabricated plots. This has hampered their use in applications such as finance and law, where accuracy is crucial.

[0056] Human-in-the-loop (HITL) is an approach that integrates human oversight and intervention into automated systems. In this model, humans and machines collaborate, with machines performing some tasks and humans overseeing, verifying, and correcting the machine's decisions. This approach emphasizes maintaining human involvement in the automated decision-making process to improve system accuracy and reliability. This approach, used in AI-based automated systems, can effectively improve accuracy, reliability, flexibility, and explainability. However, this approach also faces challenges such as increased labor costs, slow response times, and high system integration complexity.

[0057] In order to address the technical deficiencies of the above two technologies, in this embodiment, multiple base large language models are integrated and used (the base large language model refers to the original large language model that has not been optimized for the domain and aligned with human preferences. Its core positioning is to serve as a general infrastructure for artificial intelligence development. It forms basic language capabilities by pre-training on massive unlabeled texts (such as public data on the Internet). It usually has a parameter scale of hundreds of billions to carry the potential for complex language pattern understanding and generation. The core value of this type of model lies in its "base" attribute - developers can conduct in-depth customized development based on its open technical framework (such as open source model weights or API interfaces). For example, they can use private domain data for fine-tuning to build professional models for vertical industries such as medical care and finance, or integrate enterprise knowledge bases and business system plug-ins to form closed-loop applications.), and use the "human in the loop" multi-round dialogue intent recognition mode to reduce the "machine hallucination" of the large language model, and ultimately use it to improve the accuracy and efficiency of information entity and relationship mining.

[0058] Large model of the base

[0059] The present invention is applicable to entities involved in financial activities and regulatory agencies, helping them to quickly identify key entity information and the relationships between entities in massive amounts of financial information text data, thereby improving work efficiency and accuracy.

[0060] More specifically, in this embodiment, the module integrates multiple large language models (typically including Meta's Llama series and Baichuan Intelligence's Baichuan Basic Edition) and combines them with entity prompt word engineering templates for entity recognition.

[0061] The first storage unit stores preset template entity names (such as "financial institution," "financial product," etc.), providing structured prompts for the model, narrowing the recognition scope and improving the efficiency of targeted entity mining. For example, the specific process involves processing a press release about "XX Bank launching a new digital currency wealth management product." The first storage unit pre-stores entity templates in the financial field, including categories such as "financial institution," "financial product," and "laws and regulations," with specific entity names such as "bank," "fund," and "securities law." The system extracts keywords from the "financial institution" category (such as "bank," "securities company") and the "financial product" category (such as "wealth management product," "digital currency") as prompts for the base large language model. Based on the "bank" and "wealth management product" prompts in the template, the model focuses on "XX Bank" and "digital currency wealth management product" in the text for recognition.

[0062] The entity recognition unit inputs the preprocessed text and template entity names into multiple large language models, leveraging the models' deep understanding of semantics to extract financial entities. For example, consider the input text: "Securities Company A and Fund B jointly issue a 'stable and profitable growth' bond fund, subject to the Securities Investment Fund Law." This processing involves parallel processing using multiple models: Model A (e.g., GPT-4) outputs the entities "Securities Company A," "Fund B," "stable and profitable growth," "bond fund," and "Securities Investment Fund Law." Model B (e.g., PaLM) outputs the entities "Securities Company A," "Fund B," "stable and profitable growth fund," and "Securities Law." Initial recognition of entities such as "Securities Company A," "Fund B," "stable and profitable growth," and "Securities Investment Fund Law" occurs, but there are differences between the models (e.g., "bond fund" versus "stable and profitable growth fund"). The entity candidate list contains the outputs from multiple models for subsequent cross-validation processing.

[0063] The first cross-validation unit: Cross-compares the output results of different models, retaining only entities that are consistently recognized by multiple models, reducing the risk of "machine hallucination" by a single model. For example, for the text input in the embodiment of the entity recognition unit, the output of the three base large language models is as follows:

[0064] Model A: Securities Company A, Fund B, Stable Profit Growth, Bond Fund, Securities Investment Fund Law Model B: Securities Company A, Fund B, Stable Profit Growth Fund, Securities Law

[0065] Model C: Securities Company A, Fund B, and Stable Growth Bond Fund

[0066] This is followed by a verification process, which includes:

[0067] 1. Consistency comparison:

[0068] All models identified "Securities Company A" and "Fund B".

[0069] There are naming differences between "Stable Profit Growth" and "Stable Profit Growth Fund" and "Stable Profit Growth Bond Fund".

[0070] The Securities Investment Fund Law only recognizes Model A, and the Securities Law only recognizes Model B.

[0071] 2. Validation rules:

[0072] Retain consistent entities: "A Securities Company" and "B Fund".

[0073] Marked disputed entities: "Stable Profit Growth", "Securities Investment Fund Law", etc. require further processing.

[0074] 3. Result: Only "Securities Company A" and "Fund B" are retained directly, while the remaining entities are transferred to the manual verification process.

[0075] The first comprehensive scoring unit assesses entity confidence based on cross-validation results. Low-scoring entities trigger manual verification, and manually supplemented entities are updated to the template, forming a closed-loop optimization process. The first comprehensive scoring unit verifies the results of the first cross-validation unit through multiple rounds of human-in-the-loop dialogue, using a local large language model prompt word engineering template and combining it with local algorithms. Simultaneously with human-in-the-loop intelligent analysis and optimization, the content of the local large language model prompt word engineering template is adjusted and improved, thereby improving the accuracy and efficiency of mining financial activity information entities and association relationships.

[0076] The specific process is, for example: scoring and processing the controversial entities "stable profit growth" and "Securities Investment Fund Law" output by the first cross-validation unit.

[0077] Scoring and manual intervention are then carried out. The specific processing process includes:

[0078] 1. Scoring rules:

[0079] The more times an entity is recognized by the model, the higher the score.

[0080] “Stable profit growth” is recognized by models A and C (score 80 / 100), but the naming is inconsistent.

[0081] The Securities Investment Funds Act is only recognized by Model A (score 40 / 100).

[0082] 2. Human intervention:

[0083] The low-scoring entity "Securities Investment Fund Law" triggered manual review. Through a human-in-the-loop approach, the intelligent analysis results were manually reviewed and revised to improve their accuracy and completeness. The manually identified information was used to dynamically adjust and improve the local large language model prompt engineering template. The reviewer confirmed that the text explicitly mentioned the "Securities Investment Fund Law" and added it to the entity list.

[0084] After manual verification, "Stable Profit Growth" was corrected to the full name "Stable Profit Growth Bond Fund" and updated to the entity prompt word engineering template.

[0085] 3. Results:

[0086] Final entity list: Securities Company A, Fund B, Stable and Profitable Growth Bond Fund, and Securities Investment Fund Law.

[0087] Template update: Added "Stable and Profitable Growth Bond Fund" to the "Financial Products" category to improve subsequent recognition accuracy.

[0088] The above embodiments demonstrate that the first storage unit uses template-directed model guidance to reduce interference from irrelevant entities. The entity recognition unit leverages multi-model collaboration to expand entity coverage. The cross-validation unit filters noise through consistency checks, improving result reliability. The comprehensive scoring unit dynamically optimizes templates using manual loopback, enabling system self-learning capabilities. This module significantly improves the accuracy and adaptability of financial entity recognition through a closed-loop process of "template guidance → multi-model recognition → cross-validation → manual calibration."

[0089] The beneficial effects of the entity recognition module are:

[0090] 1. Improved accuracy: Multi-model cross-validation significantly reduces the false positive rate, and the manual loopback mechanism further corrects bias.

[0091] 2. Enhanced adaptability: The dynamic template update mechanism enables the system to adapt to emerging entities in the financial field (such as new financial products).

[0092] 3. Efficiency optimization: Use prompt word engineering templates to guide the model in a targeted manner, reduce invalid calculations, and shorten entity recognition time.

[0093] In a preferred embodiment of the present invention, the relationship extraction module 2 is as follows: Figure 1 As shown, including:

[0094] The second storage unit 21 is used to store a relationship prompt word engineering template, in which a plurality of template entity relationships are stored;

[0095] The relationship identification unit 22 is connected to the second storage unit 21 and is used to input the financial information text, financial information entities and relationship prompt word engineering templates into each base large language model to obtain the entity relationship between each financial information entity corresponding to the template entity relationship.

[0096] In a preferred embodiment of the present invention, the relationship identification module 2 is as follows Figure 1 As shown, it also includes:

[0097] The second cross-validation unit 23 and the connection relationship identification unit 22 are used to cross-validate the entity relationships of all financial information entities obtained by each base large language model, retain the entity relationships of the same financial information entities output by multiple base large language models, and repeatedly identify the financial information texts and financial information entities corresponding to the entity relationships of different financial information entities output by multiple base large language models and then cross-validate again;

[0098] The second comprehensive scoring unit 24 is connected to the second cross-validation unit 23 and the second storage unit 21, and is used to score each financial information entity based on the cross-validation results, and then manually verify and supplement the entity relationships of each financial information entity that are below a preset scoring threshold, as well as the financial information text and financial information entity requests for which the corresponding entity relationships are not identified, and save the received supplementary content into the relationship prompt word engineering template.

[0099] Specifically, in this embodiment, based on entity recognition, this module drives multi-model collaborative mining of entity relationships through relationship prompt word engineering templates (such as "investment relationship" and "creditor's rights and liabilities").

[0100] The second storage unit stores predefined financial relationship templates, providing a semantic framework for relationship extraction. For example, consider analyzing a paragraph from a listed company's annual report: "XX Group, through its subsidiary, invested 500 million yuan in YY Technology, acquiring a 30% stake and signing a strategic cooperation agreement."

[0101] Then the relationship extraction is performed, and the process includes:

[0102] 1. Template call: The second storage unit pre-stores financial relationship templates, including categories such as "investment relationship", "equity ratio", and "cooperation agreement", and specific relationships such as "investment-amount", "controlling-ratio", and "cooperation-type".

[0103] 2. Template matching: The system extracts prompt words related to "investment relationship" (such as "capital injection", "shareholding", and "agreement") to guide the model to focus on the flow of funds and cooperation agreements in the text.

[0104] 3. Results: Based on the keywords in the template, the model targetedly identified the "investment relationship" and "strategic cooperation" relationship between "XX Group → YY Technology".

[0105] The relationship recognition unit inputs entities and text into multiple large language models, combines them with templates to generate candidate relationships, and uses the model's contextual reasoning capabilities to capture implicit associations. For example, consider the input text: "Bank A provides a 2 billion yuan loan to Real Estate Company B, with the collateral being the assets of Shopping Center C held by Company B."

[0106] The relationship identification process is then carried out, including:

[0107] 1. Multi-model parallel processing:

[0108] Model A (such as Meta Llama 3): Outputs the relationships "Bank A - Loan - Real Estate Company B (Amount: 2 billion)" and "Real Estate Company B - Mortgage - Shopping Center C".

[0109] Model B (such as Baichuan 2): Output relations "Bank A - Loan - Real Estate Company B (2 billion yuan)" and "Real Estate Company B - Asset Mortgage - Shopping Center C".

[0110] Model C (such as the Qwen 1.5 series): Output relationships "Bank A - Loan - Real Estate Company B (2 billion)" and "Real Estate Company B - Collateral - Shopping Center C".

[0111] 2. Result integration: Two types of relationships, "loan" and "mortgage", were initially extracted, but there were differences in their expressions (such as "loan" and "lending" and "borrowing").

[0112] 3. Results: The relationship candidate list contains similar semantic relationships output by multiple models for subsequent cross-validation unit processing.

[0113] The second cross-validation unit: Verify the consistency of the relationships output by multiple models, retain relationships with high repetition rates, and filter out noisy data. For example, for the entity relationships identified by the relationship recognition unit, the outputs of the three models are as follows:

[0114] Model A: Loan (Bank A → Real Estate Company B, 2 billion), Mortgage (Real Estate Company B → Shopping Center C)

[0115] Model B: Lending (Bank A → Real Estate Company B, RMB 2 billion), Asset Mortgage (Real Estate Company B → Shopping Center C)

[0116] Model C: Loan (Bank A → Real Estate Company B, 2 billion), Collateral (Real Estate Company B → Shopping Center C)

[0117] Cross-validation is then performed, and the specific process includes:

[0118] 1. Semantic alignment:

[0119] "Loan", "lending" and "borrowing" are uniformly mapped to "creditor-debtor relationship", with the subtype being "loan".

[0120] "Mortgage", "Asset Mortgage" and "Collateral" are uniformly mapped to "Guarantee Relationship", with the subtype being "Asset Mortgage".

[0121] 2. Consistency rules:

[0122] Preserving consistent relations: All models identify the semantic core of “loan” and “mortgage”.

[0123] Marking difference details: The amount units "100 million" and "100 million yuan" need to be standardized, and "collateral" needs to have additional attributes.

[0124] 3. Result: After standardization, the relationships are "Bank A - Loan - Real Estate Company B (2 billion yuan)" and "Real Estate Company B - Asset Mortgage - Shopping Center C".

[0125] The second comprehensive scoring unit initiates manual review of low-confidence relationships and feeds the manual correction results back to the template library to enhance the coverage of relationship templates. For example, the scoring and processing of the disputed relationship "Bank A - Loan - Real Estate Company B" after cross-validation. The processing process includes:

[0126] 1. Scoring rules:

[0127] The more times a relationship is recognized by the model, the higher the score.

[0128] "Loan" is recognized by models A and B (score 90 / 100); "Borrow" is only used by model C (score 60 / 100).

[0129] 2. Human intervention:

[0130] The low-scoring relationship "Loan" triggered manual review. The reviewer confirmed that the word "loan" was clearly used in the text, revised the relationship to "Loan," and added the attribute "Type: Long-term loan."

[0131] Remove "borrowing" as a redundant relationship, and add synonyms of "loan" such as "lending" and "borrowing" to the relationship template to improve the semantic mapping rules.

[0132] 3. Results:

[0133] The final relationship list: Bank A - Loan - Real Estate Company B (2 billion yuan, type: long-term loan), Real Estate Company B - Asset Mortgage - Shopping Center C.

[0134] Template update: Added synonym mapping under the "Credit-Debtor Relationship" category to improve the semantic generalization capability of subsequent relationship extraction.

[0135] From the above embodiments, it can be seen that the second storage unit limits the relationship type through templates to reduce the noise caused by the free play of the model. The relationship identification unit uses the complementarity of multiple models to capture diverse relationship expressions. The cross-validation unit solves the problem of expression differences through semantic alignment and standardization. The comprehensive scoring unit combines manual correction with dynamic optimization templates to achieve accurate expression of relationship logic. This module significantly improves the reliability and interpretability of financial relationship mining through the closed-loop process of "template guidance → multi-model extraction → semantic alignment → manual calibration". The beneficial effects achieved include:

[0136] 1. Complex relationship mining capabilities: Multi-model collaboration can identify non-explicit relationships such as capital flows and legal disputes.

[0137] 2. Dynamic optimization mechanism: Continuous updating of the template library enables the system to adapt to changes in financial regulatory policies.

[0138] 3. Enhanced anti-interference ability: Cross-validation effectively avoids incorrect associations caused by deviations in model training data.

[0139] In general, the main technical effects of the present invention include the following three aspects:

[0140] 1. A method combining multiple large language models with human-in-the-loop (HIL) is used to improve the accuracy of identifying and mining information entities and relationships. Specifically, based on multiple large language models, entity recognition and mining are performed on preprocessed text. The recognition results are then cross-matched and verified using different large language models, improving the accuracy of identifying and mining information entities and entity relationships.

[0141] 2. Human-in-the-loop intelligent analysis and optimization improves the accuracy of information entity recognition extracted from financial information text data. Specifically, a method combining multiple base large language models with local large language model prompt word engineering templates improves the efficiency of financial information text processing. Through multiple rounds of dialogue and local large language model prompt word engineering templates, it reduces the "machine hallucination" of the large language model and improves the accuracy of information entity recognition and mining from financial information text data.

[0142] 3. Human-in-the-loop intelligent analysis and optimization improves the accuracy of identifying information entity relationships extracted from financial information text data. Specifically, this method combines multiple base large language models with local large language model prompt word engineering templates. While improving the efficiency of financial information text processing, it also reduces the "machine hallucination" of the large language model through multiple rounds of dialogue and local large language model prompt word engineering templates, thereby improving the accuracy of information entity relationship identification and mining from financial information text data.

[0143] In a preferred embodiment of the present invention, the result output module 3 includes:

[0144] The visualization unit 31 is used to correspond financial information texts with financial information entities, correspond financial information entities with the entity relationships between financial information entities, and then sequentially associate all financial information texts, financial information entities and the entity relationships between financial information entities according to the corresponding relationships and generate a visualization chart in a predetermined format.

[0145] In a preferred embodiment of the present invention, the predetermined format is one of a resource description framework, an extensible markup language, a JavaScript object notation, a character separated value, a graph database, and a relational database.

[0146] Specifically, the visualization unit in this embodiment uses a graph database (such as Neo4j) to construct an entity-relationship network diagram, supporting dynamic interactions (such as node expansion and path tracing). For example:

[0147] The input text is the annual report of a listed company. The system has identified the entities "XX Company", "Chairman Zhang", "Subsidiary YY Technology", "Investment Agreement", and the relationships "Control" and "Signature". After entity identification and relationship identification, the following are obtained:

[0148] Entity List: XX Company (Enterprise), Zhang (Person), YY Technology (Enterprise), Investment Agreement (Document).

[0149] Relationship list: XX Company → Holding → YY Technology, Zhang → Signed → Investment Agreement, XX Company → Association → Investment Agreement.

[0150] Using the Neo4j graph database, nodes (entities) and edges (relationships) were created to generate an interactive network diagram. Users can click on a node to expand detailed information (such as Zhang's position and shareholding percentage). Key relationships are highlighted (for example, "controlling" is indicated by a thick red edge). Ultimately, users can visually see the capital chain formed by "XX Company" through its controlling stake in "YY Technology" and Zhang's central role in the signing of the agreement.

[0151] Furthermore, users need to import entity and relationship data into an external risk analysis system, which only supports JSON input. Therefore, format conversion is performed to support structured formats such as RDF and JSON. The types of format conversion include but are not limited to Resource Description Framework (RDF), Extensible Markup Language (XML), JavaScript Object Notation (JSON), Character Separated Values ​​(CSV), graph databases, relational databases, etc., facilitating seamless integration with external systems (such as regulatory databases).

[0152] The beneficial effects of visual output by the result output module are as follows:

[0153] 1. Enhanced decision support: Visual charts help users quickly locate key entities (such as high-risk transaction entities).

[0154] 2. System compatibility expansion: Multi-format output meets the needs of different downstream analysis tools (such as JSON for API calls and CSV for report generation).

[0155] The present invention also provides a financial activity information mining method based on multiple base large language models, which is applied to the above-mentioned financial activity information mining system, such as Figure 2 As shown, including:

[0156] Step S1: The financial activity information mining system uses multiple base large language models to perform entity recognition and mining on the input financial information text to obtain multiple financial information entities;

[0157] Step S2: The financial activity information mining system identifies and mines entity relationships between financial information texts and financial information entities to obtain entity relationships between various financial information entities.

[0158] In step S3, the financial activity information mining system organizes the financial information text and its corresponding financial information entities, as well as the corresponding entity relationships, into a chart for display according to the corresponding relationships.

[0159] In a preferred embodiment of the present invention, a text preprocessing process is further included before executing step S1, including:

[0160] The financial activity information mining system divides the input original text into multiple segmented texts according to semantic paragraphs or fixed lengths, and then performs denoising, cleaning, word segmentation and part-of-speech tagging preprocessing on each segmented text. The original text and the preprocessed segmented texts are then input into the entity recognition module as financial information texts.

[0161] The above are only preferred embodiments of the present invention and do not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of this specification and illustrations should be included in the protection scope of the present invention.

Claims

1. A financial activity information mining system based on multiple base large language models, characterized by: include: The entity recognition module is used to use multiple base large language models to perform entity recognition and mining on the input financial information text to obtain multiple financial information entities; a relationship extraction module, connected to the entity recognition module, for identifying and mining entity relationships between the financial information text and the financial information entities, and obtaining entity relationships between the financial information entities; The result output module is connected to the relationship extraction module and the entity recognition module, and is used to organize the financial information text and its corresponding financial information entities, as well as the corresponding entity relationships, into a chart for display according to the corresponding relationship.

2. The financial activity information mining system according to claim 1, characterized in that: It also includes a text preprocessing module, connected to the entity recognition module, including: The text segmentation unit is used to segment the input original text into multiple segments according to semantic paragraphs or fixed lengths; A text preprocessing unit is connected to the text segmentation unit and is used to perform denoising, cleaning, word segmentation and part-of-speech tagging preprocessing on each of the segmented texts, and then input the original text and each of the preprocessed segmented texts as the financial information text into the entity recognition module.

3. The financial activity information mining system according to claim 1, characterized in that: The entity recognition module includes: A first storage unit is used to store an entity prompt word engineering template, wherein the entity prompt word engineering template stores a plurality of template entity names; The entity recognition unit is connected to the first storage unit and is used to input the financial information text and the entity prompt word engineering template into each of the base large language models to obtain multiple financial information entities corresponding to the template entity name.

4. The financial activity information mining system according to claim 3, characterized in that: The entity recognition module further includes: a first cross-validation unit, connected to the entity recognition unit, configured to cross-validate all the financial information entities obtained by the base large language models, retain the same financial information entities output by multiple base large language models, and repeatedly recognize the financial information texts corresponding to different financial information entities output by multiple base large language models before cross-validating again; A first comprehensive scoring unit is connected to the first cross-validation unit and the first storage unit, and is used to score each of the financial information entities based on the results of the cross-validation, and then request manual verification and supplementation for each of the financial information entities whose score is below a preset threshold, and for the financial information text for which the corresponding financial information entity is not identified, and save the received supplementary content into the entity prompt word engineering template.

5. The financial activity information mining system according to claim 1, characterized in that: The relationship extraction module includes: A second storage unit is used to store a relationship prompt word engineering template, wherein the relationship prompt word engineering template stores a plurality of template entity relationships; A relationship identification unit, connected to the second storage unit, is used to input the financial information text, the financial information entity, and the relationship prompt word engineering template into each of the base large language models to obtain an entity relationship between each of the financial information entities corresponding to the template entity relationship.

6. The financial activity information mining system according to claim 5, characterized in that: The relationship identification module also includes: a second cross-validation unit, connected to the relationship identification unit, configured to cross-validate the entity relationships of all the financial information entities obtained by the base large language models, retain the entity relationships of the same financial information entities output by multiple base large language models, and repeatedly identify the financial information texts and the financial information entities corresponding to the entity relationships of different financial information entities output by multiple base large language models, and then cross-validate again; A second comprehensive scoring unit is connected to the second cross-validation unit and the second storage unit, and is used to score each of the financial information entities based on the results of the cross-validation, and then manually verify and supplement the entity relationships of each of the financial information entities that are below a preset scoring threshold, as well as the financial information texts and the financial information entities for which the corresponding entity relationships are not identified, and save the received supplementary content into the relationship prompt word engineering template.

7. The financial activity information mining system according to claim 1, characterized in that: The result output module includes: A visualization unit is used to correspond the financial information text with the financial information entity, correspond the financial information entity and the entity relationship between the financial information entity, and then sequentially associate all the financial information texts, the financial information entities and the entity relationship between the financial information entities according to the corresponding relationship and generate a visualization chart in a predetermined format.

8. The financial activity information mining system according to claim 7, characterized in that: The predetermined format is one of a resource description framework, an extensible markup language, a JavaScript object notation, a character separated value, a graph database, and a relational database.

9. A method for mining financial activity information based on multiple base large language models, characterized in that: A financial activity information mining system according to any one of claims 1 to 8, comprising: Step S1: The financial activity information mining system uses multiple base large language models to perform entity recognition and mining on the input financial information text to obtain multiple financial information entities; Step S2: the financial activity information mining system identifies and mines entity relationships between the financial information text and the financial information entities to obtain entity relationships between the financial information entities; In step S3, the financial activity information mining system organizes the financial information text and the corresponding financial information entities and the corresponding entity relationships into a chart for display according to the corresponding relationships.

10. The financial activity information mining method according to claim 9, characterized in that: Before executing step S1, a text preprocessing process is also included, including: The financial activity information mining system divides the input original text into multiple segmented texts according to semantic paragraphs or fixed lengths, and then performs denoising, cleaning, word segmentation and part-of-speech tagging preprocessing on each segmented text, and then inputs the original text and each preprocessed segmented text as the financial information text into the entity recognition module.

Images