Financial activity information mining system and method based on multiple base large language models

Abstract

The application provides a financial activity information mining system and method based on multiple base large language models, relates to the technical field of big data processing, and comprises the following steps: adopting multiple base large language models to perform entity recognition and mining on input financial information text to obtain multiple financial information entities; performing entity relationship recognition and mining on the financial information text and the financial information entities to obtain entity relationships between the financial information entities; and arranging the financial information text, the corresponding financial information entities and the corresponding entity relationships according to the corresponding relationship into a chart for display. The beneficial effects are as follows: the multiple base large language models are adopted to perform entity recognition and mining and extract relationships between entities, which can improve the recognition efficiency and meet the accuracy requirements of financial activity business, thereby avoiding the problems in the prior art that the analysis of financial activity entities and associated relationships often depends on manual methods, which is low in efficiency and easy to miss important information.

Description

Technical Field

[0001] This invention relates to the field of big data processing technology, and in particular to a financial activity information mining system and method based on multiple base language models. Background Technology

[0002] With the rapid development of my country's financial market and information technology, various new types of financial activities are gradually increasing, which places higher demands on financial activity participants and regulatory agencies. In traditional financial activity information mining and analysis, the analysis of financial activity entities and their relationships often relies on manual methods, which are inefficient and prone to missing important information. Therefore, developing a method that can improve the efficiency of identifying entities and their relationships in financial activities while meeting the accuracy requirements of financial activity business is particularly important. Summary of the Invention

[0003] To address the problems existing in the prior art, this invention provides a financial activity information mining system based on multiple base-based large language models, comprising:

[0004] The entity recognition module is used to perform entity recognition and mining on the input financial information text using multiple base-based large language models to obtain multiple financial information entities;

[0005] The relationship extraction module, connected to the entity recognition module, is used to identify and mine entity relationships between the financial information text and the financial information entities to obtain the entity relationships between each of the financial information entities.

[0006] The result output module, which connects the relationship extraction module and the entity recognition module, is used to organize the financial information text and its corresponding financial information entities, as well as the corresponding entity relationships, into a chart for display.

[0007] Preferably, it further includes a text preprocessing module connected to the entity recognition module, comprising:

[0008] Text segmentation unit is used to divide the input raw text into multiple segments according to semantic paragraphs or fixed lengths;

[0009] Text preprocessing, connected to the text segmentation unit, is used to perform noise removal, word segmentation, and part-of-speech tagging preprocessing on each segmented text. Then, the original text and the preprocessed segmented text are input into the entity recognition module as the financial information text.

[0010] Preferably, the entity recognition module includes:

[0011] The first storage unit is used to store entity prompt word project templates, wherein the entity prompt word project templates store multiple template entity names;

[0012] An entity recognition unit, connected to the first storage unit, is used to input the financial information text and the entity prompt word engineering template into each of the base large language models to obtain multiple financial information entities corresponding to the template entity names.

[0013] Preferably, the entity recognition module further includes:

[0014] The first cross-validation unit, connected to the entity recognition unit, is used to cross-validate all the financial information entities obtained by each of the base language models, retain the same financial information entities output by multiple base language models, and cross-validate again after repeatedly recognizing the financial information texts corresponding to different financial information entities output by multiple base language models.

[0015] The first comprehensive scoring unit, connected to the first cross-validation unit and the first storage unit, is used to score each of the financial information entities based on the cross-validation results. Then, for each of the financial information entities that are below a preset scoring threshold, and for financial information texts that do not identify the corresponding financial information entity, manual verification and supplementation are requested, and the received supplementary content is saved to the entity prompt word engineering template.

[0016] Preferably, the relationship extraction module includes:

[0017] The second storage unit is used to store relationship prompt word project templates, wherein the relationship prompt word project templates store multiple template entity relationships;

[0018] The relationship identification unit, connected to the second storage unit, is used to input the financial information text, the financial information entities, and the relationship prompt word engineering template into each of the base large language models to obtain the entity relationships between each of the financial information entities and the template entity relationships.

[0019] Preferably, the relationship identification module further includes:

[0020] The second cross-validation unit, connected to the relation recognition unit, is used to cross-validate the entity relations of all financial information entities obtained by each of the base big language models, retain the entity relations of the same financial information entities output by multiple base big language models, and cross-validate again after repeatedly identifying the financial information text and the financial information entities corresponding to the entity relations of different financial information entities output by multiple base big language models.

[0021] The second comprehensive scoring unit, connected to the second cross-validation unit and the second storage unit, is used to score each of the financial information entities based on the cross-validation results. Subsequently, for the entity relationships of each of the financial information entities that are below the preset scoring threshold, as well as for financial information texts and financial information entities for which no corresponding entity relationship is identified, manual verification and supplementation are requested, and the received supplementary content is saved to the relationship prompt word project template.

[0022] Preferably, the result output module includes:

[0023] The visualization unit is used to map the financial information text to the financial information entity, map the financial information entity to the entity relationship of the financial information entity, and then sequentially associate all the financial information text, the financial information entity and the entity relationship of the financial information entity according to the mapping relationship and generate a visualization chart in a predetermined format.

[0024] Preferably, the predetermined format is one of the following: resource description framework, extensible markup language, JavaScript object representation, character delimiter, graph database, or relational database.

[0025] This invention also provides a method for mining financial activity information based on multiple base-based large language models, applied to the aforementioned financial activity information mining system, comprising:

[0026] Step S1: The financial activity information mining system uses multiple base-based large language models to perform entity recognition and mining on the input financial information text, and obtains multiple financial information entities.

[0027] Step S2: The financial activity information mining system identifies and mines entity relationships between the financial information text and the financial information entities to obtain the entity relationships between each of the financial information entities.

[0028] Step S3: The financial activity information mining system organizes the financial information text and its corresponding financial information entities, as well as the corresponding entity relationships, into a chart for display.

[0029] Preferably, a text preprocessing process is included before performing step S1, including:

[0030] The financial activity information mining system segments the input original text into multiple segments according to semantic paragraphs or fixed lengths. Then, each segment is preprocessed with noise removal, word segmentation, and part-of-speech tagging. Finally, the original text and the preprocessed segment texts are input into the entity recognition module as the financial information text.

[0031] The above technical solution has the following advantages or beneficial effects: by using multiple base-based large language models to perform entity recognition and mining on the input financial information text and to extract the relationships between entities, it can improve the efficiency of entity and relationship recognition in financial activities and meet the accuracy requirements of financial activities. It avoids the problem that the analysis of financial activity entities and relationships in traditional technologies often relies on manual methods, which is inefficient and prone to missing important information. Attached Figure Description

[0032] Figure 1 A schematic diagram of the results of a financial activity information mining system based on multiple base large language models, which is a preferred embodiment of the present invention;

[0033] Figure 2 This is a flowchart illustrating a preferred embodiment of the present invention for a method of mining financial activity information based on multiple base large language models. Detailed Implementation

[0034] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The present invention is not limited to this embodiment; other embodiments that conform to the spirit of the present invention may also fall within the scope of the present invention.

[0035] In a preferred embodiment of the present invention, based on the aforementioned problems existing in the prior art, a financial activity information mining system based on multiple base-based large language models is provided, such as... Figure 1 As shown, it includes:

[0036] Entity recognition module 1 is used to perform entity recognition and mining on the input financial information text using multiple base-based large language models to obtain multiple financial information entities;

[0037] Relationship extraction module 2, connected to entity recognition module 2, is used to identify and mine entity relationships between financial information text and financial information entities to obtain entity relationships between various financial information entities.

[0038] The output module 3, the connection relationship extraction module 2, and the entity recognition module 1 are used to organize the financial information text and its corresponding financial information entities, as well as the corresponding entity relationships, into a chart for display.

[0039] Specifically, Large Language Models (LLMs, such as GPT-4, PaLM, Claude, etc.) are a deep learning-based artificial intelligence technology primarily used for Natural Language Processing (NLP) tasks.

[0040] In this invention, multiple base-based large language models are used to perform entity recognition and mining on the input financial information text, as well as to extract the relationships between entities. This can improve the efficiency of entity and relationship recognition in financial activities and meet the accuracy requirements of financial business. It avoids the problem that the analysis of entities and relationships in financial activities in traditional technologies often relies on manual methods, which are inefficient and prone to missing important information.

[0041] In a preferred embodiment of the present invention, a text preprocessing module 4 is further included, connected to the entity recognition module 1, comprising:

[0042] The text segmentation unit 41 is used to segment the input raw text into multiple segments according to semantic paragraphs or fixed lengths.

[0043] The text preprocessing unit 42 is connected to the text segmentation unit 41 and is used to perform noise removal, word segmentation and part-of-speech tagging preprocessing on each segment of text. Then, the original text and the preprocessed segmented text are used as financial information text input to the entity recognition module.

[0044] Specifically, in this embodiment, the text segmentation unit divides the text into semantic paragraphs or fixed lengths to ensure that the data blocks input to the model conform to their context window limits (such as 4096 tokens).

[0045] In the text preprocessing unit, noise removal and cleaning employs regular expressions and a rule engine to remove irrelevant characters (such as HTML tags and special symbols), preserving the plain text content. Word segmentation and part-of-speech tagging utilize a financial-specific dictionary (such as the FinBERT vocabulary) for word segmentation, combined with part-of-speech tagging (such as "noun-institution" and "verb-transaction") to assist in subsequent entity classification.

[0046] The text preprocessing module offers several beneficial effects:

[0047] 1. Input standardization: Segmentation and cleaning reduce parsing errors caused by messy formatting in the model.

[0048] 2. Domain adaptability: Financial-specific word segmentation improves the accuracy of entity recognition (e.g., distinguishing between "Apple Inc." and "Apple futures").

[0049] In a preferred embodiment of the present invention, the entity recognition module 1, as shown in the figure, is... Figure 1 As shown, it includes:

[0050] The first storage unit 11 is used to store entity prompt word project templates, and the entity prompt word project templates store multiple template entity names;

[0051] The entity recognition unit 12 is connected to the first storage unit 11 and is used to input financial information text and entity prompt word engineering templates into the large language model of each base to obtain multiple financial information entities corresponding to the template entity names.

[0052] In a preferred embodiment of the present invention, the entity recognition module 1 is as follows: Figure 1 As shown, it also includes:

[0053] The first cross-validation unit 13 is connected to the entity recognition unit 12 and is used to cross-validate all financial information entities obtained by each base big language model, retain the same financial information entities output by multiple base big language models, and repeatedly identify and cross-validate the financial information texts corresponding to different financial information entities output by multiple base big language models.

[0054] The first comprehensive scoring unit 14 is connected to the first cross-validation unit 13 and the first storage unit 11. It is used to score each financial information entity based on the cross-validation results. Then, for each financial information entity that is lower than the preset scoring threshold, as well as for financial information text that does not identify the corresponding financial information entity, manual verification and supplementation are requested. The received supplementary content is saved to the entity prompt word engineering template.

[0055] Specifically, Large Language Model (LLM) is a deep learning-based artificial intelligence technique primarily used in Natural Language Processing (NLP) tasks. While LLM has made significant progress in NLP, its predominantly black-box approach, which doesn't provide complete intermediate computational steps for the output, can lead to problems like machine hallucination due to training data bias, logical errors, and fabricated scenarios. This limits its application in scenarios like finance and law, where high accuracy is crucial.

[0056] Human-in-the-loop (HITL) is a method of integrating human supervision and intervention into automated systems. In this model, humans and machines collaborate, with machines performing some tasks while humans supervise, verify, and correct the machine's decisions. This approach emphasizes retaining human involvement in the automated decision-making process to improve the system's accuracy and reliability. When applied to AI-driven automated systems, this method can effectively improve accuracy, reliability, flexibility, and interpretability. However, it also faces challenges such as increased labor costs, slower response times, and higher system integration complexity.

[0057] To address the technical shortcomings of the two technologies mentioned above, this embodiment integrates multiple foundational large language models (foundational large language models refer to unoptimized, unaligned large language models with human preferences; their core purpose is to serve as general-purpose infrastructure for AI development, forming basic language capabilities through pre-training on massive amounts of unlabeled text (such as publicly available internet data), typically possessing hundreds of billions of parameters to support the potential for understanding and generating complex language patterns. The core value of these models lies in their "foundational" attribute—developers can perform deep customization based on their open technical frameworks (such as open-source model weights or API interfaces), for example, fine-tuning with private domain data to build specialized models for vertical industries such as healthcare and finance, or integrating enterprise knowledge bases and business system plugins to form closed-loop applications), and utilizes a "human-in-the-loop" multi-turn dialogue intent recognition mode to reduce the "machine illusion" of large language models, ultimately improving the accuracy and efficiency of information entity and relation mining.

[0058] Large base model

[0059] This invention is applicable to financial activity participants and regulatory agencies, helping them to quickly locate key entity information and relationships between entities in massive amounts of financial information text data, thereby improving work efficiency and accuracy.

[0060] More specifically, in this embodiment, the module integrates multiple base-based large language models (typical examples include Meta's Llama series, Baichuan Intelligence's Baichuan basic version, etc.) and combines them with entity prompt word engineering templates for entity recognition. Among them:

[0061] The first storage unit stores pre-defined template entity names (such as "financial institutions" and "financial products"), providing structured prompts for the model to narrow the recognition scope and improve the efficiency of targeted entity mining. For example, consider processing a press release about "XX Bank launching a new digital currency wealth management product." The first storage unit pre-stores entity templates in the financial field, including categories such as "financial institutions," "financial products," and "laws and regulations," with specific entity names like "bank," "fund," and "securities law." The system extracts keywords from the templates for the "financial institutions" category (such as "bank" and "securities company") and the "financial products" category (such as "wealth management product" and "digital currency"), using them as prompts for the base language model. Based on the prompts "bank" and "wealth management product" from the templates, the model focuses on recognizing "XX Bank" and "digital currency wealth management product" in the text.

[0062] Entity Recognition Unit: This unit inputs preprocessed text and template entity names into multiple large language models, leveraging the models' deep semantic understanding to extract financial entities. For example, the input text is: "Securities Company A and Fund B jointly issue a 'Steady Growth' bond fund, subject to the 'Securities Investment Fund Law'." The processing involves parallel processing using multiple models: Model A (e.g., GPT-4): Outputs entities "Securities Company A," "Fund B," "Steady Growth," "Bond Fund," and "Securities Investment Fund Law." Model B (e.g., PaLM): Outputs entities "Securities Company A," "Fund B," "Steady Growth Fund," and "Securities Law." Initial identification of entities such as "Securities Company A," "Fund B," "Steady Growth," and "Securities Investment Fund Law" is achieved, but differences exist between models (e.g., between "Bond Fund" and "Steady Growth Fund"). The entity candidate list contains the outputs of multiple models for subsequent cross-validation processing.

[0063] The first cross-validation unit compares the outputs of different models, retaining only entities consistently identified by multiple models to reduce the risk of "machine illusion" from a single model. For example, in the embodiment of the entity recognition unit, the outputs of the three base language models for the input text are as follows:

[0064] Model A: Securities Company A, Fund B, Steady Growth Fund, Bond Fund, Securities Investment Fund Method Model B: Securities Company A, Fund B, Steady Growth Fund, Securities Law

[0065] Model C: Securities Company A, Fund B, and Stable Growth Bond Fund

[0066] The verification process then proceeds, including:

[0067] 1. Consistency comparison:

[0068] All models identified "Securities Company A" and "Fund B".

[0069] There are naming differences between "Steady Growth" and "Steady Growth Fund" and "Steady Growth Bond Fund".

[0070] The "Securities Investment Fund Law" only recognizes Model A, while the "Securities Law" only recognizes Model B.

[0071] 2. Verification rules:

[0072] Retain consistent entities: "Securities Company A" and "Fund B".

[0073] Disputed entities: “Steady Growth” and “Securities Investment Fund Law” require further processing.

[0074] 3. Results: Only "Securities Company A" and "Fund B" were directly retained, while the remaining entities were transferred to the manual verification process.

[0075] The first comprehensive scoring unit scores the confidence level of entities based on the cross-validation results. Entities with low scores trigger a manual verification process, and manually supplemented entities are updated to the template, forming a closed-loop optimization. In the first comprehensive scoring unit, the verification results of the first cross-validation unit are evaluated using a human-in-the-loop multi-turn dialogue. This utilizes a local large language model prompt word engineering template combined with a local algorithm. While performing intelligent analysis and optimization based on the human-in-the-loop approach, the content of the local large language model prompt word engineering template is also adjusted and improved, thereby enhancing the accuracy and efficiency of mining financial activity information entities and relationships.

[0076] For example, the scoring and processing of the disputed entities “steady growth” and “securities investment fund law” output by the first cross-validation unit.

[0077] Then, scoring and human intervention are carried out. The specific process includes:

[0078] 1. Scoring Rules:

[0079] The more times an entity is recognized by the model, the higher its score.

[0080] "Steady growth" was identified by models A and C (score 80 / 100), but the naming was inconsistent.

[0081] The "Securities Investment Fund Law" was only recognized by Model A (score 40 / 100).

[0082] 2. Human intervention:

[0083] The low-scoring entity "Securities Investment Fund Law" triggers manual review. Through the human-in-the-loop approach, the intelligent analysis results are subjected to manual auxiliary review and correction to improve the accuracy and completeness of the results, and the information identified manually is used to dynamically adjust and improve the local large language model prompt engineering template in real time. The reviewer confirms that the "Securities Investment Fund Law" is explicitly mentioned in the text, and adds it to the entity list.

[0084] After manual verification, "Stable Profit Growth" is corrected to the full name "Stable Profit Growth Bond Fund", which is then updated to the entity prompt engineering template.

[0085] 3. Result:

[0086] Final entity list: A Securities Company, B Fund, Stable Profit Growth Bond Fund, Securities Investment Fund Law.

[0087] Template update: Add "Stable Profit Growth Bond Fund" to the "Financial Product" category to improve the subsequent recognition accuracy.

[0088] It can be seen from the above embodiments that the first storage unit guides the model in a template-oriented manner to reduce interference from irrelevant entities. The entity recognition unit uses multi-model collaboration to expand the entity coverage. The cross-validation unit filters noise through consistency check to improve the reliability of results. The comprehensive scoring unit dynamically optimizes the template in combination with the human-in-the-loop to realize the self-learning capability of the system. This module significantly improves the accuracy and adaptability of financial entity recognition through the closed-loop process of "template guidance → multi-model recognition → cross-validation → manual calibration".

[0089] The beneficial effects of the entity recognition module lie in:

[0090] 1. Improved accuracy: Multi-model cross-validation significantly reduces the false positive rate, and the human-in-the-loop mechanism further corrects deviations.

[0091] 2. Enhanced adaptive capability: The dynamic template update mechanism enables the system to adapt to emerging entities in the financial field (such as new types of financial products).

[0092] 3. Optimized efficiency: The model is guided in a targeted manner through the prompt engineering template, which reduces invalid calculations and shortens the entity recognition time.

[0093] In the preferred embodiment of the present invention, the relation extraction module 2 is as shown in Figure 1 the figure, comprising:

[0094] A second storage unit 21, configured to store a relation prompt engineering template, where a plurality of template entity relations are stored in the relation prompt engineering template;

[0095] The relationship identification unit 22 is connected to the second storage unit 21 and is used to input financial information text, financial information entities and relationship prompt word engineering templates into each base large language model to obtain the entity relationship between each financial information entity and the template entity relationship.

[0096] In a preferred embodiment of the present invention, the relationship identification module 2 is as follows: Figure 1 As shown, it also includes:

[0097] The second cross-validation unit 23 and the connection relationship recognition unit 22 are used to cross-validate the entity relationships of all financial information entities obtained by each base big language model, retain the entity relationships of the same financial information entities output by multiple base big language models, and cross-validate again after repeatedly recognizing the financial information text and financial information entities corresponding to the entity relationships of different financial information entities output by multiple base big language models.

[0098] The second comprehensive scoring unit 24, connected to the second cross-validation unit 23 and the second storage unit 21, is used to score each financial information entity based on the cross-validation results. Then, for the entity relationships of each financial information entity that are lower than the preset scoring threshold, as well as for financial information texts and financial information entities for which no corresponding entity relationship is identified, manual verification and supplementation are requested, and the received supplementary content is saved to the relationship prompt word engineering template.

[0099] Specifically, in this embodiment, based on entity recognition, this module drives multi-model collaborative mining of relationships between entities through relational prompt word engineering templates (such as "investment relationship," "creditor-debtor relationship," etc.), wherein:

[0100] The second storage unit stores predefined financial relationship templates, providing a semantic framework for relationship extraction. A specific example is analyzing a paragraph in a listed company's annual report: "XX Group, through its subsidiary, invested 500 million yuan in YY Technology, acquiring a 30% stake, and signed a strategic cooperation agreement."

[0101] The relation extraction process then proceeds, including:

[0102] 1. Template call: The second storage unit pre-stores financial relationship templates, including categories such as "investment relationship", "equity ratio" and "cooperation agreement", with specific relationships such as "investment-amount", "holding-ratio" and "cooperation-type".

[0103] 2. Template matching: The system extracts prompts related to "investment relationship" (such as "capital injection", "shareholding", "agreement") to guide the model to focus on the flow of funds and cooperation agreements in the text.

[0104] 3. Results: Based on the keywords in the template, the model identifies the "investment relationship" and "strategic cooperation" relationship between "XX Group and YY Technology".

[0105] Relationship recognition unit: Input entities and text into multiple large language models, combine templates to generate candidate relationships, and use the model's contextual reasoning capabilities to capture implicit associations. For example: Input text: "Bank A provides a loan of 2 billion yuan to Real Estate Company B, with the collateral being the C shopping center assets held by Company B."

[0106] The subsequent relationship identification process includes:

[0107] 1. Parallel processing of multiple models:

[0108] Model A (e.g., Meta Llama 3): Output relationship “Bank A - Loan - Real Estate Company B (Amount: 2 billion)” “Real Estate Company B - Mortgage - Shopping Center C”.

[0109] Model B (e.g., Baichuan 2): Output relationship “Bank A - Lending - Real Estate Company B (2 billion yuan)” and “Real Estate Company B - Asset Mortgage - Shopping Center C”.

[0110] Model C (e.g., Qwen 1.5 series): Output relationship “Bank A - Loan - Real Estate Company B (2 billion)” “Real Estate Company B - Collateral - Shopping Center C”.

[0111] 2. Results integration: Initially, two types of relationships were extracted: "loan" and "mortgage", but there are differences in the descriptions (such as "loan" versus "lending" and "borrowing").

[0112] 3. Results: The candidate relation list contains similar semantic relations output by multiple models, which are then processed by the subsequent cross-validation unit.

[0113] The second cross-validation unit performs consistency verification on the relationships output by multiple models, retaining highly repetitive relationships and filtering out noisy data. Specifically, for the entity relationships identified by the relationship recognition unit, the outputs of the three models are as follows:

[0114] Model A: Loan (Bank A → Real Estate Company B, 2 billion), Mortgage (Real Estate Company B → Shopping Mall C)

[0115] Model B: Lending (Bank A → Real Estate Company B, 2 billion yuan), Asset Collateral (Real Estate Company B → Shopping Center C)

[0116] Model C: Loan (Bank A → Real Estate Company B, 2 billion), Collateral (Real Estate Company B → Shopping Mall C)

[0117] Cross-validation is then performed, and the specific process includes:

[0118] 1. Semantic alignment:

[0119] The terms "loan," "lending," and "borrowing" are uniformly mapped to "creditor-debtor relationship," with the subtype being "loan."

[0120] The terms "mortgage," "asset mortgage," and "mortgaged property" are uniformly mapped to "guarantee relationship," with the subtype being "asset mortgage."

[0121] 2. Consistency rules:

[0122] Preserving consistency: All models identify the semantic core of “loan” and “mortgage”.

[0123] Mark the differences in details: the monetary units “hundred million” and “hundred million yuan” need to be standardized, and the attributes of “collateral” need to be supplemented.

[0124] 3. Results: The standardized relationship is "Bank A - Loan - Real Estate Company B (2 billion yuan)" and "Real Estate Company B - Asset Mortgage - Shopping Center C".

[0125] The second comprehensive scoring unit: initiates manual review of low-confidence relationships and feeds back the manually corrected results to the template library to enhance the coverage of relationship templates. Specifically, this includes the scoring and handling of the disputed relationship "Bank A - Loan - Real Estate Company B" after cross-validation. The process includes:

[0126] 1. Scoring Rules:

[0127] The more times a relationship is identified by the model, the higher the score.

[0128] "Loan" was identified by models A and B (score 90 / 100); "borrowing" was only used by model C (score 60 / 100).

[0129] 2. Human intervention:

[0130] The low-scoring relationship "loan" triggered manual review. The reviewer confirmed that the text clearly used the word "loan," corrected the relationship to "loan," and added the attribute "Type: Long-term Loan."

[0131] The word "borrowing" was removed as a redundant relation, and synonyms for "loan" such as "lending" and "borrowing" were added to the relation template to improve the semantic mapping rules.

[0132] 3. Results:

[0133] Final relationship list: Bank A - Loan - Real Estate Company B (2 billion yuan, type: long-term loan), Real Estate Company B - Asset Mortgage - Shopping Center C.

[0134] Template Update: Added synonym mapping under the "Creditor-Debtor Relationship" category to improve the semantic generalization ability of subsequent relationship extraction.

[0135] As can be seen from the above embodiments: the second storage unit limits the relationship type through templates, reducing noise caused by model improvisation. The relationship identification unit captures diverse relationship representations by leveraging the complementarity of multiple models. The cross-validation unit addresses representation differences through semantic alignment and standardization. The comprehensive scoring unit combines manual correction with dynamically optimized templates to achieve accurate expression of relationship logic. This module, through a closed-loop process of "template guidance → multi-model extraction → semantic alignment → manual calibration," significantly improves the reliability and interpretability of financial relationship mining. The beneficial effects achieved include:

[0136] 1. Complex relationship mining capability: Multi-model collaboration can identify non-explicit relationships such as fund flows and legal disputes.

[0137] 2. Dynamic optimization mechanism: The continuous updating of the template library enables the system to adapt to changes in financial regulatory policies.

[0138] 3. Enhanced robustness: Cross-validation effectively avoids erroneous associations caused by biases in model training data.

[0139] In summary, the main technical effects of this invention include the following three aspects:

[0140] 1. This approach combines multiple base-based large language models with human-in-the-loop methods to improve the accuracy of information entity and relation identification and mining. Specifically, based on multiple base-based large language models, entity identification and mining are performed on preprocessed text. The identification results are then cross-matched and validated using different base-based large language models to improve the accuracy of information entity and entity relation identification and mining.

[0141] 2. Human-in-the-loop intelligent analysis optimizes and improves the accuracy of entity recognition extraction from financial information text data. Specifically, it employs a method combining multiple base-based large language models with local large language model prompt word engineering templates. This improves the efficiency of financial information text processing while reducing the "machine illusion" of the large language model through multi-turn dialogues and local large language model prompt word engineering templates, thereby enhancing the accuracy of entity recognition and mining from financial information text data.

[0142] 3. Human-in-the-loop intelligent analysis optimizes and improves the accuracy of identifying information entity relationships from financial information text data. Specifically, it employs a method combining multiple base-based large language models with local large language model prompt word engineering templates. This improves the efficiency of processing financial information text while reducing the "machine illusion" of the large language model through multi-turn dialogues and local large language model prompt word engineering templates, thereby enhancing the accuracy of identifying and mining information entity relationships from financial information text data.

[0143] In a preferred embodiment of the present invention, the result output module 3 includes:

[0144] The visualization unit 31 is used to map financial information texts to financial information entities, map financial information entities to their entity relationships, and then sequentially associate all financial information texts, financial information entities, and their entity relationships according to the mapping relationship to generate a visualization chart in a predetermined format.

[0145] In a preferred embodiment of the present invention, the predetermined format is one of a resource description framework, Extensible Markup Language, JavaScript object representation, character delimiter, graph database, or relational database.

[0146] Specifically, the visualization unit in this embodiment uses a graph database (such as Neo4j) to construct an entity-relationship network graph, supporting dynamic interactions (such as node expansion and path tracing). For example:

[0147] The input text is an annual report of a listed company. The system has identified the entities "XX Company," "Chairman Zhang," "Subsidiary YY Technology," and "Investment Agreement," as well as the relationships "Holding" and "Signing." After entity and relationship identification, the following results are obtained:

[0148] Entity list: XX Company (enterprise), Zhang (person), YY Technology (enterprise), Investment Agreement (document).

[0149] Relationship list: XX Company → Holding → YY Technology, Zhang → Signed → Investment Agreement, XX Company → Related → Investment Agreement.

[0150] Then, using the Neo4j graph database, nodes (entities) and edges (relationships) are created to generate an interactive network graph. Users can click on nodes to expand detailed information (such as "Zhang's" position and shareholding ratio). Key relationships are highlighted (such as "controlling stake" indicated by a thick red border). End users can intuitively see the capital chain formed by "XX Company" through its controlling stake in "YY Technology," as well as "Zhang's" core role in the signing of the agreement.

[0151] Furthermore, users need to import entity and relationship data into an external risk analysis system, which only supports JSON format input. Therefore, format conversion is performed, supporting structured formats such as RDF and JSON. The types of format conversion include, but are not limited to, Resource Description Framework (RDF), Extensible Markup Language (XML), JavaScript Object Notation (JSON), Comma-Separated Values ​​(CSV), graph databases, relational databases, etc., to facilitate seamless integration with external systems (such as regulatory databases).

[0152] The benefits of visualizing the results output module are as follows:

[0153] 1. Enhanced decision support: Visual charts help users quickly locate key entities (such as high-risk transaction entities).

[0154] 2. System compatibility extension: Multi-format output meets the needs of different downstream analysis tools (e.g., JSON for API calls, CSV for report generation).

[0155] This invention also provides a method for mining financial activity information based on multiple base-based large language models, applicable to the aforementioned financial activity information mining system, such as... Figure 2 As shown, it includes:

[0156] Step S1: The financial activity information mining system uses multiple base-based large language models to perform entity recognition and mining on the input financial information text, and obtains multiple financial information entities.

[0157] Step S2: The financial activity information mining system identifies and mines entity relationships between financial information texts and financial information entities to obtain the entity relationships between various financial information entities.

[0158] In step S3, the financial activity information mining system organizes the financial information text and its corresponding financial information entities, as well as the corresponding entity relationships, into charts for display.

[0159] In a preferred embodiment of the present invention, a text preprocessing process is further included before performing step S1, comprising:

[0160] The financial activity information mining system segments the input raw text into multiple segments according to semantic paragraphs or fixed lengths. Then, each segment is preprocessed by noise removal, word segmentation, and part-of-speech tagging. Finally, the original text and the preprocessed segment texts are used as financial information text input to the entity recognition module.

[0161] The above are merely preferred embodiments of the present invention and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should recognize that any equivalent substitutions and obvious changes made using the content of this specification and illustrations should be included within the protection scope of the present invention.

Claims

1. A financial activity information mining system based on multiple base-based large language models, characterized in that, include: The entity recognition module is used to perform entity recognition and mining on the input financial information text using multiple base-based large language models to obtain multiple financial information entities; The relationship extraction module, connected to the entity recognition module, is used to identify and mine entity relationships between the financial information text and the financial information entities to obtain the entity relationships between each of the financial information entities. The result output module connects the relationship extraction module and the entity recognition module, and is used to organize the financial information text and its corresponding financial information entities, as well as the corresponding entity relationships, into a chart for display. The entity recognition module includes: The first storage unit is used to store entity prompt word project templates, wherein the entity prompt word project templates store multiple template entity names; An entity recognition unit, connected to the first storage unit, is used to input the financial information text and the entity prompt word engineering template into each of the base large language models to obtain multiple financial information entities corresponding to the template entity name; The first cross-validation unit, connected to the entity recognition unit, is used to cross-validate all the financial information entities obtained by each of the base language models, retain the same financial information entities output by multiple base language models, and cross-validate again after repeatedly recognizing the financial information texts corresponding to different financial information entities output by multiple base language models. The first comprehensive scoring unit, connected to the first cross-validation unit and the first storage unit, is used to score each of the financial information entities based on the cross-validation results. Subsequently, for each of the financial information entities that are below a preset scoring threshold, and for financial information texts that do not identify the corresponding financial information entity, manual verification and supplementation are requested, and the received supplementary content is saved to the entity prompt word engineering template.

2. The financial activity information mining system according to claim 1, characterized in that, It also includes a text preprocessing module, connected to the entity recognition module, comprising: Text segmentation unit is used to divide the input raw text into multiple segments according to semantic paragraphs or fixed lengths; The text preprocessing unit, connected to the text segmentation unit, is used to perform noise removal, word segmentation, and part-of-speech tagging preprocessing on each segmented text. Then, the original text and the preprocessed segmented text are input into the entity recognition module as the financial information text.

3. The financial activity information mining system according to claim 1, characterized in that, The relation extraction module includes: The second storage unit is used to store relationship prompt word project templates, wherein the relationship prompt word project templates store multiple template entity relationships; The relationship identification unit, connected to the second storage unit, is used to input the financial information text, the financial information entities, and the relationship prompt word engineering template into each of the base large language models to obtain the entity relationships between each of the financial information entities and the template entity relationships.

4. The financial activity information mining system according to claim 3, characterized in that, The relationship identification unit also includes: The second cross-validation unit, connected to the relation recognition unit, is used to cross-validate the entity relations of all financial information entities obtained by each of the base big language models, retain the entity relations of the same financial information entities output by multiple base big language models, and cross-validate again after repeatedly identifying the financial information text and the financial information entities corresponding to the entity relations of different financial information entities output by multiple base big language models. The second comprehensive scoring unit, connected to the second cross-validation unit and the second storage unit, is used to score each of the financial information entities based on the cross-validation results. Subsequently, for the entity relationships of each of the financial information entities that are below the preset scoring threshold, as well as for financial information texts and financial information entities for which no corresponding entity relationship is identified, manual verification and supplementation are requested, and the received supplementary content is saved to the relationship prompt word project template.

5. The financial activity information mining system according to claim 1, characterized in that, The result output module includes: The visualization unit is used to map the financial information text to the financial information entity, map the financial information entity to the entity relationship of the financial information entity, and then sequentially associate all the financial information text, the financial information entity and the entity relationship of the financial information entity according to the mapping relationship and generate a visualization chart in a predetermined format.

6. The financial activity information mining system according to claim 5, characterized in that, The predetermined format is one of the following: resource description framework, extensible markup language, JavaScript object representation, character-separated values, graph database, or relational database.

7. A method for mining financial activity information based on a multi-base large language model, characterized in that, The financial activity information mining system as described in any one of claims 1-6 includes: Step S1: The financial activity information mining system uses multiple base-based large language models to perform entity recognition and mining on the input financial information text, and obtains multiple financial information entities. Step S2: The financial activity information mining system identifies and mines entity relationships between the financial information text and the financial information entities to obtain the entity relationships between each of the financial information entities. Step S3: The financial activity information mining system organizes the financial information text and its corresponding financial information entities, as well as the corresponding entity relationships, into a chart for display.

8. The financial activity information mining method according to claim 7, characterized in that, Prior to performing step S1, a text preprocessing process is also included, including: The financial activity information mining system segments the input original text into multiple segments according to semantic paragraphs or fixed lengths. Then, each segment is preprocessed with noise removal, word segmentation, and part-of-speech tagging. Finally, the original text and the preprocessed segment texts are input into the entity recognition module as the financial information text.

Images