RAG knowledge base large model enhanced dialogue method and system for financial bank

Abstract

The invention discloses an RAG knowledge base large model enhanced dialogue system and method for a financial bank, and the method comprises the following steps: obtaining a document in the field of the financial bank, uploading the document to a dialogue system knowledge base, and periodically pulling a latest to-be-analyzed document task from a task message queue through a deep document understanding algorithm; obtaining a document text and an HTML (Hypertext Markup Language) table which are obtained through analysis, and carrying out block Chunki ng processing on the document text obtained through analysis; obtaining a block Chunki ng result, performing vectorization coding on the block Chunki ng result to extract text features, and storing the text features in a vector index database; obtaining cue words and simplified and reorganized knowledge fragments; and obtaining a reasoning result of the large model reasoning engine, performing content security review on the reasoning result, and if detecting that the large model generation content is not compliant, replying the user by using the recommendation verbal skill. According to the method, the intelligent analysis capability of the complex document is improved, the table data modeling analysis effect is greatly improved, and the data analysis quality on the financial statement of the financial bank is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence big model technology, and in particular to a RAG knowledge base big model enhanced dialogue method and system for financial banks. Background Art

[0002] Current large language models (LLMs) have achieved remarkable success, but they still face great limitations, especially in specific domains or knowledge-intensive tasks, especially when processing queries that exceed the training data or require current real-time information, and the results generated by large models will produce "hallucinations".

[0003] First, the training data set of the big model has a deadline, and the data set comes from the accumulation of historical data in the past, and does not cover the current real-time data or the enterprise's private domain knowledge documents. Therefore, the big model will show gibberish and hallucinations when answering questions that require current information or professional knowledge.

[0004] Secondly, storing enterprise knowledge documents in index databases can lead to problems such as low quality of knowledge recall retrieval and redundant recalled knowledge. In addition, problems such as unclear user query intentions and missing key information in queries ultimately lead to low or irrelevant answer quality from the large model.

[0005] Thirdly, enterprise knowledge documents contain a large number of important business report tables. These data will be discarded during the common document parsing process. Therefore, the report table data cannot be matched during index recall, causing the LLMs large model to fabricate core indicator data in the absence of appropriate report table data.

[0006] Finally, generally speaking, the application of large-model retrieval to professional knowledge in vertical fields enhances generation capabilities and improves the quality of generated answers in professional fields, but lacks the ability to call tools and plan tasks, and cannot call resources outside the vertical knowledge base well. Summary of the invention

[0007] The object of the present invention is to provide a RAG knowledge base large model enhanced dialogue method and system for financial banks, so as to solve the above-mentioned problems existing in the prior art.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0009] A RAG knowledge base large model enhanced dialogue method for financial banks, comprising the following steps:

[0010] S100, obtaining documents in the financial banking field, uploading the documents to the dialogue system knowledge base, asynchronously loading the documents and obtaining the number of pages of each document, dividing each 15 pages into a document content parsing task and storing them in a task message queue, and periodically pulling the latest document tasks to be parsed from the task message queue through a deep document understanding algorithm;

[0011] S200, obtaining the document text and HTML table obtained by parsing, and performing chunking processing on the document text obtained by parsing;

[0012] S300, obtaining chunking results, performing vectorized encoding on them to extract text features, and storing them in a vector index library;

[0013] S400, obtaining prompt words and simplified and reorganized knowledge fragments, converting their formats and splicing them into inputs into the large model reasoning engine;

[0014] S500. Obtain the inference results of the large model inference engine and conduct a content security review on it. If it is detected that the content generated by the large model is non-compliant, use recommended words to reply to the user.

[0015] In some specific embodiments, the deep document understanding algorithm parses and extracts the content of the document text and identifies the report table, and the document text is represented in Markdown, and the table is represented in HTML code;

[0016] The deep document understanding algorithm supports the processing of merged cells and nested cells, and uses HTML code to restore the original table structure; at the same time, the deep document understanding algorithm has the function of removing headers and footers and the ability to bind table content with table titles.

[0017] In some specific embodiments, in step S200, during the chunking process, the table is firstly stripped of HTML tags and then chunked.

[0018] In some specific embodiments, in step S300, it further includes: extracting knowledge graph triple relationship entities from the chunking results and storing them in a graph database;

[0019] Extract keywords from the chunking results and complete the small2big mapping relationship from keywords to text blocks.

[0020] In some specific embodiments, step S300 also includes: completing the construction and storage of various types of indexes of text blocks, at which point the knowledge base document parsing task is completed and the status of each document is in a dialogable stage.

[0021] In some specific embodiments, the method further comprises:

[0022] The knowledge base management module is used to edit, modify, and remove the results of the segmented text blocks, and manually write keyword tags;

[0023] The reflection quality evaluator is used to measure the contribution of each sentence to answering the user's query and filter out low-quality sentences.

[0024] In some specific embodiments, the method further comprises: reassembling the sentences retained after being screened by the reflection quality assessor into new knowledge fragments;

[0025] At the same time, if no new knowledge fragments are obtained, the search tool is used to query the knowledge fragments in real time online;

[0026] The knowledge fragments obtained with the help of the search tool are also processed by the reflective quality assessor to simplify and reorganize the knowledge.

[0027] In some specific embodiments, the method further comprises:

[0028] If the conversation involves chart drawing, the big model engine first analyzes and extracts the coordinate axis data points, and then the front end completes the rendering of the chart. Finally, the big model engine further generates a summary description based on the current conversation and chart information.

[0029] In some specific embodiments, it includes: a deep document understanding module, a text segmentation module, a page layout analysis module, a multi-functional retriever, a reflection quality assessor, a large model understanding generation module and a tool calling module;

[0030] Deep document understanding module, used for document reading, text segmentation, layout analysis, and text indexing;

[0031] Text chunking is used to split the read document information into smaller text chunks;

[0032] The layout analysis module is used to analyze and detect the complex layout structure of the document, divide the document layout into different areas such as text, table, image, etc., and then perform content recognition on different layout areas;

[0033] A multifunctional retriever is used to create a query index for text information in an indexing manner and to sort the matching results recalled by the retrieval by relevance;

[0034] Reflection quality assessor, used to comprehensively assess the quality of context and knowledge simplification and reorganization;

[0035] Large model understanding and generation module, used for user query understanding and answer generation functions;

[0036] Tool calling module, used to support web search, weather query, chart drawing and stock financial analysis.

[0037] In some specific embodiments, indexing methods include: traditional keyword inverted index, semantic vector index and graph index.

[0038] The beneficial effects of the present invention are as follows: the present invention discloses a RAG knowledge base large model enhanced dialogue system and method for financial banks. The present invention improves the intelligent parsing capability of complex documents, especially the parsing capability of report table data, and greatly improves the modeling and analysis effect of table data; it also solves the "hallucination" problem of large models in the reasoning generation process through self-reflection quality evaluation and knowledge reorganization algorithms, which greatly improves the generation quality of large models and avoids the problem of rigid copying of large model understanding generation modules. The present invention incorporates ReAct tool calls and MathPrompter prompt engineering, which better solves the problem of poor numerical calculation effects of large models and improves the quality of data analysis on financial bank financial statements. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of a RAG knowledge base large model enhanced dialogue method for financial banks of the present invention;

[0040] Figure 2 It is a system structure block diagram of a RAG knowledge base large model enhanced dialogue system for financial banks of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation methods described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] Reference Figure 1 and Figure 2 The RAG knowledge base large model enhanced dialogue method for financial banks shown in the figure includes the following steps:

[0043] S100, obtain documents in the financial and banking field, upload the documents to the dialogue system knowledge base, load the documents asynchronously and obtain the number of pages of each document, divide them into a document content parsing task for every 15 pages and store them in the task message queue, and periodically pull the latest document tasks to be parsed from the task message queue through the deep document understanding algorithm. It should be noted here that the documents obtained in the financial and banking field can be the annual reports of listed companies, internal rules and regulations of enterprises, financial compliance supervision policy documents of the State Financial Supervision and Administration Bureau, major announcements of listed companies, and documents on overseas investment and cooperation matters of enterprises, etc.

[0044] S200: Obtain the document text and HTML table obtained through parsing, and perform chunking processing on the document text obtained through parsing.

[0045] S300, obtain chunking results, perform vectorized encoding on them to extract text features, and store them in a vector index library.

[0046] S400, obtaining prompt words and simplified and reorganized knowledge fragments, converting their formats and splicing them into inputs into the large model reasoning engine.

[0047] S500. Obtain the inference results of the large model inference engine and conduct a content security review on it. If it is detected that the content generated by the large model is non-compliant, use recommended words to reply to the user.

[0048] In some specific embodiments, the deep document understanding algorithm parses and extracts the content of the document text and identifies the report table, and the document text is represented in Markdown, and the table is represented in HTML code;

[0049] The deep document understanding algorithm supports processing merged cells and nested cells, and uses HTML code to restore the original table structure; at the same time, the deep document understanding algorithm has the function of removing headers and footers, and the ability to bind table content to table titles. It should be noted here that if the document uploaded by the user is a scanned document, it will first be converted into a text document through OCR, and then the above document processing will be performed.

[0050] In some specific embodiments, in step S200, during the chunking process, the table is firstly stripped of HTML tags and then chunked.

[0051] In some specific embodiments, in step S300, it further includes: extracting knowledge graph triple relationship entities from the chunking results and storing them in a graph database;

[0052] Extract keywords from the chunking results and complete the small2big mapping relationship from keywords to text blocks.

[0053] In some specific embodiments, step S300 also includes: completing the construction and storage of various types of indexes of text blocks, at which point the knowledge base document parsing task is completed and the status of each document is in a dialogable stage.

[0054] In some specific embodiments, the method further comprises:

[0055] The knowledge base management module is used to edit, modify, and remove the results of the segmented text blocks, and manually write keyword tags.

[0056] In this embodiment, the user inputs a query question, and the dialogue system removes noise through question rewriting, keyword extraction, etc., vectorizes it using a semantic vector encoder, and recalls and matches knowledge fragments related to the user's query using multiple strategies. The knowledge fragments that are recalled and matched are decomposed into sentences to obtain sentences with complete semantic expressions.

[0057] The reflection quality evaluator is used to measure the contribution of each sentence to answering the user's query and filter out low-quality sentences.

[0058] In some specific embodiments, the method further comprises: reassembling the sentences retained after being screened by the reflection quality assessor into new knowledge fragments;

[0059] At the same time, if no new knowledge fragments are obtained, the search tool is used to query the knowledge fragments in real time online;

[0060] The knowledge fragments obtained by the search tool are also processed by the reflective quality assessor knowledge simplification and reorganization. The large model prompt project dynamically configures the financial indicator calculation formula Few Shot examples, such as "operating profit growth rate = (current period operating profit - previous period operating profit) / previous period operating profit * 100%".

[0061] At the same time, tool calling skill support has been added to the large model prompt project, and the capabilities of each tool and the applicable scenarios of the tool have been defined and described.

[0062] The tool skills include drawing charts such as line charts, pie charts and bar charts, and support customized functions such as chart titles and axis names.

[0063] In some specific embodiments, the method further comprises:

[0064] If the conversation involves chart drawing, the big model engine first analyzes and extracts the coordinate axis data points, and then the front end completes the rendering of the chart. Finally, the big model engine further generates a summary description based on the current conversation and chart information.

[0065] In some specific embodiments, it includes: a deep document understanding module, text segmentation, a page layout analysis module, a multi-functional retriever, a reflection quality assessor, a large model understanding generation module and a tool calling module.

[0066] Deep document understanding module, used for document reading, text segmentation, layout analysis, and text indexing. Document reading supports txt, excel, word, pdf, ppt, html, and csv file formats. Select the corresponding reading method according to different file extensions.

[0067] Text chunking is used to divide the read document information into smaller text chunks.

[0068] The purpose is to enable downstream models such as LLMs and vectorized models to perform algorithm analysis, identification and understanding without losing information, so as to avoid discarding important information due to exceeding the maximum capacity of the model input, affecting the overall system performance.

[0069] Text segmentation includes segmentation with fixed text size, segmentation with special rules, and recursive segmentation.

[0070] 1. Fixed-size chunking: This is the simplest and most direct approach, where you set the number of words in a chunk and choose whether to repeat content between chunks. Typically, we keep some overlap between chunks to ensure that semantic context is not lost between chunks.

[0071] 2. Block by rules: Use paragraphs and special tags to block the document.

[0072] 3. Recursive Chunking: This is the recommended method in most cases. It recursively decomposes the text by repeatedly applying chunking rules. The program will first split by paragraph line breaks (\n\n). Then, check the size of these chunks. If the size does not exceed a certain threshold, the chunk is retained. For chunks that exceed the standard size, split again using a single line break (\n). And so on, continuously updating smaller chunking rules (such as spaces, periods) based on the chunk size. This method allows for flexible adjustment of chunk size. For example, for information-intensive parts of the text, finer segmentation may be required to capture details; whereas for parts with less information, larger chunks can be used.

[0073] The layout analysis module is used to analyze and detect the complex layout structure of the document, divide the document layout into different areas such as text, table, image, etc., and then perform content recognition on different layout areas;

[0074] In this module, the table area is sent to the table recognition module for structural recognition, and the text area is sent to the OCR engine for text recognition. The table structure recognition module is implemented based on the RARE algorithm. The RARE model can process image input and output text information describing the image; for table image input, the RARE model outputs the table content represented by the HTML protocol.

[0075] Specifically, the text detection model analyzes the input image to obtain the coordinate points of a single line of text, then cuts the space containing the text area according to the coordinates, and then transmits it to the OCR recognition model to remove the image content;

[0076] The RARE model analyzes the image to obtain table structure information and cell coordinates.

[0077] The cell recognition result is obtained by aggregate analysis of three important information: single-line text coordinate points, single-line text recognition results, and cell coordinate points.

[0078] Finally, the cell recognition results and table structure information are combined to output the table information of HTML protocol.

[0079] A multifunctional retriever is used to create a query index for text information in an indexing manner, and to sort the matching results recalled by retrieval by relevance.

[0080] In this embodiment, specifically, a traditional keyword index is constructed for the above-mentioned segmentation results, and a large model LLMs is used to analyze and mine important keywords of the text block. In combination with the Small To Big idea, a mapping relationship between the segmented text and keywords is established to facilitate subsequent keyword-based search queries to obtain the segmented text associated with the keyword.

[0081] In order to better model the business models in the financial and banking fields, more professional financial knowledge is injected, such as banking encyclopedia, financial and banking regulatory policies, banking laws and regulations and other data, to further fine-tune the performance of the quantitative model in financial and banking business.

[0082] The vectorization model is used to encode the above-mentioned block text to obtain vector representation, which is stored in the vector database.

[0083] Build a graph index, identify all entities and their attributes from the above-mentioned text blocks, and then identify the relationships between entities, representing the entity relationships in the form of triples.

[0084] Large models (LLMs) are used to abstract and summarize entities and relationships to form a refined summary.

[0085] The above graphs are further integrated through community detection algorithms to divide them into multiple graph communities. Each graph community represents a collection of related concepts or topics.

[0086] LLMs are used to generate graph community summaries that contain global structure and semantics.

[0087] A summary of the index blocks constructed based on traditional keyword methods is given using the RAPTOR algorithm.

[0088] Specifically, a vectorized model is used to encode the index slices to obtain the slice vector representation.

[0089] The Gaussian mixture algorithm is used to cluster the feature vectors and group the texts with similar semantics together.

[0090] The large model LLMs is used to generate summary results for the obtained clustering results. These summary results are represented by vectors according to the above encoding to form the nodes of the previous level of the tree structure.

[0091] The clustering and summary generation steps are repeated recursively until no clustering can be achieved or the preset book structure depth is reached.

[0092] Through multi-way recall strategies such as keywords and semantic vectors, multiple candidate search sets are matched from the knowledge index, and then the ranking model is used to evaluate and calculate the relevance score between the user input query and each record in the search set. The relevance score range is a real number between 0 and 1. The higher the ranking score, the more relevant the search record is to the user query. 1 means that the user query and the search record are completely matched and highly relevant.

[0093] Reflection quality assessor, used to comprehensively evaluate the quality of context, knowledge condensation and reorganization.

[0094] In this embodiment, the reflection quality assessor specifically, 1. Use the large model LLMs to measure and evaluate the context quality of the above search records in detail. First, the context content is divided into single sentences according to the punctuation, and the large model LLMs batch evaluates the relevance between the sentences and the user query input Query. The relevance calculation score range is 0 to 1. The larger the score, the more relevant the sentence is to the user query.

[0095] 2. Filter out low-quality sentences based on the relevance threshold configuration and retain high-quality sentences. Reorganize the context information according to the original segmentation order.

[0096] If no valid information is obtained after the above knowledge simplification and reorganization, external data is queried through the Web search tool, and the above 1 and 2 are repeated until the number of Web search tool calls reaches the maximum or the knowledge reorganization obtains valid context information.

[0097] The large model understanding and generation module is used for user query understanding and answer generation functions.

[0098] Specifically, for the calculation of financial indicators, the mathematical calculation MathPrompter prompt project and the idea of ​​analogy prompt are introduced, and the closely related calculation indicator formula examples Few-Shot are dynamically generated in the prompt project according to the specific formula; at the same time, the user input query is analyzed before the large model LLMs reasoning, and the mathematical problems involved in the user query are abstracted into algebraic solution problems, and the specific numerical values ​​are shielded and replaced with variable names.

[0099] The general big model LLMs are further fine-tuned in the financial and banking field to enhance the big model's ability to understand financial and banking business.

[0100] Tool calling module, used to support web search, weather query, chart drawing and stock financial analysis.

[0101] In this embodiment, specifically, the tool autonomously calls the corresponding tool using the ReAct think-first-then-act approach.

[0102] ReAct prompts the project template as follows

[0103] Answer the following questions as best you can. If it is in order, you can use some tools appropriately. You have access to the following tools:

[0104] {tools}

[0105] Use the following format:

[0106] Question:the input question you must answer1

[0107] Thought:you should always think about what to do and what tools to use.

[0108] Action:the action to take,should be one of{tool_names}

[0109] Action Input: the input to the action

[0110] Observation: the result of the action

[0111] ...(this Thought / Action / Action Input / Observation can be repeated zeroor more times)

[0112] Thought:I now know the final answer

[0113] Final Answer:the final answer to the original input question

[0114] Begin!

[0115] In some specific embodiments, indexing methods include: traditional keyword inverted index, semantic vector index and graph index.

[0116] By adopting the above technical solution disclosed in the present invention, the following beneficial effects are obtained:

[0117] The present invention discloses a RAG knowledge base large model enhanced dialogue system and method for financial banks. The present invention improves the intelligent parsing capability of complex documents, especially the parsing capability of report table data, and greatly improves the modeling and analysis effect of table data; it also solves the "hallucination" problem of large models in the reasoning generation process through self-reflection quality evaluation and knowledge reorganization algorithm, and effectively improves the generation quality of large models, and avoids the problem of rigid copying of large model understanding generation modules. The present invention incorporates ReAct tool calling and MathPrompter prompt engineering, which effectively solves the problem of poor numerical calculation effect of large models and improves the data analysis quality on financial bank financial statements.

[0118] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be considered as the scope of protection of the present invention.

Claims

1. A RAG knowledge base large model enhanced dialogue method for financial banks, characterized in that: The following steps are involved: S100, obtaining documents in the field of finance and banking, uploading the documents to the dialogue system knowledge base, loading the documents in an asynchronous manner and obtaining the number of pages of each document, dividing the documents into one document content parsing task for every 15 pages and storing them in a task message queue, and periodically pulling the latest document tasks to be parsed from the task message queue through a deep document understanding algorithm; S200, obtaining the document text and HTML table obtained by parsing, and performing chunking processing on the document text obtained by parsing; S300, obtaining the chunking result, performing vectorized encoding on it to extract text features, and storing them in a vector index library; S400, obtaining prompt words and simplified and reorganized knowledge fragments, converting their formats and splicing them into inputs into the large model reasoning engine; S500: Obtain the inference result of the large model inference engine and conduct a content security review on it. If it is detected that the content generated by the large model is non-compliant, use recommended words to reply to the user.

2. The RAG knowledge base large model enhanced dialogue method for financial banks according to claim 1 is characterized in that: The deep document understanding algorithm parses and extracts the content of the document text and identifies the report table, and the document text is represented in Markdown, and the table is represented in HTML code; The deep document understanding algorithm supports the processing of merged cells and nested cells, and uses the HTML code to restore the original table structure; at the same time, the deep document understanding algorithm has the function of removing headers and footers and the ability to bind table content with table titles.

3. The RAG knowledge base large model enhanced dialogue method for financial banks according to claim 2 is characterized in that: In the step S200, during the chunking process, the table is firstly stripped of HTML tags and then chunked.

4. The RAG knowledge base large model enhanced dialogue method for financial banks according to claim 3 is characterized in that: In the step S300, it also includes: extracting knowledge graph triple relationship entities from the chunking result and storing them in a graph database; Perform keyword extraction on the Chunking result to complete the small 2 big mapping relationship from the keyword to the text block.

5. The RAG knowledge base large model enhanced dialogue method for financial banks according to claim 4 is characterized in that: In the step S300, it also includes: completing the construction and storage of various types of indexes of the text block, so that the knowledge base document parsing task is completed, and the status of each document is in the dialogue stage.

6. The RAG knowledge base large model enhanced dialogue method for financial banks according to claim 5 is characterized in that: The method further comprises: A knowledge base management module, used to edit, modify, and remove the text block results of the segmentation, and manually write keyword tags; The reflection quality evaluator is used to measure the contribution of each sentence to answering the user's query and filter out low-quality sentences.

7. The RAG knowledge base large model enhanced dialogue method for financial banks according to claim 6 is characterized in that: The method further includes: The sentences retained after being screened by the reflection quality assessor are reassembled into new knowledge fragments; Meanwhile, if the new knowledge fragment is not obtained, a search tool is used to query the knowledge fragment in real time online; The reflective quality assessor knowledge simplification and reorganization process is also performed on the knowledge fragments obtained by means of the search tool.

8. The RAG knowledge base large model enhanced dialogue method for financial banks according to claim 6, characterized in that: The method further includes: If the conversation involves a chart drawing function, the large model engine first analyzes and extracts the coordinate axis data points, and then the front end completes the rendering of the chart. Finally, the large model engine further generates a summary description based on the current conversation and chart information.

9. A RAG knowledge base large model enhanced dialogue system for financial banks, characterized in that: include: Deep document understanding module, text segmentation, page layout analysis module, multi-function retriever, reflection quality assessor, large model understanding generation module and tool calling module; The deep document understanding module is used for document reading, text segmentation, page layout analysis, and text indexing; The text chunking is used to chunk the read document information into smaller text chunks; The layout analysis module is used to analyze and detect the complex layout structure of the document, divide the document layout into different areas such as text, table, image, etc., and then perform content recognition on different layout areas respectively; The multifunctional retriever is used to create a query index for the text information in an indexing manner, and to sort the matching results recalled by the retrieval by relevance; The reflection quality assessor is used to comprehensively assess the quality of context and knowledge simplification and reorganization; The large model understanding generation module is used for user query understanding and answer generation functions; The tool calling module is used to support web search, weather query, chart drawing and stock financial analysis.

10. The RAG knowledge base large model enhanced dialogue system for financial banks according to claim 9, characterized in that: The indexing methods include: traditional keyword inverted index, semantic vector index and graph index.

Images