A financial field RAG keyword retrieval method based on inter-word relationship mining
By constructing word density and new word discovery logic in the financial field, and combining the BERT model to mine keyword importance and inter-word relationships, the problem of the imbalance between recall and precision in keyword retrieval in the RAG method is solved, and efficient identification and accurate retrieval of new information are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- EAST MONEY INFORMATION CO LTD
- Filing Date
- 2026-01-14
- Publication Date
- 2026-05-29
AI Technical Summary
Existing RAG methods are difficult to efficiently mine keywords and establish relationships between words in the financial field, resulting in an imbalance between retrieval recall and precision, and insufficient ability to identify new information.
We adopt the RAG keyword retrieval method in the financial field based on word relationship mining. By constructing word density and new word discovery logic, we combine the BERT model to mine keyword importance and word relationships, and use prefix tree dictionary and financial entity database for completion and updating to optimize the keyword combination recall strategy.
It improves the accuracy of keyword retrieval and the ability to identify new information, reduces reliance on word segmenters, lowers resource consumption, and improves retrieval response speed and recall accuracy.
Smart Images
Figure CN122112221A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a search enhancement generation technology, and more particularly to a RAG keyword retrieval method in the financial field based on word relationship mining. Background Technology
[0002] Retrieval-Augmented Generation (RAG) systems represent a significant breakthrough in natural language processing in recent years. By combining large language models (LLMs) with external knowledge retrieval, they significantly improve the accuracy, timeliness, and reliability of generated content. In traditional LLM applications, the model relies solely on the parameterized knowledge acquired during pre-training, leading to three main limitations: lagging knowledge updates, factual errors, and insufficient domain-specific knowledge. RAG methods effectively address these issues by introducing a dynamic retrieval mechanism, enabling LLMs to function without requiring the latest and most relevant information.
[0003] Keyword retrieval, as a core component of the RAG method, plays a fundamental role, and its effectiveness directly impacts the overall satisfaction of the search system. A key challenge facing current mainstream RAG methods is balancing recall and precision. Overly broad keywords lead to the retrieval of numerous irrelevant documents, increasing the processing burden on LLM and potentially introducing noise; conversely, overly strict keywords may miss important relevant information, resulting in incomplete search results. Therefore, developing efficient keyword mining and inter-word relationship extraction techniques is crucial for improving the performance of RAG methods.
[0004] Invention 1 proposes a method for constructing word density. By statistically analyzing the frequency of characters appearing in each word in a large corpus, the density relationship between characters and between characters and words is obtained, thereby determining the relevance of characters in word formation. This invention focuses on character-level correlation; when applied to retrieval methods, it needs to be further extended to the mining of inter-word relationships. Invention 1: Liang Jiguang; Xu Kaibo. A method, device, electronic device and storage medium for determining word density. CN113342936A
[2021] Invention 2 proposes a retrieval enhancement generation method based on multi-level semantics. It constructs a knowledge base using corpora from specialized fields and achieves multi-level understanding of search statements through a combination of vector recall and keyword retrieval. In the keyword retrieval part, this invention uses a language model to generate keywords to obtain key information, and then compares the similarity of the obtained keywords with the keywords in the text blocks of the content library one by one. Its effectiveness largely depends on the capabilities of the language model itself, and it does not further explore the relationships between keywords. Invention 2: Wang Chongyang; Chen Wenguang. A retrieval enhancement generation method based on multi-level semantics. CN120011535A
[2024] There are generally two technical approaches to existing related technologies: (1) Key content extraction based on rules and statistical information (regular expression matching, syntactic matching, TF-IDF, etc.); (2) Key information extraction based on large language models.
[0005] 1. Key content extraction based on rules and statistics is easily affected by the word segmenter and the original corpus. The distribution of general and vertical corpora often differs, and transfer applications often have biases, leading to poor results.
[0006] 2. Text statistics or rule matching methods suffer from time lag in the timeliness of rules and dictionaries, high costs for manually maintaining and updating the full vocabulary, and difficulty in ensuring the coverage of new content understanding.
[0007] 3. The approach of extracting key information from large language models requires a lot of computing power for training, and it is also difficult to directly map the obtained key information to the importance scores of different key contents during retrieval. Therefore, it is difficult to make detailed control over the importance level of key information groups.
[0008] 4. When running online, the processing latency of the language big model for extracting key information is also relatively high, which puts greater pressure on the overall retrieval method. Summary of the Invention
[0009] To address the challenge of current mainstream RAG methods in efficiently mining keywords and understanding inter-word relationships in the financial field to improve performance, a new RAG keyword retrieval method based on inter-word relationship mining is proposed for the financial sector. The specific problems addressed are as follows: 1. The typical RAG keyword retrieval method first requires segmenting the user's question into words, and then matching it with pre-segmented document slices in the content library. Therefore, keyword retrieval relies heavily on the basic capabilities of the word segmenter. Because the word segmenter needs to consider text across all domains, its update frequency is very limited. This invention adopts a word segmentation completion scheme, aiming to restore the main subject of the query when word segmentation omissions occur, thereby improving retrieval accuracy. 2. The financial sector is characterized by a wealth of specialized information, rapid information iteration, and constantly emerging trending topics. Therefore, keyword retrieval methods must be able to identify and understand the latest content; otherwise, they cannot meet user requirements. This invention integrates new word discovery logic, broadly covering emerging trending topics and new entities, maintaining the timeliness of the retrieval method's understanding.
[0010] 3. Currently, there are many identification schemes for search keywords, but there is still a need to enhance the mining of word relationships such as word co-occurrence and word grouping. This invention uses a dictionary + BERT model to construct binary or ternary keyword groups. By emphasizing these word groups, it can effectively recall content more relevant to the user's question.
[0011] The technical solution of this invention is as follows: A method for RAG keyword retrieval in the financial field based on inter-word relationship mining, and the steps are as follows: Step S1: Collection and statistics of massive vertical domain documents and common questions; obtain user questions from online documents and search services, use the same word segmenter as during retrieval to segment all documents, and obtain a list of word frequency statistics after word segmentation, including word term + part-of-speech pos; traverse the list to obtain the association frequency of two adjacent words termA and termB, and then calculate the point mutual information of term(A+B) by the probability of termA and termB appearing in the full set of documents; Step S2: New word discovery thesaurus construction and automated hot word update; After obtaining all valid pairwise term combinations from step S1, use a prefix tree dictionary to store all possible term combinations; Step S3: Simultaneously call two fine-tuned BERT models, the word importance model and the sentence reduction model, to perform keyword discovery (word importance) and word relationship mining (word density) functions respectively, and obtain candidate combined keywords; First, the entire system builds a word segmenter for financial vocabulary enhancement. This word segmenter adds financial vocabulary classification on the basis of general Chinese vocabulary and clearly divides the subcategories of financial parts of speech; The training method of the word importance model and the sentence reduction model is based on the word segmentation results of the customized word segmenter, using a new word discovery dictionary to obtain merged words, then interspersing mask tokens in the word segmentation results, and predicting these mask tokens; Step S4: For candidate keyword combinations, use the importance score of the words and the positional distance between words to calculate the comprehensive score of the multi-word combination and verify the feasibility of triple and binary combinations; after obtaining the effective binary keyword combinations, perform pairwise matching to upgrade the co-occurring combination words AB, AC, and BC into more reliable triple words ABC. Step S5: Integrate high-importance keywords with mined binary or ternary phrases or short sentences. By requiring multiple keywords to appear simultaneously and in close proximity, enhance the retrieval capability on the recall side of the retrieval method.
[0012] 2. The financial RAG keyword retrieval method based on inter-word relationship mining according to claim 1, characterized in that, the point mutual information in step S1 represents the co-occurrence probability of two terms, serving as the basis for judging whether term (A+B) is a new word unknown to the existing word segmenter; Point mutual information PMI:
[0013] Let a and b, referring to two adjacent words, term A and term B, be the probability of both appearing. Let be the probability of 'a' occurring. Similarly.
[0014] Furthermore, in step S2, during the actual retrieval, each term after word segmentation of the question will be searched for possible consecutive term combinations in the prefix tree dictionary layer by layer. If a match is found, multiple terms will be concatenated to form a new word discovery vocabulary before being sent to subsequent steps. This new word discovery vocabulary will be connected to two information sources: one is a real-time updated list of financial entities, in which different combinations of word segmentation methods for each financial entity will be included, ensuring the completeness of the entities; the other is the vertical domain document statistics information in step S1, in which the latest content of the documents is pulled and the statistics are updated regularly according to the plan, ensuring the timeliness of the new words.
[0015] Furthermore, in step S3, the word importance model aims to provide an importance score for each word segment. This is achieved by assigning a score to each position of the mask token replaced with [MASK]. The score is a continuous value; words with higher scores receive greater attention weight during retrieval. The sentence reduction model aims to directly provide candidate keyword combinations that consider inter-word relationships; the output is a subset of the input word segmentation list. This is achieved by classifying each [MASK] position using a 0-1 classification; words classified as 1 are retained. Words with high importance are retained in the sentence reduction. In the output list of the sentence model, words with low importance are also removed as candidate combination keywords. However, there are some distinctions: Case 1) When financial word segmentation is performed, entities are fragmented. In this case, the importance model may not give high scores to all words in the entity, but the sentence reduction model will retain all words corresponding to the entity. Case 2) Non-key information in the syntax, including conjunctions and prepositions, do not have as high text importance scores as entity nouns. However, when considering the relationship between words, such words that show the relationship between entities, combined with the financial word attribute tags of the word segmenter, will be retained in the results of the sentence reduction model.
[0016] Furthermore, in step S4, successfully combined multi-word phrases or short sentences will be given higher weight during actual retrieval, and triples have a higher priority than binary pairs; the formula for calculating the binary pair combination score is as follows:
[0017] and It is the importance score of candidate word A and candidate word B. and It refers to the positions of candidate word A and candidate word B in the word segmentation list; This is a configurable parameter used to control the range of combined scores, by assigning higher weights to words containing financial terms. This is to enhance and focus on financial corpora; It is the combination score of the binary word pair AB. In actual execution, a certain threshold limit will be given, and binary keyword combinations that are higher than the judgment threshold will be retained.
[0018] The beneficial effects of this invention are as follows: This invention focuses on word discovery, word completion, and word relationship mining capabilities in keyword retrieval, enabling more accurate extraction of key information needed for retrieval using limited resources.
[0019] 1. Reduce the reliance of retrieval methods on the capabilities of word segmenters: This invention constructs a word completion library in the form of a prefix tree dictionary, which can provide additional fallback capabilities for scenarios that word segmenters cannot handle, and integrate fragmented words to restore their original complete meaning.
[0020] 2. Enhanced ability to identify new information in the financial field: This invention integrates a financial entity database and a new word discovery table, and updates the new word discovery table based on the massive amount of new financial information in vertical categories on the Internet, which can effectively ensure that the key content retrieval method can identify and extract new content and new entities.
[0021] 3. Deeply explore the relationships between multiple words to improve recall accuracy: This invention uses the method of word importance + density to locate and filter key search terms, and further obtains the combination of multiple key search terms, which can bring significant gains to the accuracy of retrieval and recall.
[0022] 4. Balancing system response, performance, and resource consumption: This invention uses only the BERT model to support word importance scoring and word relationship mining throughout the entire process. Unlike solutions that use generative large models, it does not consume excessive computing resources and is more in line with the requirements of RAG retrieval methods that require fast response. Attached Figure Description
[0023] Figure 1 The data construction format for training the BERT model; Figure 2 The flowchart below illustrates the specific implementation of the RAG keyword retrieval method in the financial field according to the present invention. Detailed Implementation
[0024] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.
[0025] This invention presents a RAG keyword retrieval method for the financial field based on inter-word relationship mining. To accurately improve the semantic recall capability of RAG keyword retrieval methods in the financial field, this invention focuses on the unique characteristics of financial texts, such as high terminology density, complex concept associations, and strong market dynamics, and constructs a complete technical chain for deep mining of financial concept relationships. This chain includes key steps such as tracking market hot topics, completing professional terms, extracting core concepts, and modeling financial concept combinations, ultimately outputting a core set of financial concepts covering multi-dimensional semantics and structured concept combination relationships. The above results can be directly applied to search and analysis engines such as ElasticSearch, enhancing the depth and breadth of understanding of financial professional semantics, achieving more accurate and comprehensive financial information search and recall, and meeting the timeliness and accuracy requirements of financial businesses. The specific implementation process is as follows: Step S1: Collection and statistics of massive vertical domain documents and common questions. Obtain millions of recent online documents and user questions from search services. Use the same word segmenter as during retrieval to segment all documents, obtaining a list of word frequency statistics (term + part-of-speech pos) for each segmented document. Traverse this list to obtain the association frequency of adjacent words, term A and term B. Then, calculate the point-to-point mutual information of term (A+B) based on the probability of term A and term B appearing in all documents. Point-to-point mutual information represents the co-occurrence probability of two terms, serving as the basis for judging whether term (A+B) is a new word unknown to the current word segmenter.
[0026] Point-to-Point Information (PMI):
[0027] Let a and b (i.e., the probabilities of two adjacent words, term A and term B, appearing together) be the probability of each other. Let be the probability of 'a' occurring. Similarly.
[0028] Step S2: New word discovery thesaurus construction and automated hot word update. After obtaining all valid pairwise term combinations from Step S1, a prefix tree dictionary is used to store all possible term combinations. During actual retrieval, the solution of this invention searches for possible consecutive term combinations in the prefix tree dictionary layer by layer for each term after question segmentation. If a match is found, multiple terms are concatenated to form a new word discovery thesaurus before being sent to subsequent steps. This new word discovery thesaurus is connected to two information sources: one is a real-time updated list of financial entities, in which different combinations of segmentation methods for each financial entity are included to ensure the completeness of the entities; the other is the vertical domain document statistics information from Step S1, in which the latest content of documents is retrieved and the statistics are updated regularly to ensure the timeliness of new words.
[0029] Step S3: This invention simultaneously calls two fine-tuned BERT models, a word importance model and a sentence reduction model, to perform keyword discovery (word importance) and word relationship mining (word density) functions respectively. Considering the complexity of word combinations in financial scenarios, the entire system first builds a word segmenter specifically for financial vocabulary enhancement. This segmenter, based on general Chinese vocabulary, specifically adds classifications of financial vocabulary and clearly divides them into subcategories such as financial nouns (entities, indicators, etc.), financial verbs (trading behaviors in the market, etc.), and financial prepositions. Compared to general word segmenters, this segmenter can significantly improve the understanding of common entities, frequently used terms, and popular phrases in financial market activities. The training methods of the word importance model and the sentence reduction model are similar. Both are based on the word segmentation results of the above-mentioned customized word segmenter, using a new word discovery dictionary to obtain merged words, then interspersing mask tokens ([MASK]) in the word segmentation results, and predicting these mask tokens. See details. Figure 1 The advantages of doing this are: 1) It fully preserves the results of Chinese word segmentation and retains the enhancement effect of word segmentation for financial scenarios. It transforms the original BERT model word segmenter, which could only predict Chinese characters one by one, into one that can predict Chinese words, which can significantly improve accuracy; 2) This downstream task performs the judgment of the [MASK] position, which is more in line with the pre-training method of the BERT model and is conducive to stimulating the capabilities of the BERT model.
[0030] The word importance model aims to provide an importance score for each word segment. This is achieved by assigning a score to each [MASK] position. The score is a continuous value, ranging from 0 to 5. Words with higher scores receive greater attention during retrieval.
[0031] The goal of the sentence reduction model is to directly provide candidate keyword combinations that consider inter-word relationships; that is, the output is a subset of the input word segmentation list. This is achieved by classifying each [MASK] position using a 0-1 classification, retaining words classified as 1. Generally, words with high importance are retained in the output list as candidate keyword combinations, while words with low importance are usually discarded. However, there are some exceptions: 1) When financial word segmentation is performed, the entity is broken down. At this time, the importance model may not give high scores to all words in the entity, but the sentence reduction model will retain all words corresponding to the entity. 2) Non-critical syntactic information, including conjunctions and prepositions, generally does not score as highly in text importance as entity nouns. However, when considering inter-word relationships, these words that reveal relationships between entities are very important. Combined with the financial word attribute tags of the word segmenter, these words will be retained in the results of the sentence reduction model.
[0032] Step S4: For candidate keyword combinations, calculate the comprehensive score of multi-word combinations using word importance scores and the positional distance between words, verifying the feasibility of triple and binary combinations. Successfully combined multi-word phrases or short sentences will be given higher weight in actual retrieval, and triples have a higher priority than binary combinations. The formula for calculating the binary combination score is as follows:
[0033] and It is the importance score of candidate word A and candidate word B. and It refers to the positions of candidate word A and candidate word B in the word segmentation list; This is a configurable parameter used to control the range of combined scores, by assigning higher weights to words containing financial terms. This is to enhance and focus on financial corpora; It is the combination score of the binary word pair AB. In actual execution, a certain threshold limit will be given, and binary keyword combinations that are higher than the judgment threshold will be retained.
[0034] After obtaining effective binary keyword combinations, perform pairwise matching to upgrade co-occurring combinations AB, AC, and BC into more reliable ternary words ABC. At this point, the most important financial entities, financial events, or relationships in the user's question can be extracted.
[0035] Step S5: Integrate high-importance keywords with mined binary or trigram phrases or short sentences, and significantly enhance retrieval capabilities on the recall side of the retrieval method by using the logic of "prioritizing matching documents containing similar keyword phrases or short sentences" (i.e., requiring multiple keywords to appear simultaneously and in close proximity).
[0036] Advantages of this invention: (1) New word discovery and word segmentation completion in the financial vertical field, as well as the corresponding logic for rapid updating of the thesaurus. This solution can maintain the ability to understand the complete semantics of new entities and new content.
[0037] (2) The training method of the BERT model for mining word importance and inter-word relationships, and the post-processing scheme for combining multi-word relationships after extracting key information in the sentence. The special training method supports the BERT model's attention to Chinese words, especially Chinese financial words, and enhances its understanding ability; the mining of multi-word relationships takes into account word importance and positional information, and has strong reliability.
[0038] To achieve reasonable identification of keywords and keyword phrases, a fine-tuned generative model approach can be considered, allowing the model to generate multiple keyword phrases. The importance ranking of these multiple keyword phrases requires an additional design. It is also necessary to carefully evaluate the fidelity of the obtained keyword phrases to the original text.
[0039] The above-described embodiments are merely one implementation of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention should be determined by the appended claims.
Claims
1. A method for retrieving RAG keywords in the financial field based on inter-word relationship mining, characterized in that, The steps are as follows: Step S1: Collection and statistics of massive vertical domain documents and common questions; obtain user questions from online documents and search services, use the same word segmenter as during retrieval to segment all documents, and obtain a list of word frequency statistics after word segmentation, including word term + part-of-speech pos; traverse the list to obtain the association frequency of two adjacent words termA and termB, and then calculate the point mutual information of term(A+B) by the probability of termA and termB appearing in the full set of documents; Step S2: New word discovery thesaurus construction and automated hot word update; After obtaining all valid pairwise term combinations from step S1, use a prefix tree dictionary to store all possible term combinations; Step S3: Simultaneously call two fine-tuned BERT models, the word importance model and the sentence reduction model, to perform keyword discovery (word importance) and word relationship mining (word density) functions respectively, and obtain candidate combined keywords; First, the entire system builds a word segmenter for financial vocabulary enhancement. This word segmenter adds financial vocabulary classification on the basis of general Chinese vocabulary and clearly divides the subcategories of financial parts of speech; The training method of the word importance model and the sentence reduction model is based on the word segmentation results of the customized word segmenter, using a new word discovery dictionary to obtain merged words, then interspersing mask tokens in the word segmentation results, and predicting these mask tokens; Step S4: For candidate keyword combinations, use the importance score of the words and the positional distance between words to calculate the comprehensive score of the multi-word combination and verify the feasibility of triple and binary combinations; after obtaining the effective binary keyword combinations, perform pairwise matching to upgrade the co-occurring combination words AB, AC, and BC into more reliable triple words ABC. Step S5: Integrate high-importance keywords with mined binary or ternary phrases or short sentences. By requiring multiple keywords to appear simultaneously and in close proximity, enhance the retrieval capability on the recall side of the retrieval method.
2. The financial RAG keyword retrieval method based on inter-word relationship mining according to claim 1, characterized in that, The point mutual information (PMI) in step S1 represents the co-occurrence probability of two terms, serving as the basis for judging whether term (A+B) is a new word unknown to the existing word segmenter; Point Mutual Information (PMI): Let a and b, referring to two adjacent words, term A and term B, be the probability of both appearing. Let be the probability of 'a' occurring. Similarly.
3. The financial RAG keyword retrieval method based on inter-word relationship mining according to claim 1, characterized in that, In step S2, during the actual retrieval, each term after word segmentation of the question is searched for possible consecutive term combinations in the prefix tree dictionary layer by layer. If a match is found, multiple terms are concatenated to form a new word discovery vocabulary before being sent to subsequent steps. This new word discovery vocabulary is connected to two information sources: one is a real-time updated list of financial entities, in which different combinations of word segmentation methods for each financial entity are included to ensure the completeness of the entities; the other is the vertical domain document statistics information from step S1, in which the latest content of the documents is pulled and the statistics are updated regularly according to the plan to ensure the timeliness of the new words.
4. The financial RAG keyword retrieval method based on inter-word relationship mining according to claim 1, characterized in that, In step S3, the word importance model aims to provide an importance score for each word segment. This is achieved by assigning a score to each position of the mask token replaced with [MASK]. The score is a continuous value; words with higher scores receive greater attention weight during retrieval. The sentence reduction model aims to directly provide candidate keyword combinations that consider inter-word relationships; the output is a subset of the input word segmentation list. This is achieved by classifying each [MASK] position using a 0-1 classification; words classified as 1 are retained. Words with high importance are retained in the sentence reduction model. In the output list of the type, words with low importance are also removed as candidate combination keywords, but there are some different situations: Case 1) When financial word segmentation, the entity is fragmented. In this case, the importance model may not give high scores to all words in the entity, but the sentence reduction model will retain all words corresponding to the entity; Case 2) Non-key information in the syntax, including conjunctions and prepositions, have lower text importance scores than entity nouns; however, when considering the relationship between words, such words that show the relationship between entities, combined with the financial word attribute tags of the word segmenter, such words will be retained in the result of the sentence reduction model.
5. The financial RAG keyword retrieval method based on inter-word relationship mining according to claim 1, characterized in that, In step S4, successfully combined multi-word phrases or short sentences will be given higher weight during actual retrieval, and triples have a higher priority than binary pairs; the formula for calculating the binary pair combination score is: and It is the importance score of candidate word A and candidate word B. and It refers to the positions of candidate word A and candidate word B in the word segmentation list; This is a configurable parameter used to control the range of combined scores, by assigning higher weights to words containing financial terms. This is to enhance and focus on financial corpora; It is the combination score of the binary word pair AB. In actual execution, a certain threshold limit will be given, and binary keyword combinations that are higher than the judgment threshold will be retained.