Business processing method and system based on large model and retrieval optimization

Abstract

The invention provides a business processing method and system based on a large model and retrieval optimization, and the method comprises the steps: constructing a collection knowledge base, and carrying out the fine adjustment of a pre-trained large language model through a quantification low-rank adaptation technology, and obtaining a large language model for the collection business optimization. And a unique strategy of model parameter fixation and only low-rank matrix optimization is adopted in model training, so that the calculation resource occupation is greatly reduced, and the accuracy and performance of the large model in the collection professional field are remarkably improved. Furthermore, dynamic updating of collection related knowledge is achieved through application of the RAG technology, the problem of knowledge updating lag is effectively solved, the retrieval recall rate is remarkably increased through the synergistic effect of synonymous query enhancement and a mixed retrieval strategy, a large language model can accurately obtain related knowledge from a small amount of knowledge resources, and the retrieval efficiency is improved. Therefore, the accuracy and reliability of answering are greatly improved, more powerful support is provided for collection business processing, and the collection effect is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of financial technology, and in particular to a business processing method and system based on a large model and retrieval optimization for post-loan collection business scenarios. Background Art

[0002] In the field of post-loan debt collection, traditional business models rely heavily on manual operations. Collectors must expend considerable effort studying historical collection cases, professional sales pitches, and compliance knowledge to answer debtors' questions and expedite the resolution of overdue bills. Currently, faced with a massive volume of overdue bills, manual collection is extremely inefficient, making it difficult to cover all overdue accounts and resulting in high labor costs.

[0003] Currently, some financial institutions are experimenting with intelligent methods, such as using machine learning algorithms to assess debtors' repayment ability and willingness, and using intelligent outbound call robots to conduct preliminary telephone collection efforts. However, current intelligent collection systems have numerous flaws. First, intelligent outbound call robots primarily operate based on a pre-set library of compliant scripts, lacking flexibility and incapable of personalized collection. When debtors offer complex and diverse responses, such as a sudden serious illness that temporarily leaves them unable to repay and they wish to negotiate a new repayment plan, the system often mechanically repeats the pre-set script, unable to flexibly adjust strategies. This makes effective communication and negotiation with debtors difficult, not only failing to resolve practical issues but also potentially antagonizing them. Furthermore, existing intelligent collection methods are only applied to certain parts of the collection process, while other steps still require significant manual intervention, leaving them far from truly intelligent and automated.

[0004] Taking a typical intelligent debt collection platform as an example, while it implements automated outbound calls and strategy recommendations, it still suffers from several core issues in actual use. First, strategy generation relies heavily on pre-set rule templates, lacking flexibility and personalization. This makes it difficult to dynamically adjust collection strategies in response to debtors' personalized negotiation requests, such as those for installment repayments, resulting in poor collection effectiveness. Second, knowledge updates lag behind the iteration rate of regulatory policies. For example, updates to the latest installment policy require manual synchronization. This leads to significant deficiencies in the system's semantic understanding, knowledge iteration, and strategy generation. The system is unable to promptly incorporate the latest policies and knowledge into business processing, resulting in responses and strategy formulation that do not meet the latest requirements, impacting business compliance and effectiveness. Third, the search recall rate is low. In actual collection processes, debtors' questions are often vague, making it difficult to accurately capture their intent. Existing search methods struggle to accurately retrieve relevant information from a vast amount of knowledge resources. This results in a lack of accurate knowledge support when answering questions in large models, impacting the accuracy and reliability of the answers, and thus, hindering collection effectiveness. Fourth, business processing efficiency and coverage are low. Traditional manual collection methods, or even some intelligent ones, cannot fully cover all overdue accounts, and collection efficiency is low. Faced with growing loan volumes, it is difficult to effectively improve the recovery rate, failing to meet the business needs of financial institutions. Summary of the Invention

[0005] In order to solve the above-mentioned problems, the present invention provides a technical solution as follows:

[0006] The present invention provides a business processing method based on a large model and retrieval optimization, which has the following technical features: step S1, constructing a collection knowledge base based on a variety of collected collection-related data; step S2, fine-tuning the pre-trained base large language model through quantized low-rank adaptation technology to obtain a fine-tuned large language model; step S3, interacting with the customer, searching the collection knowledge base according to the query input by the customer, and generating query results based on the retrieval results using the large language model, wherein, in step S3, the collection database is searched based on the query using retrieval enhancement generation technology, and it is determined whether to perform synonymous enhanced retrieval based on the retrieval results. If the judgment is yes, multiple synonymous queries of the query are generated using synonymous query enhancement technology, and synonymous enhanced retrieval is performed on the collection database based on the query and the multiple synonymous queries, and then the query results are generated based on the retrieval results or the results of the synonymous enhanced retrieval using the large language model.

[0007] The business processing method based on large models and retrieval optimization provided by the present invention may also have such technical features, wherein step S1 includes the following sub-steps: step S1-1, performing data preprocessing on the various collection-related data respectively, wherein the collection-related data include legal and regulatory texts, policy texts, collection process specification texts, expert speech experience texts, historical collection case data of multiple customers, system usage documents and the bank's latest activity strategy data, and the data preprocessing includes noise data cleaning, stop word removal, duplicate data filtering and text segmentation; step S1-2, using a vector model to vectorize and index the segmented text to obtain vectorized encoded text blocks and indexes; step S1-3, storing the text blocks and indexes in a vector database as the collection knowledge base.

[0008] The business processing method based on large model and retrieval optimization provided by the present invention may also have such technical features, wherein step S2 includes the following sub-steps: step S2-1, obtaining the pre-trained base large language model; step S2-2, fixing the model parameters of the pre-trained base large language model, and initializing the weight parameters of the two low-rank matrices of the low-rank adaptation technology; step S2-3, based on the weight parameters of the two initialized low-rank matrices and the collection business data, fine-tuning the pre-trained base large language model through the quantized low-rank adaptation technology to obtain the fine-tuned large language model.

[0009] The business processing method based on large model and retrieval optimization provided by the present invention may also have such technical features, wherein, in step S2-2, the two low-rank matrices are low-rank matrices A and B, respectively, and the weight parameters of the low-rank matrix A are initialized by a Gaussian function, and the weight parameters of the low-rank matrix B are initialized to a zero matrix.

[0010] The business processing method based on large model and retrieval optimization provided by the present invention may also have such technical features, wherein step S3 includes the following sub-steps: step S3-1, receiving the query input by the customer, and vectorizing the query using the vector model; step S3-2, based on the query after vectorization encoding, using the retrieval enhancement generation technology to retrieve the top m text blocks with the highest relevance in the collection knowledge base; step S3-3, judging whether a synonym enhancement retrieval is needed based on the retrieval result of step S3-2, and entering step S3-4 when the judgment is yes, If the judgment is no, proceed to step S3-6; in step S3-4, generate N synonymous queries of the query using the synonym query enhancement technology; in step S3-5, use the vector model to vectorize the synonymous queries, and based on the vectorized query and the N synonymous queries, use the retrieval enhancement generation technology to retrieve the top m text blocks with the highest relevance in the collection knowledge base; in step S3-6, use a predetermined prompt word template to arrange the m text blocks and the text of the query, and input the arranged text into the large language model to generate the query result.

[0011] The business processing method based on large models and retrieval optimization provided by the present invention may also have such technical features, wherein, in step S3-2, the collection knowledge base is searched using a hybrid retrieval method, including: for the query input by the customer, the vector retrieval method and the BM25 retrieval method are used to search the collection knowledge base based on the query after vectorized encoding, respectively, to obtain respective retrieval results, and the retrieval results include multiple text blocks; the retrieval results of the two retrieval methods are integrated through a reordering algorithm, and the correlation between the text blocks and the query during reordering is calculated, and the text blocks with the highest correlation are used as the final retrieval results.

[0012] The business processing method based on large model and retrieval optimization provided by the present invention may also have such a technical feature, wherein the reordering algorithm is a reciprocal sorting fusion algorithm.

[0013] The business processing method based on large models and retrieval optimization provided by the present invention may also have such a technical feature, wherein, in step S3-3, it is determined whether the relevance between the m text blocks and the query reaches a predetermined relevance threshold.

[0014] The business processing method based on large models and retrieval optimization provided by the present invention may also have such technical features, the method also includes: step S4, after the interaction with the customer is completed, generating a collection risk assessment analysis result for the customer based on the query and the query results and the risk grading model.

[0015] The present invention provides a business processing system based on a large model and retrieval optimization, which has the following technical features: the system includes: a storage module for storing a debt collection knowledge base and a fine-tuned large language model, wherein the debt collection knowledge base is constructed based on a variety of collected debt collection-related data; a parameter efficient fine-tuning module for fine-tuning the pre-trained base large language model through a quantized low-rank adaptation technology to obtain the fine-tuned large language model and store it in the storage module; a customer interaction module for interacting with customers, allowing the customers to input queries and providing the customers with corresponding query results; an enhanced query technology module , used to search the debt collection knowledge base using retrieval enhancement generation technology based on the query input by the customer; an enhanced query optimization module, used to determine whether to perform synonym enhanced retrieval based on the retrieval results of the enhanced query technology module, and when the judgment is yes, use synonym query enhancement technology to generate multiple synonymous queries of the query, and perform synonym enhanced retrieval on the debt collection database based on the query and the multiple synonymous queries; and a query result generation module, used to generate the query result using the large language model based on the retrieval results of the enhanced query technology module or the retrieval results of the enhanced query optimization module.

[0016] Functions and effects of the invention

[0017] The business processing method and system based on large models and retrieval optimization provided by the present invention have the following outstanding advantages over the debt collection business processing methods and systems in the prior art:

[0018] First, strategy flexibility and personalization are significantly improved. By organically combining large language model fine-tuning with Retrieval Enhanced Generation (RAG) technology, the system of this invention can dynamically and accurately generate personalized business processing strategies based on the specific circumstances of the debtor and real-time knowledge related to debt collection. This completely solves the problem that traditional debt collection strategies rely on preset rules and lack flexibility and personalization, significantly improving the targetedness and effectiveness of business processing.

[0019] Second, knowledge updates are timely and accurate. The application of RAG technology enables dynamic updates of debt collection-related knowledge. The system of the present invention can access a debt collection knowledge base containing the latest policies in real time, ensuring that business processing strategies and responses always comply with the latest regulatory requirements and business knowledge. This effectively solves the problem of lagging knowledge updates and ensures the compliance and timeliness of debt collection services.

[0020] Third, search recall rates have been significantly improved. The synergy between synonym query enhancement and hybrid search strategies significantly improves search recall rates, enabling large language models to accurately extract relevant knowledge from a small amount of knowledge resources. This significantly enhances the accuracy and reliability of answers, providing stronger support for debt collection operations and effectively improving debt collection effectiveness.

[0021] Fourth, business processing efficiency and coverage have been significantly improved. Through the automation of the entire intelligent business processing process, the collection processing time has been significantly shortened, the collection rate has been significantly improved, and full-cycle coverage of overdue cases has been achieved. This has completely solved the problems of low efficiency and insufficient coverage in traditional business processing, and fully met the growing business needs of financial institutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the structure of a business processing system based on a large model and retrieval optimization in an embodiment of the present invention;

[0023] Figure 2 This is a flow chart of a business processing method based on a large model and retrieval optimization in an embodiment of the present invention;

[0024] Figure 3 is a flow chart of step S1 in an embodiment of the present invention;

[0025] Figure 4 is a flow chart of step S2 in an embodiment of the present invention;

[0026] Figure 5 is a flow chart of step S3 in an embodiment of the present invention;

[0027] Figure 6 is a schematic diagram of hybrid retrieval in an embodiment of the present invention;

[0028] Figure 7 is a schematic diagram of a reordering algorithm according to an embodiment of the present invention;

[0029] Figure 8 is a processing flow chart of a general query in an embodiment of the present invention;

[0030] Figure 9 This is a flowchart of processing a special query in an embodiment of the present invention.

[0031] Reference numerals:

[0032] A business processing system based on a large model and retrieval optimization 10; a storage module 11; a debt collection knowledge base 111; a model storage unit 112; a query data storage unit 113; a parameter efficient fine-tuning module 12; a RAG technology module 13; a RAG optimization module 14; a query result generation module 15; a customer interaction module 16; a risk assessment and analysis module 17; and a control module 18. DETAILED DESCRIPTION

[0033] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the business processing method and system based on large models and retrieval optimization of the present invention are specifically explained below in combination with embodiments and drawings.

[0034] <Example>

[0035] Figure 1 This is a schematic diagram of the structure of the business processing system based on large models and retrieval optimization in this embodiment.

[0036] like Figure 1 As shown, the business processing system 10 based on large models and retrieval optimization (hereinafter referred to as the business processing system 10) of this embodiment includes a storage module 11, a parameter efficient fine-tuning module 12, a retrieval enhancement generation (RAG) technology module 13, a retrieval enhancement generation (RAG) optimization module 14, a query result generation module 15, a customer interaction module 16, a risk assessment and analysis module 17 and a control module 18.

[0037] The storage module 11 is used to store the required data and models, and includes a debt collection knowledge base 111 , a model storage unit 112 , and a query data storage unit 113 .

[0038] The collection knowledge base 111 is used to store various collection-related data. In this embodiment, it is used to store vectorized encoded text and indexes. Optionally, the collection knowledge base 111 can also be used to store various business processing strategies for different customer types generated based on the above collection-related data.

[0039] The model storage unit 112 is used to store the trained (fine-tuned) large language model.

[0040] The query data storage unit 113 is used to store queries input by customers and query results generated by the system so that these data can be further utilized later.

[0041] The parameter efficient fine-tuning module 12 is used to efficiently fine-tune the pre-trained base large language model to obtain a large language model fine-tuned for the debt collection business, and store the fine-tuned large language model in the model storage unit 112.

[0042] The RAG technology module 13 is used to search the debt collection knowledge base 111 using the Retrieval Enhancement Generation (RAG) technology based on the query input by the customer.

[0043] The RAG optimization module 14 is used to generate multiple synonymous queries of the original query using the synonym query enhancement technology when the RAG technical module 13 cannot obtain an ideal search result, and search the debt collection knowledge base 111 based on the original query and the multiple synonymous queries.

[0044] The query result generation module 15 generates a query result based on the original query, the search result of the RAG technology module 13 or the search result of the RAG optimization module 14 using the fine-tuned large language model, and stores it in the query data storage unit 113.

[0045] The customer interaction module 16 is used to implement interaction with customers. For example, the customer interaction module 16 can provide an interactive input and display device for customers to input queries and display query results to customers; or, the customer interaction module 16 can communicate remotely with the terminal held by the customer, obtain queries from the customer terminal and send query results to the customer terminal.

[0046] The risk assessment and analysis module 17 is used to generate a debt collection risk assessment and analysis result for the customer based on the customer's query and query results and the risk grading model after the customer interaction module 16 completes its interaction work, and store it in the query data storage unit 113.

[0047] The control module 18 is used to control the operation of the above modules, including controlling the efficient parameter fine-tuning module 12 to store the fine-tuned model in the storage module 11 after the fine-tuning of the model is completed; controlling the customer interaction module 16 to provide the query results to the customer after the query result generation module 15 generates the query results, and so on.

[0048] The specific algorithms involved in each of the above modules will be described in detail in the following corresponding methods.

[0049] Figure 2 This is a flowchart of the business processing method based on large models and retrieval optimization in this embodiment.

[0050] like Figure 2 As shown, based on the above-mentioned business processing device 10, the business processing method based on large model and retrieval optimization of this embodiment includes the following steps:

[0051] Step S1, building a debt collection knowledge base based on the collected various debt collection related data;

[0052] Step S2: fine-tune the pre-trained base large language model using quantized low-rank adaptation technology to obtain a fine-tuned large language model;

[0053] Step S3: interact with the customer, search the debt collection knowledge base using search enhancement generation technology based on the query input by the customer, and generate query results based on the search results using a large language model;

[0054] Step S4: After the interaction with the customer is completed, a collection risk assessment analysis result for the customer is generated based on the query and query results and the risk grading model.

[0055] The above steps will be described in detail below.

[0056] Step S1: Build a collection knowledge base based on the collected collection-related data. The collection-related data includes legal and regulatory texts, policy texts, collection process specification texts, expert speech experience texts, historical collection case data of multiple customers, system usage documents, and the bank's latest activity strategy data.

[0057] Figure 3 This is a flow chart of step S1 in this embodiment.

[0058] like Figure 3 As shown, step S1 specifically includes the following sub-steps:

[0059] In step S1-1, comprehensive data preprocessing is performed on the collected various collection-related data, including key operations such as noise data cleaning, stop word removal, duplicate data filtering, and text segmentation.

[0060] Step S1-2: Use a vector model to perform accurate vector encoding and index construction on the divided text, wherein the vector model may be, for example, a general vector model BGE-M3.

[0061] In step S1-3, the vectorized text and index are stored in a powerful vector database, serving as a debt collection knowledge base. For example, Elasticsearch can be used as the vector database, easily deployed via Docker. Elasticsearch boasts a powerful distributed architecture, a mature ecosystem, and comprehensive security mechanisms. It also seamlessly supports both vector and keyword searches, providing a solid foundation for subsequent searches and knowledge acquisition.

[0062] Optionally, based on the debt collection knowledge base, customized business processing strategies can be accurately generated for different types of customer groups, and the generated strategies can also be stored in the debt collection knowledge base.

[0063] Through step S1, the above-mentioned debt collection knowledge base 111 can be constructed.

[0064] Step S2: Fine-tune the pre-trained base large language model using quantized low-rank adaptation (QLoRA) technology to obtain a fine-tuned large language model.

[0065] Figure 4 This is a flow chart of step S2 in this embodiment.

[0066] like Figure 4 As shown, step S2 specifically includes the following sub-steps:

[0067] Step S2-1, obtain a pre-trained base large language model, the pre-trained base large language model may be, for example, Qwen2.5-72B.

[0068] Step S2-2, fix the model parameters of the pre-trained base large language model, and initialize the weight parameters of the two low-rank matrices of the low-rank adaptation technology (LoRA method).

[0069] That is, in this embodiment, the model parameters of the pre-trained base large language model are fixed, and only the weight parameters of the two newly added low-rank matrices A and B (introduced by the LoRA method) are optimized. Among them, the weight parameters of the low-rank matrix A are initialized by a carefully designed Gaussian function (zero-mean normal distribution), while the weight parameters of the low-rank matrix B are initialized to a zero matrix. Such a design can ensure that when the model training (fine-tuning) starts, the newly added pathway has no effect on the model results. In the inference stage, the final inference result can be obtained by simply adding the results of the pre-trained base large language model and the newly added pathway.

[0070] In step S2-3, based on the weight parameters of the two initialized low-rank matrices and the collection business data, the pre-trained base large language model is fine-tuned by the QLoRA method to obtain a fine-tuned large language model.

[0071] The collection business data used for model training (fine-tuning) is primarily obtained from desensitized conversation records, training documents, legal and regulatory documents, and collection logs from real-world collection business scenarios. The model includes five scenarios: collection business knowledge Q&A, security and compliance identification, text comprehension, logical calculation, and content generation. The content generation scenario consists of subjective questions, while all other scenarios are objective. Each scenario is further divided into multiple subtasks. The collection business knowledge Q&A scenario includes subtasks such as collection laws and regulations, collection process specifications, basic financial knowledge, and credit card knowledge. The security and compliance identification scenario includes subtasks such as collection compliance policies, profanity and insults, personal privacy, and role-playing attacks. The text comprehension scenario is further divided into subtasks such as customer intent recognition, emotional state analysis, repayment willingness assessment, and customer complaint tendency judgment. The logical calculation scenario is further divided into subtasks such as repayment amount calculation, installment interest rate calculation, and date calculation. The content generation scenario includes subtasks such as collection speech, collection strategy, collection conclusion, and follow-up collection suggestions.

[0072] The above datasets are uniformly stored in JSON format. Objective questions contain fields such as question, options, answers, scenario name, and subtask name. Subjective questions contain fields such as question, answer, knowledge source, scenario name, and subtask name.

[0073] The goal of fine-tuning is to improve the generalization ability of the general large model in debt collection scenarios.

[0074] QLoRA technology, based on LoRA, innovatively introduces advanced strategies such as the 4-bit NormalFloat (NF4) data type, NF4 double quantization, and a paging optimizer. Compared to traditional LoRA technology, QLoRA technology significantly reduces video memory usage by 300%. This enables fine-tuning of a 72-byte parameter model on a single GPU with 80GB of video memory while maintaining excellent task performance comparable to LoRA fine-tuning. This effectively addresses the issue of inaccurate responses to customer inquiries from large language models based on a lack of debt collection expertise.

[0075] Step S2-4: Save the fine-tuned large language model.

[0076] Step S2 is also the specific algorithm involved in the above-mentioned parameter efficient fine-tuning module 12. Figure 2 As shown, step S1 and step S2 can be performed simultaneously.

[0077] Step S3: interact with the customer (debtor), search the debt collection knowledge base through the retrieval enhancement generation (RAG) technology according to the query input by the customer, and generate query results based on the retrieval results using the large language model.

[0078] Figure 5 This is a flow chart of step S3 in this embodiment.

[0079] like Figure 5 As shown, step S3 specifically includes the following sub-steps:

[0080] Step S3-1: Receive a query input by a customer and use a vector model to quickly vectorize and encode the query.

[0081] The query input by the customer is also the question raised by the customer (debtor), and the system vectorizes the text of the question.

[0082] In step S3-2, based on the vectorized encoded query, the top m text blocks with the highest relevance are retrieved in the collection knowledge base (vector database) through RAG technology, where m is the predetermined number of text blocks to be retrieved, which can be set as needed to balance the answer accuracy and processing efficiency.

[0083] Figure 6 Schematic diagram of hybrid retrieval in this embodiment.

[0084] like Figure 6 As shown, in order to further improve the search effect, this embodiment adopts a hybrid search method that combines vector search and BM25 search.

[0085] Vector retrieval is primarily based on calculating similarity between vectors. Using a vector model, customer queries and document sets are vectorized and encoded. The similarity between the query vector and each vector stored in the index is then calculated to quickly find the vector most similar to the query vector, enabling efficient retrieval. This retrieval method excels at understanding complex questions and retrieving long texts. It also boasts cross-language processing capabilities, meeting the retrieval needs of multilingual content.

[0086] BM25 retrieval is a document ranking algorithm based on probability theory, widely used in information retrieval and search engines. It calculates document relevance by precisely measuring the degree of match between query terms and document terms. This makes it particularly advantageous for accurately searching for specific case numbers (e.g., "CS2023-JD-0567") or industry-specific terms (e.g., "M1 Overdue Grade" and "Lost Contact Repair").

[0087] In this embodiment, for the query input by the customer, the vector retrieval method and the BM25 retrieval method are used to search in the collection knowledge base based on the vectorized encoded query, and the respective retrieval results (the most relevant text blocks matched) are obtained. Then, the retrieval results of the two retrieval methods are integrated through the reranking algorithm, and the relevance score of the text block with the original query during reranking is calculated. Finally, the text blocks with the highest relevance scores are output as the final retrieval results for use in subsequent processes.

[0088] Figure 7 Schematic diagram of the reordering algorithm in this embodiment.

[0089] like Figure 7 As shown, in this embodiment, the reordering algorithm is the Reciprocal Ranking Fusion (RRF) algorithm. The RRF algorithm is an efficient algorithm widely used in the fields of meta-search and information retrieval. It is mainly suitable for fusing results from multiple search systems or algorithms. The uniqueness of this algorithm is that it does not directly focus on the specific relevance score of each result, but focuses on the ranking position of the results in their respective lists. The calculation formula of the RRF algorithm is:

[0090]

[0091] Where d represents the text block, R is the set of retrievers, r(d) is the ranking of the text block in a certain retriever, and k is a constant, usually set to 60.

[0092] In practice, the RRF algorithm first identifies all unique text blocks (documents) retrieved by all search engines. Then, for each text block, it calculates the inverse of its ranking position in the search list and sums all these inverses to obtain the fusion score for that text block. If a text block does not appear in a search list, the corresponding inverse score for that document is 0. Finally, all text blocks are sorted in descending order based on the fusion score, completing an efficient re-ranking operation.

[0093] Step S3-2 is the specific algorithm involved in the above-mentioned RAG technical module 13.

[0094] Step S3-3, judging whether a synonym enhanced search is required based on the search result of step S3-2, and proceeding to step S3-4 if the judgment is yes, and proceeding to step S3-6 if the judgment is no.

[0095] For example, for the m text blocks obtained in step S3-2, it can be determined whether the relevance of these text blocks to the original query reaches a predetermined relevance threshold. If the relevance is negative, the search result is unsatisfactory, indicating that the query (question) raised by the client (debtor) may be ambiguous and difficult to capture, and a synonym-enhanced search is therefore required. Therefore, in this embodiment, different search strategies are used for different questions. In addition, in alternative solutions, other methods can also be used to automatically determine whether the question raised by the client is ambiguous and difficult to capture.

[0096] Step S3-4: Generate N synonymous queries of the original query using synonym query enhancement technology, where N is a predetermined number of synonymous queries and can be set as needed.

[0097] Specifically, the original query is input into the fine-tuned large language model, and the powerful capabilities of the large language model are used to generate N synonymous queries of the original query. These synonymous queries can fully capture the various expressions of the original query (question).

[0098] In step S3-5, the synonymous query is quickly vectorized and encoded using the vector model, and based on the vectorized original query and N synonymous queries, the top m text blocks with the highest relevance are retrieved in the debt collection knowledge base (vector database) using RAG technology.

[0099] Steps S3 - 3 to S3 - 5 are the specific algorithms involved in the above-mentioned RAG optimization module 14 .

[0100] In step S3-6, the m text blocks retrieved are rationally arranged with the original query text using a predetermined prompt template to obtain prompt words, and the prompt words are input into the fine-tuned large language model, which performs deep reasoning to generate accurate and effective query results (i.e., answers to questions raised by customers).

[0101] Step S3 - 6 is the specific algorithm involved in the query result generating module 15 .

[0102] Figure 8 This is a processing flow chart of a general query in this embodiment.

[0103] like Figure 8 As shown, for general queries (questions that are clearly stated and relatively easy to understand), the process of step S3 can be simplified to Figure 5 The process shown is to generate corresponding prompt words based on the RAG search results and directly input them into the fine-tuned large language model to obtain the query results.

[0104] For example, if a customer enters a query (question) such as "How do I apply for installment payments?", the above method can generate an accurate and effective response: "You can apply for installment payments on the debt management page through the official website, mobile app, or WeChat official account, select the relevant information, and submit the application." Because the debt collection knowledge base also includes data on the bank's latest campaign strategies, including text and data related to fees, if a customer asks a question about fee details, the large language model can automatically calculate the fee details based on the search results and provide the calculated fee details to the customer.

[0105] Figure 9 This is a flowchart of the processing of special queries in this embodiment.

[0106] like Figure 9 As shown, for special queries (questions with vague expressions, difficult to capture intentions, or complex situations), multiple synonymous queries will be generated through the large language model as described above, and then retrieval will be performed based on the original query and its multiple synonymous queries to improve the accuracy of the answer.

[0107] Step S4: After the interaction with the customer is completed, a collection risk assessment analysis result for the customer is generated based on the query and query results and the risk grading model.

[0108] Specifically, a comprehensive and multi-dimensional evaluation of the business processing effect is conducted based on the risk grading model, and the customer's repayment willingness label and complaint warning signal are accurately extracted based on the customer's query and query results, the customer's historical collection data, etc., and based on the extracted labels and signals, an intelligent analysis report is finally generated that includes the optimization level, contact strategy and compliance recommendations for the customer's subsequent business processing.

[0109] Step S4 is the specific algorithm involved in the risk assessment and analysis module 17 .

[0110] The above methods can be summarized as follows:

[0111] During the business preparation stage, the compliance speech system and collection strategy template based on the industry knowledge base fully utilize the powerful advantages of RAG technology, dynamically integrate key information such as historical collection case analysis, compliance speech experience and risk management specifications, and accurately generate customized business processing strategies for different types of customer groups.

[0112] During the execution phase, the fine-tuned large-scale model, leveraging an intelligent voice interaction platform, efficiently and accurately completes the entire automated process, including customer identity verification, debt confirmation, intent recognition, emotion perception, and compliance verification. When customers raise complex requests, such as installment plan negotiations, the system leverages RAG technology to access the latest policy database, including pandemic relief programs and interest rate promotions, to quickly generate structured installment plans and automatically calculate detailed expenses, enabling flexible and efficient negotiation responses.

[0113] During the business completion stage, the system conducts a comprehensive and multi-dimensional assessment of the business processing effect based on the risk grading model, accurately extracts customer repayment willingness labels and complaint warning signals, and ultimately generates an intelligent analysis report containing subsequent business processing priorities, contact strategies and compliance recommendations, providing strong support for subsequent business decisions.

[0114] Functions and Effects of the Embodiments

[0115] The business processing method and system based on a large model and retrieval optimization provided in this embodiment have the following outstanding advantages over the debt collection business processing methods and systems in the prior art:

[0116] First, strategy flexibility and personalization are significantly improved. By organically combining large language model fine-tuning with RAG technology, the system of this embodiment can dynamically and accurately generate personalized business processing strategies based on the specific circumstances of the debtor and real-time knowledge related to debt collection. This completely solves the problem that traditional debt collection strategies rely on preset rules and lack flexibility and personalization, significantly improving the targetedness and effectiveness of business processing.

[0117] Second, knowledge updates are timely and accurate. The application of RAG technology enables dynamic updates of debt collection-related knowledge. The system can access the debt collection knowledge base containing the latest policies in real time, ensuring that business processing strategies and responses always comply with the latest regulatory requirements and business knowledge. This effectively solves the problem of lagging knowledge updates and ensures the compliance and timeliness of debt collection services.

[0118] Third, search recall rates have been significantly improved. The synergy between synonym query enhancement and hybrid search strategies significantly improves search recall rates, enabling large language models to accurately extract relevant knowledge from a small amount of knowledge resources. This significantly enhances the accuracy and reliability of answers, providing stronger support for debt collection operations and effectively improving debt collection effectiveness.

[0119] Fourth, business processing efficiency and coverage have been significantly improved. Through the automation of the entire intelligent business processing process, the collection processing time has been significantly shortened, the collection rate has been significantly improved, and full-cycle coverage of overdue cases has been achieved. This has completely solved the problems of low efficiency and insufficient coverage in traditional business processing, and fully met the growing business needs of financial institutions.

[0120] In the above embodiment, a quantization strategy (QLoRA technology) was adopted when fine-tuning the pre-trained base large language model, and a unique strategy of fixing model parameters and optimizing only the low-rank matrix was adopted. While greatly reducing the computing resource usage, it significantly improved the accuracy and performance of the large model in the professional field of debt collection.

[0121] In addition, in the above embodiment, it is possible to automatically determine whether the questions raised by customers are vague in expression or difficult to capture in intention based on the RAG search results and the predetermined threshold, and automatically apply the synonym query enhancement search strategy to improve the accuracy of the generated answers when it is determined that these situations exist. Therefore, the degree of intelligence is very high and the need for manual intervention is lower.

[0122] The above embodiments are merely illustrative of specific implementations of the present invention, and the present invention is not limited to the scope of the description of the above embodiments. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention as claimed. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A business processing method based on large model and retrieval optimization, characterized in that: The following steps are involved: Step S1, building a debt collection knowledge base based on the collected various debt collection related data; Step S2: fine-tune the pre-trained base large language model using quantized low-rank adaptation technology to obtain a fine-tuned large language model; Step S3: interact with the customer, search the debt collection knowledge base according to the query input by the customer, and generate query results based on the search results using the large language model. Among them, in step S3, the collection database is searched based on the query using the retrieval enhancement generation technology, and it is determined whether to perform a synonym enhanced retrieval based on the retrieval results. When the judgment is yes, multiple synonymous queries of the query are generated using the synonym query enhancement technology, and the collection database is searched based on the query and the multiple synonymous queries. Then, the query result is generated based on the retrieval result or the result of the synonym enhanced retrieval using the large language model.

2. The business processing method based on large model and retrieval optimization according to claim 1 is characterized in that: in, Step S1 includes the following sub-steps: Step S1-1: Preprocessing the various types of collection-related data. The collection-related data includes legal and regulatory documents, policy documents, collection process specifications, expert experience documents, historical collection case data for multiple clients, system usage documentation, and the bank's latest campaign strategy data. The data preprocessing includes noise data cleaning, stop word removal, duplicate data filtering, and text segmentation. Step S1-2, using a vector model to perform vectorized encoding and index construction on the divided text to obtain vectorized encoded text blocks and indexes; Step S1-3: storing the text block and the index in a vector database as the debt collection knowledge base.

3. The business processing method based on large model and retrieval optimization according to claim 2 is characterized in that: in, Step S2 includes the following sub-steps: Step S2-1, obtaining the pre-trained base large language model; Step S2-2, fixing the model parameters of the pre-trained base large language model and initializing the weight parameters of the two low-rank matrices of the low-rank adaptation technology; Step S2-3, based on the weight parameters of the two initialized low-rank matrices and the collection business data, the pre-trained base large language model is fine-tuned through the quantized low-rank adaptation technology to obtain the fine-tuned large language model.

4. The business processing method based on large model and retrieval optimization according to claim 3 is characterized by: in, In step S2-2, the two low-rank matrices are low-rank matrices A and B respectively. The weight parameters of the low-rank matrix A are initialized by a Gaussian function, The weight parameters of the low-rank matrix B are initialized to a zero matrix.

5. The business processing method based on large model and retrieval optimization according to claim 2, Its characteristics are: Wherein, step S3 includes the following sub-steps: Step S3-1, receiving the query input by the customer, and vectorizing and encoding the query using the vector model; Step S3-2, based on the vectorized encoded query, using the search enhancement generation technology to search the debt collection knowledge base for the top m text blocks with the highest relevance; Step S3-3, judging whether a synonym enhanced search is required based on the search result of step S3-2, and proceeding to step S3-4 if the judgment is yes, and proceeding to step S3-6 if the judgment is no; Step S3-4, generating N synonymous queries of the query by using synonymous query enhancement technology; Step S3-5: vectorize the synonymous query using the vector model, and based on the vectorized query and the N synonymous queries, retrieve the top m text blocks with the highest relevance in the debt collection knowledge base using the search enhancement generation technology; Step S3-6: Arrange the m text blocks and the query text using a predetermined prompt word template, and input the arranged text into the large language model to generate the query result.

6. The business processing method based on large model and retrieval optimization according to claim 5 is characterized in that: in, In step S3-2, the debt collection knowledge base is searched using a hybrid search method, including: For the query input by the customer, searching the debt collection knowledge base using a vector search method and a BM25 search method based on the vectorized encoded query, respectively, to obtain respective search results, wherein the search results include the plurality of text blocks; The search results of the two search methods are integrated by a re-ranking algorithm, and the relevance between the text blocks and the query during re-ranking is calculated, and the text blocks with the highest relevance are used as the final search results.

7. The business processing method based on large model and retrieval optimization according to claim 6 is characterized in that: in, The reordering algorithm is a reciprocal sorting fusion algorithm.

8. The business processing method based on large model and retrieval optimization according to claim 5 is characterized by: in, In step S3-3, it is determined whether the relevance between the m text blocks and the query reaches a predetermined relevance threshold.

9. The business processing method based on large model and retrieval optimization according to claim 1 is characterized in that: Also includes: Step S4: After the interaction with the customer is completed, a collection risk assessment analysis result for the customer is generated based on the query, the query result, and the risk grading model.

10. A business processing system based on large models and retrieval optimization, characterized in that: include: A storage module, configured to store a debt collection knowledge base and a fine-tuned large language model, wherein the debt collection knowledge base is constructed based on a variety of collected debt collection-related data; A parameter efficient fine-tuning module, configured to fine-tune the pre-trained base large language model by using a quantized low-rank adaptation technique to obtain the fine-tuned large language model and store it in the storage module; A customer interaction module, configured to interact with customers, allow the customers to input queries and provide corresponding query results to the customers; A search enhancement generation technology module, configured to search the debt collection knowledge base using search enhancement generation technology according to the query input by the customer; a search enhancement generation optimization module, configured to determine whether to perform a synonym enhancement search based on the search results of the search enhancement generation technology module, and if the determination is yes, to generate multiple synonymous queries of the query using a synonym query enhancement technology, and to perform a synonym enhancement search on the debt collection database based on the query and the multiple synonymous queries; and A query result generation module is used to generate the query result using the large language model according to the retrieval result of the retrieval enhancement generation technology module or the retrieval result of the retrieval enhancement generation optimization module.

Images