Product classification recommendation method and system and storage medium

Through the combination of large language model and fine sorting model, financial product categories and candidate products are obtained, which solves the problem of difficulty in personalizing financial product recommendations in the existing technology, and achieves higher recommendation accuracy and personalization.

CN119027220AInactive Publication Date: 2024-11-26ZHEJIANG FULIN TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411510380.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-28
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing financial product classification recommendation model is difficult to fully capture all the characteristics of user data, making it difficult for the screened financial products to meet the personalized needs of users.

Method used

Using a combination of large language model and fine sorting model, by constructing user information prompt statements, obtain financial product categories and candidate financial products that meet user needs, and match user information characteristics with candidate financial product characteristics to generate users in different orders. Product group, enter the fine sorting model for scoring to obtain the final financial recommended product.

Benefits of technology

It improves the accuracy and personalization of financial product recommendations to meet the diverse needs of users.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027220A_ABST
    Figure CN119027220A_ABST
Patent Text Reader

Abstract

The invention discloses a product classification recommendation method and system and a storage medium, and the method comprises the steps: obtaining a user information construction prompt statement of a target user in a target recommendation scene in response to a recommendation request, inputting the prompt statement into a large language model after fine tuning training, obtaining a plurality of candidate financial product categories meeting the demands of the target user, and carrying out the recommendation of the candidate financial products. According to each candidate financial product category, screening a plurality of candidate financial products conforming to the user information in a corresponding category product knowledge base, and adding each category of candidate financial products into a candidate financial product queue, and pairing each user information feature with each candidate financial product feature in the candidate financial product queue by adopting a first sequence and a second sequence to form a plurality of user product groups, and finally performing sorting scoring on each user product group by using a trained fine sorting model. And obtaining the financial products in the set recommendation number of the user product groups which are ranked in the front as final recommendation products, so as to improve the accuracy of product classification recommendation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a product classification recommendation method, system and storage medium. Background Art

[0002] One of the current trends in financial services is that financial institutions recommend financial products to users by using product classification recommendation models in combination with user needs. Through product classification recommendation models, financial institutions can better understand and grasp customer needs, push products more accurately and effectively, and improve the conversion rate of customer purchase behavior. However, most of the product classification recommendation models currently on the market only use a single machine learning model to analyze and predict user behavior data, which often makes it difficult to fully capture all the characteristics of user data, and the selected financial products often fail to meet the personalized needs of users. Summary of the invention

[0003] In view of the deficiencies in the prior art, the present invention provides a product classification recommendation method, comprising the following steps: S1, responding to the recommendation request, obtaining the target user's user information for the target recommendation scenario to construct a prompt sentence, inputting it into the fine-tuned large language model, and obtaining multiple candidate financial product categories that meet the needs of the target user; S2, according to each candidate financial product category, screen and obtain multiple candidate financial products that meet the user information in the corresponding category product knowledge base, and add the candidate financial products of each category to the candidate financial product queue; S3, respectively pairing each user information feature with each candidate financial product feature in the candidate financial product queue in a first order and a second order to form a plurality of user product groups, wherein the user product group includes a user information feature and a candidate financial product feature, wherein the first order is to put the user information feature first and the candidate financial product feature second in the user product group, and the second order is to put the candidate financial product feature first and the user information feature second in the user product group; S4, using the trained fine ranking model to rank and score each user product group, and obtaining the financial products in the user product groups with a set number of recommendations that are ranked first as the final recommended products.

[0004] Preferably, the fine-tuning training steps of the large language model are as follows: obtaining historical financial product recommendation transaction records, wherein each transaction record includes but is not limited to basic information of the personnel, work information, credit record and financial status, and the corresponding label is the name of the financial product traded by the user and the financial product category to which it belongs; dividing the data set into a training sample set, a verification sample set and a test sample set according to the financial product category of the label, and then performing supervised fine-tuning training on the text classification model.

[0005] Preferably, the user information features include historical conversation information, financial information, work information and credit information, and the financial product features include product descriptions, product specifications and product requirements. The specific training steps of the precise ranking model are as follows: user information features and transaction characteristics of financial products in historical financial product recommendation transaction records are recommended in advance, and the user information features are split into a first user feature group and a second user feature group, wherein the first user feature group includes historical conversation information of the target user, and the second user feature group includes financial information, work information and credit information; and financial product features are split into a first product feature group and a second product feature group, wherein the first product feature group includes product descriptions of corresponding financial products, and the second product feature group includes product descriptions of corresponding financial products. Product specifications and product requirements; constructing the user information features and the features of the traded financial products in each historical financial product recommendation transaction record into four positive training data, wherein the four positive training data are the first user feature group and the first product feature group, the first user feature group and the second product feature group, the second user feature group and the first product feature group, and the second user feature group and the second product feature group; replacing the financial product features in the historical financial product recommendation transaction records with other financial product categories or with products with large specification differences in the same financial product category, and constructing the user information features and the replaced financial product features into four negative training data; mixing the positive training data and the negative training data to form a training set, a validation set and a test set according to a set ratio, and then inputting them into a precise sorting model for training.

[0006] Preferably, the precise sorting model includes an embedding input layer, an encoder layer and a loss function layer, wherein the embedding input layer includes a text embedding layer, a position embedding layer and a token type embedding layer, wherein the encoder layer is composed of 24 consecutive Transformer Encoder layers stacked together, and the loss function layer is configured to communicate with a first loss function and a second loss function, wherein the first loss function is configured to determine the semantic relevance of user information and product information, and the second loss function is configured to determine the word order of user information and financial product information.

[0007] Preferably, the first loss function is: ; The second loss function is: ; The final loss function is configured as: ; in and Represents two labels respectively. As learnable parameters, they are used to represent the weights of the two parts of the loss.

[0008] The present invention also discloses a product classification recommendation system, including a category acquisition module, a product screening module, a feature matching module and a refined ranking recommendation module, wherein the category acquisition module is used to respond to recommendation requests, obtain user information of the target user in the target recommendation scenario for constructing a prompt sentence, and input it into a large language model after fine-tuning training to obtain multiple candidate financial product categories that meet the needs of the target user; the product screening module is used to screen and obtain multiple candidate financial products that meet the user information in the corresponding category product knowledge base according to each candidate financial product category, and add the candidate financial products of each category to the candidate financial product queue; the feature matching module is used to A first order and a second order are respectively used to pair each user information feature with each candidate financial product feature in the candidate financial product queue to form a plurality of user product groups, wherein the user product group includes a user information feature and a candidate financial product feature, wherein the first order is to rank the user information feature first and the candidate financial product feature second in the user product group, and the second order is to rank the candidate financial product feature first and the user information feature second in the user product group; a refined ranking recommendation module is used to rank and score each user product group using a refined ranking model that has been trained, and obtain financial products in the user product group with a set number of recommendations ranked first as final recommended products.

[0009] Preferably, it also includes a large language model fine-tuning training module, specifically including a record acquisition module and a fine-tuning training module, wherein the record acquisition module is used to obtain historical financial product recommendation transaction records, wherein each transaction record includes but is not limited to basic personnel information, work information, credit record and financial status, and the corresponding label is the name of the financial product traded by the user and the financial product category to which it belongs; the fine-tuning training module is used to split the data set into a training sample set, a verification sample set and a test sample set according to the labeled financial product category, and then perform supervised fine-tuning training on the text classification model.

[0010] Preferably, the user information features include historical conversation information, financial situation information, work situation information and credit situation information, the financial product features include product description, product specifications and product requirements, and also include a precise sorting model training module, specifically including a feature splitting module, a positive example construction module, a negative example construction module and a model training module, wherein the feature splitting module is used to recommend user information features and transaction financial product features in historical financial product transaction records in advance, split the user information features into a first user feature group and a second user feature group, the first user feature group includes the historical conversation information of the target user, and the second user feature group includes financial situation information, work situation information and credit situation information; split the financial product features into a first product feature group and a second product feature group, the first product feature group includes the product description of the corresponding financial product, and the second product feature group includes the In response to the product specifications and product requirements of financial products; a positive example construction module, used to construct the user information features and the features of the traded financial products in each historical financial product recommendation transaction record into four positive example training data, wherein the four positive example training data are the first user feature group and the first product feature group, the first user feature group and the second product feature group, the second user feature group and the first product feature group, and the second user feature group and the second product feature group; a negative example construction module, used to replace the financial product features in the historical financial product recommendation transaction record with other financial product categories or with products with large specification differences in the same financial product category, and construct the user information features and the replaced financial product features into four negative example training data; a model training module, used to mix the positive example training data and the negative example training data, and then form a training set, a validation set and a test set according to a set ratio, and then input them into a precise sorting model for training.

[0011] The present invention also discloses a server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the aforementioned methods when executing the computer program.

[0012] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the aforementioned methods are implemented.

[0013] The product classification recommendation method, system and storage medium disclosed in the present invention combine a large language model and a fine ranking model, recommend products that meet user needs based on a full understanding of user needs and in combination with the characteristics of each financial product, and obtain financial product categories and candidate financial products that meet user needs by constructing a user information prompt sentence in a target recommendation scenario and inputting it into a fine-tuned and trained large language model, and then matching user information features with candidate financial product features to generate user product groups in different orders and inputting them into the trained fine ranking model for scoring, thereby obtaining financial recommendation products that ultimately meet user needs, thereby improving the accuracy and personalization of recommendations by the product classification recommendation model in financial scenarios.

[0014] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 A schematic diagram of the steps of a product classification recommendation method disclosed in one embodiment of the present invention.

[0016] Figure 2 This is a fine-tuning training step for the large language model disclosed in an embodiment of the present invention.

[0017] Figure 3 It is a schematic diagram of the structure of the bge-reranker-large model disclosed in one embodiment of the present invention.

[0018] Figure 4 A schematic diagram of the steps of constructing a bge-reranker-large fine-tuning dataset disclosed in an embodiment of the present invention.

[0019] Figure 5 A schematic diagram of the structure of a product classification recommendation system disclosed in one embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0022] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0023] Unless otherwise defined, the technical or scientific terms used herein shall have the common meanings understood by persons with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the patent application specification and claims of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "a" do not indicate a quantity limitation, but indicate the existence of at least one.

[0024] One of the current trends in financial services is that financial institutions recommend financial products to users by using product classification recommendation models in combination with user needs. Through product classification recommendation models, financial institutions can better understand and grasp customer needs, push products more accurately and effectively, and improve the conversion rate of customer purchase behavior. However, most of the product classification recommendation models currently on the market only use a single machine learning model to analyze and predict user behavior data, which often makes it difficult to fully capture all the characteristics of user data, and the selected financial products often fail to meet the real needs of users. Therefore, in this embodiment, a product classification recommendation method is disclosed, as shown in the attached Figure 1 As shown, the following steps are included.

[0025] Step S1, responding to a recommendation request, obtaining user information of the target user in a target recommendation scenario for constructing a prompt sentence, inputting the sentence into a fine-tuned large language model, and obtaining multiple candidate financial product categories that meet the needs of the target user.

[0026] Specifically, in view of the wide range of financial product types, including deposits, loans, investments, insurance, electronic banking, and special services, and each financial product under each category has its own unique characteristics and applicable scenarios. In order to improve the accuracy of the recommendation model and ensure the overall service quality of the recommendation system, it is particularly important to screen out the financial product categories that meet the needs of users. The present invention uses a pre-trained large language model for text classification, and uses a proprietary data set in the field of financial product transactions to fine-tune the large language model, so that the model can better adapt to specific tasks and accurately identify and classify the financial product categories that users may be interested in. Specifically, the financial product category described in this embodiment is the financial product category. After receiving the recommendation request for the target user by the financial service personnel, the system obtains various types of information of the target user through the internal application program interface, including but not limited to the user's basic information, work information, credit record and financial status, etc., and constructs a prompt sentence to accurately and comprehensively reflect the needs, preferences and potential investment or consumption intentions of the target user, and inputs the fine-tuned and trained large language model to guide the large language model to accurately and targetedly screen out the financial product categories that meet the user's needs. Among them, the large language model used in this embodiment is the Chatglm3-6B model, which can process longer input texts, support text input lengths of up to 8K characters, and obtain more detailed and useful information to enhance the model's ability to mine user intent.

[0027] In this embodiment, the fine-tuning training steps of the large language model are as shown in the attached Figure 2 As shown, it includes the following contents.

[0028] Step S101, obtaining historical financial product recommendation transaction records, wherein each transaction record includes but is not limited to basic information of the person, work information, credit record and financial status, and the corresponding label is the name of the financial product traded by the user and the category of the financial product to which it belongs.

[0029] Specifically, step S101 further includes: after obtaining the historical financial product transaction records in the industry, constructing a Chatglm3 fine-tuning dataset based on the historical financial product transaction records, wherein the Chatglm3 fine-tuning dataset contains sample data of multiple user product transactions. Each sample data includes an input and an output, wherein the input is configured as the user information of the transaction, and the output is configured as the name of the financial product traded by the user and the category of the financial product to which it belongs.

[0030] Step S102, dividing the data set into a training sample set, a validation sample set, and a test sample set according to the labeled financial product categories, and then performing supervised fine-tuning training on the text classification model.

[0031] Specifically, after the dataset is constructed, the Chatglm3 fine-tuning dataset is divided into training set, test set and validation set in proportion to train, test and validate the Chatglm3-6B model, and obtain the fine-tuned Chatglm3-6B model, which can output candidate financial product categories that meet user needs based on user information. Among them, the supervised fine-tuning process is completed by using the lora fine-tuning technology. Lora fine-tuning is an efficient fine-tuning technology for large pre-trained models. Its core idea is to simulate the effect of full parameter fine-tuning by introducing trainable low-rank matrices A and B in the Transformer layer of the model without changing most of the weights of the pre-trained model, so as to achieve rapid adaptation of the model. Specifically, the two low-rank matrices A and B introduced in the added Lora layer structure, their product AB can approximately represent the amount of change of the weight matrix during the fine-tuning process, that is: ; in, ; ; ; .

[0032] During fine-tuning, keep the weight matrix of the original pre-trained model Chatglm3-6B Only training parameters A and B are used to update the Lora low-rank matrix, so that the fine-tuned large model can run on a single card. , after Lora fine-tuning, its forward propagation process becomes: .

[0033] After training, the trained lora weights are combined with the original weights to obtain the final fine-tuning model.

[0034] In this embodiment, the constructed Chatglm3 fine-tuning data set contains 10,000 sample data, and the data set is divided into a training sample set, a validation sample set, and a test sample set in a ratio of 7:1.5:1.5. The fine-tuning environment is an A40 48G graphics card with 48GB of video memory. The fine-tuning hyperparameters are set to: learning rate 5e-5, training rounds 3epochs, maximum input length 4096, batch size 16, learning rate regulator cosine, maximum gradient norm 1, Lora rank 8, Lora random drop probability 0.1, calculation type half-precision floating point number, and learning rate warm-up steps 500. The top2 accuracy of the fine-tuned Chatglm3-6B model in the test is 75.8%.

[0035] In another embodiment, the ernie3.0 text classification model can also be selected to obtain the candidate financial product categories. The model training process is similar to the Chatglm3-6B model training process. The step S101 also includes, after obtaining the historical financial product transaction records in the industry, constructing the ernie3.0 fine-tuning data set based on the historical financial product transaction records. The Chatglm3 fine-tuning data set contains sample data of multiple user product transactions. Each sample data includes an input and an output. The input is configured as the user information of the transaction, and the output is configured as the name of the financial product traded by the user and the financial product category to which it belongs.

[0036] In this embodiment, since the maximum input length of the ernie3.0-base model is 512 characters and the model parameters are small, it is trained on a graphics card with a video memory of 16G, and the optimal training parameters are configured as follows: 100 epochs for training, 3e-5 for learning rate, 512 for maximum input length, and 32 for batch size. At the same time, since the model parameters are small, in order to ensure the normal operation of the model, the configured input information only selects relatively important information, including work information, credit record, financial situation, and the category of financial products that the user has.

[0037] Finally, the fine-tuned Ernie3.0 model had a top1 accuracy (the probability that the category with the highest predicted probability is the correct label) of 65.2% and a top2 accuracy (the probability that the two categories with the highest predicted probability contain the correct label) of 68% in the test.

[0038] Compared with the test results of Chatglm3-6B model, the accuracy of Chatglm3-6B after fine-tuning is higher than that of Ernie3.0. Step S2: According to each candidate financial product category, multiple candidate financial products that meet the user information are screened in the corresponding category product knowledge base, and the candidate financial products of each category are added to the candidate financial product queue.

[0039] Specifically, after obtaining the candidate financial product categories that meet the needs of the target user, the specific financial products that meet the user's needs are further screened out from the candidate financial product categories in combination with the user information and added to the candidate financial product queue, so as to achieve preliminary recommendations of financial products to the user and pave the way for subsequent fine sorting. This embodiment mainly achieves the above needs through RAG multi-way recall technology. Specifically, RAG multi-way recall technology is an advanced method that combines both retrieval enhanced generation (RAG) and multi-way recall technologies. It uses a variety of recall strategies and models to recall the most relevant information to the user's query from a large-scale knowledge base through multiple paths, and merges the candidate information returned by all recall paths into a large candidate set, which is passed as input to the generation model to generate the final answer or output.

[0040] In this embodiment, since the multi-way recall method is mainly divided into two categories in the application field of large language models: vector recall (dense vector recall) and inverted recall (sparse vector recall), and the recall effect and time consumption of different recall methods are different, multiple models of the two methods are comprehensively adopted to establish a knowledge base containing multiple vector libraries and retrieval libraries, and then the two methods are used to recall respectively to capture the characteristics of different dimensions of the data and increase the diversity and accuracy of the recall results. Then the step S2 specifically includes the following content.

[0041] Step S21, divide the text introduction information of each financial product into blocks to form a first knowledge base, wherein each data block contains the corresponding financial product name and the current data block content; obtain the historical transaction records of each financial product as the second knowledge base, wherein the historical transaction records include the name of the traded financial product and the transaction record information of the financial product.

[0042] Specifically, a knowledge base is first constructed based on the introduction information and historical transaction records of each financial product in the financial product category that the user is interested in. Since the text introduction information of each financial product is relatively long, this embodiment divides it into blocks to improve the speed and accuracy of the subsequent processing process. Each data block formed after the division contains a financial product name and relatively independent part of the introduction information corresponding to the financial product.

[0043] Step S22, vectorize all data blocks in the first knowledge base to form a first vector knowledge set; vectorize each transaction record information in the second knowledge base, and use the name of the financial product as a label for recall to form a second vector knowledge set.

[0044] Specifically, in order to recall relevant candidates by means of vector recall, all user features and material features are represented by vectors, that is, each financial product text introduction information block in the first knowledge base and each financial product transaction record information in the second knowledge base are converted into vectors using a vector model and stored in the corresponding vector library, so that the financial products of interest to the user can be recalled from the vector library according to the user's intention and related information by calculating the similarity between the vectors in the future. Among them, since different vector models have different sensitivities and accuracies for different types of data, their recall effects are different, so multiple vector models are selected to vectorize the data, and the step S22 specifically includes the following contents.

[0045] Step S221, use three different embedded models to vectorize each data block in the first knowledge base, and use the FAISS library tool to establish a separate vector library for the vector data generated by each embedded model, generate the corresponding first FAISS vector library, second FAISS vector library and third FAISS vector library, and store them in the first vector knowledge set.

[0046] Step S222, use three different embedded models to vectorize the transaction record information in the second knowledge base, use the corresponding financial product name as the recall label, and use the FAISS library tool to establish a separate vector library for the vector data generated by each embedded model, generate the corresponding fourth FAISS vector library, fifth FAISS vector library and sixth FAISS vector library, and store them in the second vector knowledge set.

[0047] In this embodiment, three vector models, gte-large-embeddin, bge-large-embedding, and m3e-large-embedding, are selected to vectorize each data block in the first knowledge base and each transaction record information in the second database, and a separate vector library is established for the vector data generated by each model using the FAISS library tool. Finally, a total of 6 FAISS vector libraries can be established, thereby improving the overall recall accuracy and performance.

[0048] Step S23, segmenting the first knowledge base and the second knowledge base based on the Jieba word segmentation database, and then using a text matching algorithm to establish a knowledge index to form a corresponding first search database and a second search database.

[0049] Specifically, in order to recall relevant candidates by inverted recall, the word segmenter is used to segment the document, and an inverted index is constructed for different attributes of financial products using algorithms such as TF-IDF and BM25, so as to recall relevant candidates in a targeted manner later. In this embodiment, two index libraries are established.

[0050] In this embodiment, step S23 specifically includes: performing word segmentation on the first knowledge base and the second knowledge base based on the Jieba word segmentation library, and then using the text matching algorithm BM25 to establish a knowledge index to form a corresponding first search library and a second search library.

[0051] Specifically, the BM25 algorithm is an algorithm for information retrieval. It calculates the score of a document based on the term frequency (TF) and inverse document frequency (IDF) to evaluate the degree of match between the document and the query, thereby measuring the relevance between the document and the user query. It is widely used in information retrieval fields such as search engines and natural language processing. Its calculation formula is as follows: ; in, is the i-th morpheme in the query; d is the document; R(qi, d) is the relevance score between morpheme qi and document d: , , where k1, k2, and b are adjustment factors, which are usually set based on experience. Generally, k1=2, b=0.75. fi is the frequency of occurrence of qi in d, qfi is the frequency of occurrence of qi in the question, dl is the length of document d, and avgdl is the average length of all documents. The weight representing the relevance of a word to a document. In this embodiment, the weight is obtained by inverse document frequency: ; Among them, N is the total number of documents in the index, and n(qi) is the number of documents containing qi.

[0052] Step S24, after vectorizing the target user information, similar data vectors are recalled in the first vector knowledge set and the second vector knowledge set respectively, and the financial product name corresponding to each similar data vector is obtained as the recall result; the target user information is used to use a text matching algorithm to recall matching data in the first retrieval library and the second retrieval library respectively, and the financial product name corresponding to each matching data is obtained as the recall result.

[0053] Specifically, after the knowledge base is constructed, the types of financial products that the user may be interested in are recalled based on the user information. Since this embodiment establishes 6 FAISS vector libraries and 2 bm25 search libraries, this embodiment adopts an 8 (6+2)-way recall strategy. Then the step S24 specifically includes the following contents.

[0054] Step S241, the basic information, work information, credit record and financial status of the target user in the information are sorted and merged into the information to be matched, and the three different embedded models are used to vectorize the information to be matched to obtain the corresponding three vectors to be matched. In this embodiment, the three embedded models selected are the same as the models used when building the knowledge base, namely, the gte-large-embeddin model, the bge-large-embedding model and the m3e-large-embedding model.

[0055] Step S242, recall six similar data vector groups from the corresponding first vector knowledge set and second vector knowledge set respectively for the three vectors to be matched, wherein each similar data vector group contains a set number of similar data vectors, and obtain the names of financial products corresponding to the similar data vectors as the recall results of this path.

[0056] Specifically, financial products that meet user needs are recalled in the search database by means of vector recall. Specifically, for a vector library established with financial product introduction information, specific financial products are returned; for a vector library established with historical transaction records of financial products, the last financial product purchased in the transaction record is returned.

[0057] Step S243: Use the text matching algorithm to retrieve the two matching data groups corresponding to the information to be matched in the first search database and the second search database, respectively, wherein each matching data group contains a set number of matching data, and obtain the names of financial products corresponding to the matching data as the recall results of this search. Specifically, the financial products that meet the user's needs are retrieved in the search database by reverse recall.

[0058] Step S25, removing duplicate financial product names in the recall results and combining them to form a candidate financial product queue.

[0059] In this embodiment, the step S25 specifically includes: removing duplicates from the names of the financial products in the eight-way recall results and combining them to form a candidate financial product queue.

[0060] Furthermore, a feedback mechanism can be added to dynamically adjust the weight of each recalled channel to dynamically adjust the number of recalled vectors of each channel, which also includes: within a preset time period, counting which channels the products obtained from each sample after fine sorting come from, and calculating the dynamic weight of each channel , and the number of subsequent recall vectors for each channel: ; The number of recalls for channel i = the number of basic recalls + * (total number of recalls - basic number of recalls * number of channels); in, The number of financial products from each channel obtained for subsequent fine sorting, N is the total number of samples on that day.

[0061] In this embodiment, each recall path recalls 10 candidates, and a total of 80 candidates are recalled.

[0062] Step S3, respectively pairing each user information feature with each candidate financial product feature in the candidate financial product queue using a first order and a second order to form a plurality of user product groups, wherein the user product group includes a user information feature and a candidate financial product feature, wherein the first order is to put the user information feature first and the candidate financial product feature second in the user product group, and the second order is to put the candidate financial product feature first and the user information feature second in the user product group.

[0063] Specifically, after selecting specific financial products that meet the needs of users, the user information is paired with each candidate financial product in the candidate financial product queue to form multiple user product groups, so that the trained fine ranking model can be used to rank and score each user product group in the future, and the financial products in the user product groups with a set number of recommendations ranked first are obtained as the final recommended products. The user information features include historical conversation information, financial information, work information and credit information, and the financial product features include product description, product specifications and product requirements.

[0064] Wherein, the step S3 also includes the following contents.

[0065] Step S31, obtaining user information features and financial product features corresponding to each candidate financial product recalled for each financial product category, wherein the user information features include historical conversation information, financial situation information, work situation information and credit situation information, and the financial product features include product description, product specifications and product requirements.

[0066] Step S32, splitting the user information features into a first user feature and a second user feature, wherein the first user feature includes historical conversation information, and the second user feature includes user financial status information, work status information, and credit status information; splitting the financial product features corresponding to each recalled candidate financial product into a first product feature and a second product feature, wherein the first product feature includes a product description of the corresponding financial product, and the second product feature includes product specifications and product requirements of the corresponding financial product.

[0067] Specifically, since the subsequent refined ranking model used is the bge-reranker-large model, its maximum input length is 512, and the combined length of the collected user information and financial product features is greater than 512, the input information needs to be readjusted by splitting to ensure that the input data is not truncated, that is, the user information and financial product features are split into two parts, namely, the user information is split into historical conversation information and information including financial status, work status, and credit status; the financial product features are split into product descriptions and product features.

[0068] Step S33, respectively pairing the first user feature and the second user feature corresponding to the user information feature with the first product feature and the second product feature corresponding to each recalled candidate financial product in the first order and the second order, to generate n*4 positive order user-product groups and n*4 reverse order user-product groups, where n is the total number of candidate financial products under each recalled financial product category.

[0069] Specifically, since different order combinations have different influences on the model, the robustness of the model can be further increased by constructing the same sample data in different ways using the first order and the second order. In this embodiment, n=80, the user information features in the positive order user product group are before the financial product features, and the user information features in the reverse order user product group are after the financial product features.

[0070] Step S4: Use the trained fine ranking model to rank and score each user product group, and obtain the financial products in the user product groups with a set number of recommendations that are ranked first as the final recommended products.

[0071] Specifically, after the user product group is constructed, the refined ranking model is used to rank and score the user product group to screen out the best financial products that best meet the user's needs among the candidate financial products. The refined ranking model can select open source refined ranking models such as bge-reranker-large and bce-reranker-base_v1. In this embodiment, the bge-reranker-large model with a relatively high download volume is selected, and its architecture is shown in the attached figure. Figure 3 As shown, it includes an embedding input layer, an encoder layer and a loss function layer, wherein the embedding input layer includes a text embedding layer, a position embedding layer and a token type embedding layer, wherein the encoder layer is composed of 24 consecutive encoder layers Transformer Encoder stacked together, and the loss function layer is configured to communicate with a first loss function and a second loss function, wherein the first loss function is configured to judge the semantic relevance of user information and product information, and the second loss function is configured to judge the word order of user information and financial product information.

[0072] Specifically, this embodiment adds a second loss function for judging the word order of user information features and financial product features on the basis of the first loss function for judging the semantic relevance of user information features and financial product features in the original model, so as to allow the model to learn the causal relationship between user information and product information and enhance the model's ability to understand semantics. At the same time, considering the small number of training sets, only one 1024*1 fully connected layer is added to the second loss function to reduce the difficulty of training.

[0073] In this embodiment, The first loss function is: ; The second loss function is: ; The final loss function is configured as: ; in and Represents two labels respectively. As learnable parameters, they are used to represent the weights of the two parts of the loss.

[0074] Specifically, As a learnable parameter, it represents the weight of the two parts of the loss. Using adaptive weights to adjust the ratio of the two parts of the loss is better than directly adding the two parts of the loss or manually setting the ratio of the two parts of the loss.

[0075] In this embodiment, since the open source fine ranking model belongs to a general domain model, and the product information in the bank belongs to private domain information, the fine ranking score obtained by directly using the open source model may be lower than the specified threshold, resulting in a small number of recalls and failure to recommend more products. Therefore, this embodiment fine-tunes the bge-reranker-large open source model by constructing a fine ranking fine-tuning dataset, as shown in the attached figure. Figure 4 As shown, the specific contents include the following.

[0076] Step S201, the user information features and the features of the traded financial products in the historical financial product recommendation transaction records are taken in advance, and the user information features are split into a first user feature group and a second user feature group, wherein the first user feature group includes the historical conversation information of the target user, and the second user feature group includes the financial situation information, work situation information and credit situation information; the financial product features are split into a first product feature group and a second product feature group, wherein the first product feature group includes the product description of the corresponding financial product, and the second product feature group includes the product specifications and product requirements of the corresponding financial product.

[0077] Specifically, the bge-reranker-large model is fine-tuned using the user information features and the features of the traded financial products in the historical financial product recommendation transaction records as sample data, where the features of the traded financial products can be obtained from the product knowledge base based on the names of the financial products in the historical financial product recommendation transaction records.

[0078] Step S202, constructing the user information features and the features of the traded financial products in each historical financial product recommendation transaction record into four positive training data, wherein the four positive training data are the first user feature group and the first product feature group, the first user feature group and the second product feature group, the second user feature group and the first product feature group, and the second user feature group and the second product feature group.

[0079] Step S203, replace the financial product features in the historical financial product recommendation transaction records with other financial product categories or with products with significantly different specifications in the same financial product category, and construct four negative training data with the user information features and the replaced financial product features.

[0080] Specifically, after the splitting is completed, multiple positive training data and multiple negative training data are constructed according to each feature information after the splitting. Among them, the positive training data is semantically close to the original sample data, and the negative training data is semantically different from the original sample data. It can be constructed by replacing the products in the product information with products of other financial categories, such as replacing loan products with deposit products, insurance products, etc., or by replacing the products in the product information with products with large specifications in the same financial category, so that the user information does not match the product information. Since the cross entropy loss function is used in this embodiment, the positive and negative examples have little effect, so no specific requirements are made on the positive and negative examples. The positive and negative examples ratio of the training data in this embodiment is 1:15, that is, each historical financial product recommendation transaction record constructs a semantically close positive sample and fifteen negative samples obtained by replacement, and on this basis, the positive and negative training data are constructed, and finally 120,000 positive and negative training data are cleaned.

[0081] Step S204, the positive training data and the negative training data are mixed and formed into a training set, a validation set and a test set according to a set ratio, and then input into a refined ranking model for training.

[0082] In this embodiment, after the fine-tuning data is constructed, it is divided into a training set, a validation set, and a test set in a ratio of 7:1.5:1.5, and the positive training data and the negative training data are respectively input into the model for training, validation, and testing, wherein the label of the positive training data is 1 when used as input, and the label of the negative training data is 0 when used as input. Since each test sample in this test environment has 3-5 financial products that can be recommended to users, this embodiment uniformly takes 3 financial products as labels, and counts the number of overlaps between the 3 financial products and the first 3 products selected in the fine ranking model. If there is one product overlap, the score is 1, if there are two products overlap, the score is 2, and if there are three products overlap, the score is 3. Finally, the average score of all test set samples is the evaluation index of the fine ranking model. The average score of the final test set samples of this embodiment is 2.1, while the average score of the test set samples after not using the model corresponding to the second loss function (i.e., the original bge-reranker-large open source model) is 1.9. It can be seen that adding the second loss function can significantly increase the number of valid products recalled. In order to ensure the effectiveness of the adaptive weights, this embodiment performs three fine-tuning trainings on the model, and the optimal The value is close to 0.7, that is, the first loss function for semantic relevance judgment plays a major role in the fine ranking model ( =0.7), the precise sorting model for word order causal relationship judgment plays a minor role ( =0.3).

[0083] After the model fine-tuning training is completed, each user product combination can be finely ranked and scored according to the fine-tuned bge-reranker-large model to obtain the optimal financial product recommendation result under each financial product category, that is, the n*4*2 user product groups corresponding to each financial product category are input into the fine-tuned trained model to obtain the scores corresponding to the n*4*2 user product groups, that is, the user information and each financial product can obtain eight scores, and the preset number of financial products with the largest scores are the financial products finally screened out to be recommended to users.

[0084] Furthermore, a feedback mechanism may be added to the multi-channel recall module, and the weight of each channel recalled may be dynamically adjusted to dynamically adjust the number of recall vectors of each channel. The method further includes: within a preset time period, counting the number of final recommended financial products recalled by each channel in the sample, and calculating the number of subsequent recall vectors of each channel: The number of recalls for channel i = the number of basic recalls + * (total number of recalls - basic number of recalls * number of channels); ; in, It is the number of optimal financial products from each channel within the preset time period finally obtained by the fine sorting model; N is the total number of samples in the preset time period; the basic recall number is the basic recall number for each channel, which is set to 1 in this patent, in order to retain the candidate with the highest recall score in this channel; the number of channels is the number of channels for multi-channel recall, which is 8 in this patent, i.e. 8-channel recall; the total number of recalls is also called the total number of rough sorting recalls, which is set to 80 in this patent, i.e. a total of 80 candidates are recalled, and 10 candidates are recalled for each channel; weight is a dynamic weight, and this patent calculates the channel number of each product from the results with better feedback based on the daily statistical results, and calculates the weight of each channel. Through dynamic adjustment, the accuracy of multi-channel recall is further improved.

[0085] The product classification recommendation method disclosed in this embodiment combines a large language model and a fine ranking model, and recommends products that meet user needs based on a full understanding of user needs and the characteristics of each financial product. The user information prompt sentence of the constructed user in the target recommendation scenario is input into the fine-tuned and trained large language model to obtain the financial product categories and candidate financial products that meet the user needs, and the user information features are matched with the candidate financial product features to generate user product groups with different orders, which are input into the trained fine ranking model for scoring, and finally the financial recommended products that meet the user needs are obtained, thereby improving the accuracy and personalization of the product classification recommendation model in the financial scenario. In another embodiment, as shown in the attached Figure 5As shown, a product classification recommendation system is also disclosed, including a category acquisition module 1, a product screening module 2, a feature matching module 3 and a fine ranking recommendation module 4. The category acquisition module 1 is used to respond to the recommendation request, obtain the user information of the target user in the target recommendation scenario for constructing a prompt sentence, and input it into the large language model after fine-tuning training to obtain multiple candidate financial product categories that meet the needs of the target user. The product screening module 2 is used to screen and obtain multiple candidate financial products that meet the user information in the corresponding category product knowledge base according to each candidate financial product category, and add the candidate financial products of each category to the candidate financial product queue. The feature matching module 3 is used to pair each user information feature with each candidate financial product feature in the candidate financial product queue in a first order and a second order respectively to form multiple user product groups, wherein the user product group includes a user information feature and a candidate financial product feature, wherein the first order is to put the user information feature first and the candidate financial product feature second in the user product group, and the second order is to put the candidate financial product feature first and the user information feature second in the user product group. The refined ranking recommendation module 4 is used to use the trained refined ranking model to rank and score each user product group, and obtain the financial products in the user product groups with a set number of recommendations that are ranked first as the final recommended products.

[0086] In this embodiment, a large language model fine-tuning training module is also included, which specifically includes a record acquisition module and a fine-tuning training module. The record acquisition module is used to obtain historical financial product recommendation transaction records, where each transaction record includes but is not limited to basic information of personnel, work information, credit record and financial status, and the corresponding label is the name of the financial product traded by the user and the financial product category to which it belongs. The fine-tuning training module is used to perform supervised fine-tuning training on the text classification model after dividing the data set into a training sample set, a verification sample set and a test sample set according to the financial product category of the label.

[0087] In this embodiment, the user information features include historical conversation information, financial information, work information and credit information, the financial product features include product description, product specifications and product requirements, and also include a precision ranking model training module, specifically including a feature splitting module, a positive example construction module, a negative example construction module and a model training module. The feature splitting module is used to recommend the user information features and the features of the traded financial products in the historical financial product transaction records in advance, split the user information features into a first user feature group and a second user feature group, the first user feature group includes the historical conversation information of the target user, and the second user feature group includes financial information, work information and credit information. Split the financial product features into a first product feature group and a second product feature group, the first product feature group includes the product description of the corresponding financial product, and the second product feature group includes the product specifications and product requirements of the corresponding financial product. A positive example construction module is used to construct the user information features and the features of the traded financial products in each historical financial product recommendation transaction record into four positive example training data, wherein the four positive example training data are the first user feature group and the first product feature group, the first user feature group and the second product feature group, the second user feature group and the first product feature group, and the second user feature group and the second product feature group. A negative example construction module is used to replace the financial product features in the historical financial product recommendation transaction record with other financial product categories or with products with large specification differences in the same financial product category, and construct the user information features and the replaced financial product features into four negative example training data; a model training module is used to mix the positive example training data and the negative example training data, and then form a training set, a validation set and a test set according to a set ratio, and then input them into the precision sorting model for training.

[0088] In other embodiments, a server is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, each step of the product classification recommendation method described in the above embodiments is implemented.

[0089] If the product classification recommendation system is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned product classification recommendation method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0090] In conclusion, the above is only a preferred embodiment of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the patent of the present invention.

Claims

1. A product classification recommendation method, characterized in that: The steps include: S1, responding to the recommendation request, obtaining the target user's user information for the target recommendation scenario to construct a prompt sentence, inputting it into the fine-tuned large language model, and obtaining multiple candidate financial product categories that meet the needs of the target user; S2, according to each candidate financial product category, screen and obtain multiple candidate financial products that meet the user information in the corresponding category product knowledge base, and add the candidate financial products of each category to the candidate financial product queue; S3, respectively pairing each user information feature with each candidate financial product feature in the candidate financial product queue in a first order and a second order to form a plurality of user product groups, wherein the user product group includes a user information feature and a candidate financial product feature, wherein the first order is to put the user information feature first and the candidate financial product feature second in the user product group, and the second order is to put the candidate financial product feature first and the user information feature second in the user product group; S4, using the trained fine ranking model to rank and score each user product group, and obtaining the financial products in the user product groups with a set number of recommendations that are ranked first as the final recommended products.

2. The product classification recommendation method according to claim 1, characterized in that: The fine-tuning training steps of the large language model are as follows: Obtain historical financial product recommendation transaction records, where each transaction record includes but is not limited to basic personal information, work information, credit record and financial status, and the corresponding label is the name of the financial product traded by the user and the financial product category to which it belongs; The data set is divided into training sample set, validation sample set and test sample set according to the labeled financial product categories, and then supervised fine-tuning training is performed on the text classification model.

3. The product classification recommendation method according to claim 1, characterized in that: The user information features include historical conversation information, financial information, work information and credit information, and the financial product features include product descriptions, product specifications and product requirements. The specific training steps of the precise ranking model are as follows: The user information features and the features of the traded financial products in the historical financial product recommendation transaction records are divided into a first user feature group and a second user feature group. The first user feature group includes the historical conversation information of the target user, and the second user feature group includes the financial situation information, work situation information and credit situation information; the financial product features are divided into a first product feature group and a second product feature group. The first product feature group includes the product description of the corresponding financial product, and the second product feature group includes the product specifications and product requirements of the corresponding financial product; Constructing the user information features and the features of the traded financial products in each historical financial product recommendation transaction record into four positive training data, wherein the four positive training data are the first user feature group and the first product feature group, the first user feature group and the second product feature group, the second user feature group and the first product feature group, and the second user feature group and the second product feature group; Replace the financial product features in the historical financial product recommendation transaction records with other financial product categories or with products with significantly different specifications in the same financial product category, and construct four negative training data with the user information features and the replaced financial product features; The positive training data and the negative training data are mixed and formed into a training set, a validation set and a test set according to a set ratio, and then input into the refined sorting model for training.

4. The product classification recommendation method according to claim 3, characterized in that: The precise sorting model includes an embedding input layer, an encoder layer and a loss function layer, wherein the embedding input layer includes a text embedding layer, a position embedding layer and a token type embedding layer, wherein the encoder layer is composed of 24 consecutive Transformer Encoder layers stacked together, and the loss function layer is configured to communicate with a first loss function and a second loss function, wherein the first loss function is configured to judge the semantic relevance of user information and product information, and the second loss function is configured to judge the word order of user information and financial product information.

5. The product classification recommendation method according to claim 4, characterized in that: The first loss function is: ; The second loss function is: ; The final loss function is configured as: ; in and Represent two labels respectively and are used as learnable parameters to represent the weights of the two parts of the loss.

6. A product classification recommendation system, characterized in that: include: The category acquisition module is used to respond to the recommendation request, obtain the user information of the target user in the target recommendation scenario to construct the prompt sentence, input it into the fine-tuned large language model, and obtain multiple candidate financial product categories that meet the needs of the target user; A product screening module is used to screen and obtain multiple candidate financial products that meet the user information from the corresponding category product knowledge base according to each candidate financial product category, and add the candidate financial products of each category to the candidate financial product queue; a feature matching module, used to match each user information feature with each candidate financial product feature in the candidate financial product queue in a first order and a second order respectively to form a plurality of user product groups, wherein the user product group includes a user information feature and a candidate financial product feature, wherein the first order is to put the user information feature first and the candidate financial product feature second in the user product group, and the second order is to put the candidate financial product feature first and the user information feature second in the user product group; The refined ranking recommendation module is used to use the trained refined ranking model to rank and score each user product group, and obtain the financial products in the user product groups with a set number of recommendations that are ranked first as the final recommended products.

7. The product classification recommendation system according to claim 6, characterized in that: It also includes a large language model fine-tuning training module, including: The record acquisition module is used to obtain historical financial product recommendation transaction records, where each transaction record includes but is not limited to basic information of the person, work information, credit record and financial status, and the corresponding label is the name of the financial product traded by the user and the category of the financial product to which it belongs; The fine-tuning training module is used to split the data set into training sample sets, validation sample sets, and test sample sets according to the labeled financial product categories, and then perform supervised fine-tuning training on the text classification model.

8. The product classification recommendation system according to claim 7, characterized in that: The user information features include historical conversation information, financial information, work information and credit information; the financial product features include product descriptions, product specifications and product requirements, and also include a precise sorting model training module, specifically including: A feature splitting module is used to recommend user information features and features of the traded financial products in historical financial product transaction records in advance, split the user information features into a first user feature group and a second user feature group, wherein the first user feature group includes historical conversation information of the target user, and the second user feature group includes financial information, work information, and credit information; split the financial product features into a first product feature group and a second product feature group, wherein the first product feature group includes a product description of the corresponding financial product, and the second product feature group includes product specifications and product requirements of the corresponding financial product; A positive example construction module is used to construct the user information features and the features of the traded financial products in each historical financial product recommendation transaction record into four positive example training data, wherein the four positive example training data are respectively the first user feature group and the first product feature group, the first user feature group and the second product feature group, the second user feature group and the first product feature group, and the second user feature group and the second product feature group; A negative example construction module is used to replace the financial product features in the historical financial product recommendation transaction records with other financial product categories or with products with significantly different specifications in the same financial product category, and construct four negative example training data with user information features and replaced financial product features; The model training module is used to mix the positive training data and the negative training data and form a training set, a validation set and a test set according to a set ratio, and then input them into the refined sorting model for training.

9. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Supermarket and supermarket information query method and device, storage medium and computer equipment

    CN118013006A

  • LoRA fine tuning method based on parameter-free attention module

    CN118569343A

  • Intelligent interaction method and device, medium and electronic equipment

    CN118585629A

  • Multi-path recall retrieval enhancement generation method and device based on real-time data index

    CN118820400A

  • Data processing method and related apparatus

    WO2024012360A1