Multi-mode collection and repayment reminding method and system based on large model

Abstract

The invention relates to a collection repayment reminding method and system based on multiple modes of a large model, and belongs to the field of finance, and the method comprises the steps: S1, obtaining the repayment information, consumption information and active information of a customer; s2, the customer information obtained in the step S1 is analyzed through a repayment evaluation large model, and a repayment reminding strategy of the customer is obtained by combining the RAG technology with the latest information of the user; and S3, generating corresponding reminding content according to the repayment reminding strategy, and sending the reminding content to the customer in combination with a corresponding channel in the repayment strategy at a corresponding time node. The method has the advantages that nowadays, with more and more social media and more and more abundant contents, the prompt copywriting of the traditional fixed template is very difficult to attract the attention of people. The technology not only is extremely rich and novel in prompt content, but also is combined with numerous channels, can prompt customers in a diversified manner, attracts the attention of the customers, and greatly improves the repayment efficiency.

Description

Technical Field

[0001] The present invention relates to a large-model multimodal debt collection and repayment reminder method and system thereof, and belongs to the financial field. Background Art

[0002] Traditional intelligent repayment reminders are basically based on AI technology to analyze and train a large number of customers' historical repayment information, historical communication information, consumption information, etc., and build a professional repayment prediction and evaluation model. In actual applications, the customer's business information is collected and input into the repayment prediction and evaluation model to obtain the customer's evaluation score and repayment reminder strategy, and then according to the corresponding repayment strategy, the repayment reminder operation is executed at the appropriate time predicted by the model.

[0003] Problems with current technology:

[0004] 1. Single content: Although customers can be reminded through various channels, such as phone, SMS, WeChat, email, and other social media, the generated reminder content is relatively single and often has a fixed reminder template. It is difficult to attract customers’ attention due to poor readability.

[0005] 2. Poor timeliness: The repayment prediction and evaluation model can only be constructed based on the historical repayment information, historical communication information, consumption information, etc. that existed in the past. Because the cost of AI training is high, after the initial one-time training, there is basically no secondary training and upgrading operation, so there is a lot of outdated data that cannot adapt to the latest customer information.

[0006] 3. Low accuracy: Traditional intelligent reminders are based on traditional small models, which are poor in in-depth understanding of customer information, humanization, emotion, and personalization. In particular, the generation of reminder content is based on fixed reminder templates, and the generated content is not humanized or personalized, resulting in poor reminder effects.

[0007] 4. High cost: To train a professional model suitable for repayment prediction and evaluation, it is necessary to collect a large amount of data sets and organize a large amount of manpower to cooperate with training and fine-tuning in order to customize and train an adaptive model. Moreover, if the model is to be integrated with the business system, the application of the business system for the integrated model also requires the organization of a large number of developers to customize the development according to the actual scenario. Summary of the invention

[0008] In order to overcome the defects of the prior art, the present invention provides a debt collection and repayment reminder method based on a large model multi-modality. The technical solution of the present invention is:

[0009] A debt collection and repayment reminder method based on a large model and multi-modality includes the following steps:

[0010] S1. Obtain the customer's repayment information, consumption information and activity information;

[0011] S2, analyzing the customer information obtained in step S1 through the repayment assessment model, and using RAG technology to combine the customer information to obtain the customer's repayment reminder strategy;

[0012] S3. Generate corresponding reminder content according to the repayment reminder strategy, and send it to the customer at the corresponding time node in combination with the corresponding channel in the repayment strategy.

[0013] The step S1 is specifically as follows:

[0014] S1-1. Obtain the latest loan information of customers from the financial system every half an hour; obtain the consumption information, activity information and business information of customers from Internet vendors;

[0015] S1-2. Use the vector model EmbeddingModel in langchain4j to vectorize customer information based on the neural network algorithm embedding;

[0016] S1-3. Use the MilvusEmbeddingStore tool of langchai4j to persistently store the vectorized data in step S1-2 in the open source vector database Milvus, and locate customer information based on the retrieval algorithm of the Milvus vector database.

[0017] In the step S1-2, the customer information text is vectorized. In the text vectorization process, the data in the text is represented as vectors according to two categories of words and sentences with complete expression meanings, specifically:

[0018] (1) When performing word vectorization, a single word is converted into a numerical vector:

[0019] (1-1) Assign a unique binary vector to each word, where only one position is 1 and the rest are 0;

[0020] (1-2) Map each word to a real vector that is semantically related;

[0021] (2) When performing sentence vectorization, the entire sentence is converted into a numerical vector:

[0022] (2-1) Average the word vectors in the sentence or perform weighted average based on word frequency;

[0023] The sentence representation is generated by recursively processing each word in the sentence through a recursive neural network; the convolutional layer of the convolutional neural network is used to capture the local features in the sentence and then generate the sentence representation; through the self-attention mechanism, each word in the sentence is self-attention calculated to generate the sentence representation.

[0024] In the step S1-3, specifically:

[0025] When performing retrieval algorithms, Milvus supports similarity measurement methods. When creating an index, Milvus builds the corresponding index structure and performs query operations based on the specified measurement type.

[0026] In the step S2, specifically:

[0027] (1) Set reminder assistant prompt words;

[0028] (2) Upload successful debt collection cases and upload a preset amount of historical customer debt information, consumption information, and social media information as a stored knowledge base, which will be used as model embedding when formulating repayment strategies later;

[0029] (3) After completing the above steps, publish and generate the GPT-CUI model adapted for repayment reminders; (4) Generate repayment reminder strategy: obtain the customer list information that needs to be reminded of repayment from the collection system, and use langchain4j to query the latest customer information in Milvus.

[0030] In the step (4), specifically:

[0031] A. First, make an encoder for the customer and the associated information in Milvus:

[0032] A1. Capture the global information in the sequence by performing attention operation on the input sequence itself;

[0033] A2. Implement a feedforward neural network through two linear transformations and a ReLU activation function;

[0034] A3. Each sub-layer is followed by a residual connection and layer normalization;

[0035] B. Decode the natural language processing results: mask the multi-head self-attention to prevent the decoder from seeing future information in the process of generating sequences, and perform attention operations through the encoder output and decoder input;

[0036] C. Generate repayment reminder strategy: Langchain4j parses the data results, formats json, generates the final repayment reminder strategy, and stores it in the PostgreSQL of the business system for easy use next time.

[0037] In the step S3, specifically:

[0038] (1) Dynamic decision-making based on COT:

[0039] (1-1) Summarize the customer's repayment information and social media information;

[0040] (1-2) Find the base model based on the customer summary information and repayment strategy;

[0041] (1-3) Use the base model to generate reminder content in conjunction with the customer's information summary;

[0042] (2) Multimodal content generation:

[0043] (2-1) Generative Adversarial Network:

[0044] The generator generates data, and the discriminator tries to distinguish between real data and generated data;

[0045] Initialize the generator G and the discriminator D;

[0046] For each batch of data x, the generator G generates a set of data G(x);

[0047] The discriminator D classifies the real data x and the generated data G(x);

[0048] Update G and D using adversarial training;

[0049] (2-2) Converter architecture:

[0050] Initialize the encoder and decoder;

[0051] Encode the input data and obtain the encoder output;

[0052] Decode the encoder output using the decoder to generate an output sequence;

[0053] (2-3) Use GANs to generate images and convert them into text descriptions through transformers; use the generated text descriptions to generate images again, forming a cycle; adjust model parameters to optimize the generation results.

[0054] A debt collection and repayment reminder system based on a large model and multi-modality, including:

[0055] The information collection module is used to obtain the customer's repayment information, consumption information and activity information; the repayment prediction module is used to analyze the obtained customer information through the repayment assessment model, and use RAG technology to combine the user's latest information to obtain the customer's repayment reminder strategy; the content generation module is used to generate corresponding reminder content according to the repayment reminder strategy, and send it to the customer at the corresponding time node in combination with the corresponding channel in the repayment strategy.

[0056] The advantages of the present invention are:

[0057] 1. With the increasing number of social media and richer content, it is difficult for traditional fixed template reminders to attract people's attention. This technology not only provides extremely rich and novel reminder content, but also combines a variety of channels to give customers a variety of reminders, attracting customers' attention and greatly improving repayment efficiency.

[0058] 2. This technology is low-cost. Based on customized GPTs, a large model suitable for repayment reminders can be generated without a lot of manpower and resources for customized development. In addition, with the help of large model development frameworks LangChain4j and Quarkus, the application development of the model is relatively simple and low-cost, which is more conducive to promotion and commercial application.

[0059] Among them, GPTs is a platform launched based on OpenAI's GPT series models, which allows users to choose appropriate GPT models according to their needs and use them to complete various tasks such as text generation, sentiment analysis, image recognition, etc.

[0060] The launch of GPTs provides convenience for beginners and developers. They can communicate their needs with ChatGPT through natural language and generate exclusive GPTs without having to know complex programming languages. This technology lowers the threshold for AI applications, allowing more people to turn their ideas into reality. For example, some people have used GPTs to develop practical tools such as resume generators, shopping assistants, and book creation guides.

[0061] 3. The optimization and upgrade of the model of this technology is simpler. By using the Knowledge in GPTs, the latest customer business information can be embedded into the Knowledge as vector data in real time. It is very convenient and can realize real-time optimization and upgrade of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of the main structure of the present invention. DETAILED DESCRIPTION

[0063] The present invention will be further described below in conjunction with specific embodiments, and the advantages and features of the present invention will become clearer as the description proceeds. However, these embodiments are exemplary only and do not constitute any limitation to the scope of the present invention. It should be understood by those skilled in the art that the details and forms of the technical solution of the present invention may be modified or replaced without departing from the spirit and scope of the present invention, but these modifications and replacements all fall within the scope of protection of the present invention.

[0064] See also Figure 1 The present invention relates to a debt collection and repayment reminder method based on a large model multi-modality, comprising the following steps:

[0065] S1. Obtain the customer's repayment information, consumption information and activity information;

[0066] S2. Analyze the customer information obtained in step S1 through the repayment assessment model, and use RAG technology to combine the user's latest information to obtain the customer's repayment reminder strategy;

[0067] Among them, the user's latest information is the customer's repayment information, consumption information and active information in the past month. Specifically, the EmbeddingModel in langchain4j is used to query the customer's related information in the past month from Milvs, and the retrieved information and questions are injected into the LLM prompt, and input into the repayment assessment model analysis.

[0068] S3. Generate corresponding reminder content according to the repayment reminder strategy, and send it to the customer at the corresponding time node in combination with the corresponding channel in the repayment strategy.

[0069] The step S1 is specifically as follows:

[0070] S1-1. Obtain the latest loan information of customers from financial systems (risk control system, application system, collection system, etc.) every half an hour; obtain the consumption information, activity information and financial information of customers from Internet vendors (WeChat, the three major operators, Alipay, Weibo, Douyin, etc.);

[0071] S1-2. Use the vector model EmbeddingModel in langchain4j and the neural network algorithm embedding to vectorize customer information;

[0072] S1-3. Use the MilvusEmbeddingStore tool of langchai4j to persistently store the vectorized data in step S1-2 in the open source vector database Milvus, and locate customer information based on the vector index and retrieval algorithm of the Milvus vector database.

[0073] In the step S1-2, the customer information text is vectorized. In the text vectorization process, the data in the text is represented as vectors according to two categories of words and sentences with complete expression meanings, specifically,

[0074] (1) When performing word vectorization, a single word is converted into a numerical vector:

[0075] (1-1) Assign a unique binary vector to each word, where only one position is 1 and the rest are 0; such as Word2Vec, GloVe, FastText, etc.

[0076] (1-2) Map each word to a real vector that is semantically related;

[0077] For example, use the word embedding algorithm to convert each word into a word vector. In the Transformer paper, the dimension of the word embedding vector is 512.

[0078] The representation vector X of each word in the input sentence is obtained by adding the embedding of the word and the embedding of the word position;

[0079] (2) When performing sentence vectorization, the entire sentence is converted into a numerical vector:

[0080] (2-1) Average the word vectors in the sentence or perform weighted average based on word frequency;

[0081] Generate sentence representation by recursively processing each word in a sentence through a recursive neural network; use the convolutional layer of a convolutional neural network to capture local features in a sentence and then generate sentence representation; generate sentence representation by performing self-attention calculation on each word in a sentence through a self-attention mechanism (such as Transformer): such as the BERT model;

[0082] In the step S1-3, specifically:

[0083] (1) When performing vector indexing, the vector indexing types supported by Milvus use the Approximate Nearest Neighbor Search (ANNS) algorithm. Compared with the usually very time-consuming accurate retrieval, the core idea of ​​ANNS is no longer limited to returning the most accurate result, but only searching for the neighbors of the target;

[0084] For 128-dimensional floating point embedding, the storage occupied is 128*float size = 512 bytes, and the distance metrics used for floating point embedding are Euclidean distance L2 and inner product;

[0085] These types of indexes include FLAT, IVF_FLAT, IVF_PQ, IVF_SQ8, HNSW, and SCANN(beta) for CPU-based ANN search, and GPU_IVF_FLAT and GPU_IVF_PQ for GPU-based ANN search.

[0086] (2) When performing retrieval algorithms, Milvus supports a variety of similarity measurement methods, such as Euclidean distance (L2), cosine similarity, inner product (IP), etc. Different business scenarios may be suitable for different measurement methods. When creating an index, Milvus builds the corresponding index structure and performs query operations based on the specified measurement type.

[0087] Euclidean distance is the most intuitive distance measurement method, which calculates the straight-line distance between two vectors in multidimensional space. The formula is as follows:

[0088]

[0089] Cosine similarity measures the similarity between two vectors by calculating the cosine value of the angle between them. The value range of cosine similarity is between [-1,1]. The larger the value, the more similar the vectors are. The formula is as follows:

[0090]

[0091] Inner product is the simplest method for calculating vector similarity, which directly calculates the dot product of two vectors. The formula is as follows:

[0092]

[0093] In step S2, using GPTs technology, in ChatGPT, according to the personalized creation method, using prompt words, a corresponding customized repayment reminder large model GPT is generated in one click. Find "My GPT" in ChatGPT, click "Create a GPT", and enter the creation process of the personalized repayment reminder large model GPT:

[0094] (1) Specifying basic information of personalized collection GPT: In Configure, formulate a scenario for the recommendation of collection speech. The "Name" is GPT-CUI, and the description can be added as "This is a GPT that reminds customers to repay their debts in the collection field";

[0095] (2) Scenario application description of GPT-CUI for formulating repayment reminders: Setting reminder assistant prompts: Roles and goals. As an assistant in formulating repayment reminder strategies, your task is to analyze and determine the most appropriate repayment reminder time and communication channel based on the customer's outstanding information, consumption information, and social media activity information.

[0096] It is necessary to comprehensively consider various information about customers and formulate a personalized repayment reminder strategy.

[0097] Thinking path:

[0098] 1. Analyze the customer’s outstanding debt information, including the amount of outstanding debt, duration of outstanding debt, historical repayment record, etc.

[0099] 2. Check the customer's consumption information to understand the customer's consumption habits, purchase frequency and purchasing power level.

[0100] 3. Examine the customer’s social media activity information to determine the time periods when the customer is more active and the preferred communication method.

[0101] 4. Based on the above information, determine when is the most appropriate time to send repayment reminders, ensuring that customers receive reminders in a timely manner without causing excessive disturbance to customers.

[0102] 5. Determine the appropriate communication channel, whether it is phone, text message, email or private message on social media, etc., considering the customer's usage habits and acceptance.

[0103] 6. If there is insufficient or contradictory information, priority should be given to ensuring the effectiveness of reminders while minimizing disruption to customers.

[0104] Personalization: The response should be clear and concise, and directly provide the recommended repayment reminder time and communication channel.

[0105] When giving recommendations, briefly explain why you chose that time and channel to increase trust.

[0106] The tone of your response should be professional yet friendly, and avoid using overly blunt or emotional terms.

[0107] (2) Upload successful debt collection cases and upload a preset amount of historical customer debt information, consumption information, and social media information as a stored knowledge base, which will be used as model embedding when formulating repayment strategies later;

[0108] (3) After completing the above steps, publish and generate the GPT-CUI model adapted for repayment reminders; (4) Generate repayment reminder strategy: obtain the customer list information that needs to be reminded of repayment from the collection system, and use langchain4j to query the latest customer information in Milvus.

[0109] In the step (4), specifically:

[0110] A. First, make an encoder for the customer and the associated information in Milvus:

[0111] Multi-Head Self-Attention: Capture the global information in the sequence by paying attention to the input sequence itself;

[0112] Feed-Forward Neural Network: Feed-Forward Neural Network is realized through two linear transformations and a ReLU activation function;

[0113] Each sub-layer is followed by a residual connection and layer normalization.

[0114] B. Decode the natural language processing results:

[0115] Masked Multi-Head Self-Attention: Prevents the decoder from seeing future information while generating a sequence; performs attention operations on the encoder output and the decoder input;

[0116] C. Generate repayment reminder strategy: Langchain4j parses the data results, formats json, generates the final repayment reminder strategy, and stores it in the PostgreSQL of the business system for easy use next time.

[0117] In the step S3, specifically:

[0118] (1) Dynamic decision-making based on COT:

[0119] (1-1) Summarize the customer's repayment information and social media information;

[0120] (1-2) Find the base model based on the customer summary information and repayment strategy;

[0121] (1-3) Use the base model to generate reminder content in conjunction with the customer's information summary;

[0122] (2) Multimodal content generation:

[0123] (2-1) Generative Adversarial Network:

[0124] GANs is a powerful generative model consisting of a generator and a discriminator. The generator generates data, and the discriminator tries to distinguish between real data and generated data; initialize the generator G and the discriminator D; for each batch of data x, the generator G generates a set of data G(x); the discriminator D classifies the real data x and the generated data G(x); update G and D using adversarial training.

[0125] (2-2) Transformer architecture: Transformers is a deep neural network model based on the self-attention mechanism and is widely used to process sequence data.

[0126] Initialize the encoder and decoder: Encode the input data to get the encoder output. Use the decoder to decode the encoder output to generate the output sequence.

[0127] (2-3) Generate images using GANs and convert them into text descriptions through transformers. Generate images again using the generated text descriptions, forming a cycle; adjust model parameters to optimize the generated results.

[0128] The present invention also relates to a debt collection and repayment reminder system based on a large model multi-modality, comprising:

[0129] The information collection module is used to obtain the customer's repayment information, consumption information and activity information; the repayment prediction module is used to analyze the obtained customer information through the repayment evaluation model, and use RAG technology to combine the user's latest information to obtain the customer's repayment reminder strategy;

[0130] The content generation module is used to generate the corresponding reminder content according to the repayment reminder strategy, and send it to the customer at the corresponding time node in combination with the corresponding channel in the repayment strategy; the content generation module uses LangChain Agent technology, in conjunction with a series of large model bases for multimodal content generation, Sora, Stable Diffusion 3, chatgpt4 and related docking interfaces that integrate the internal financial system, to generate an intelligent agent AiAgent that is professionally used for reminder content generation. The intelligent agent obtains the list of customers who need to be reminded from the collection system, and according to the reminder strategy, based on the COT principle, dynamically decides to call the corresponding model base to generate personalized content that fits the customer. And the content will be pushed to the customer at the corresponding time node of the repayment strategy.

[0131] Sora's personalized reminder video generation uses Scalable diffusion models with transformers at the bottom. It learns the SSM (State Space Model) at the visual level or in the superficial sense on the spatiotemporal latent variable fragments, thus showing a strong emergence ability in video generation: the consistency of movement of people and scenery in three-dimensional space; long-range temporal correlation and object persistence, such as the reappearance of objects after being blocked; the interactivity between objects and the surrounding world; simulated digital world, etc.

[0132] Stable Diffusion3 uses a new architecture MMDiT at the bottom. In text-to-image generation, the model needs to consider both text and image information. The new architecture can handle a variety of different information. As with the previous stable diffusion version, the SD3 team uses pre-trained models to obtain appropriate text and image representations.

[0133] In this application, the improvements over the prior art are:

[0134] 1. Personalization of reminder content

[0135] The main feature of this technology is the generation of personalized reminder content. Different from the fixed template of traditional intelligent reminders, this technology uses the industry's advanced multimodal large model, Sora (Venus video model) + Stable Diffusion 3 (Venus image model) + chatgpt4 (dialogue task large model), combined with the customer's personal business information, to dynamically generate reminder content with the customer's personal color. For example, if the information collection module recognizes that today is the customer's birthday, it can dynamically generate videos, pictures, and texts with blessings in the repayment reminder content. If it is recognized that the customer's daily communication is in dialect, a repayment reminder voice in the same dialect as the user can be generated. The generated content makes customers feel friendly and easier to accept.

[0136] Multimodal Generation, a multimodal generation technology, allows the model to simultaneously process and generate multiple types of modal data, such as text, images, audio, etc. The underlying application is based on Generative Adversarial Networks (GANs) and Transformer architectures.

[0137] 2. Convenient and automated upgrade of reminder model generation

[0138] Through GPTs, a large model GPT-CUI for repayment reminder prediction is extremely easy to build without consuming a lot of resources for training and fine-tuning. Combined with the GPTs Knowledge base, the latest customer information obtained from the information collection module can be uploaded to the Knowledge base in real time to achieve automatic iteration and upgrade of the model.

[0139] Optimized training of large models: Distributed training distributes the training tasks of large models to multiple machines or devices, using TensorFlow's distributed strategy and PyTorch's distributed training framework to speed up training.

[0140] For the latest data of customers, the small batch gradient descent method is used for training. Each time, a small sample set is randomly selected from the training sample set, and the weights are updated using FG iteration on the selected small sample set. The number of sample points contained in the selected small sample set is called batch_size, which is usually set to a power of 2, which is more conducive to GPU acceleration processing. In particular, if batch_size = 1, it becomes SG; if batch_size = n, it becomes FG. Its iteration form is:

[0141] 3. Timeliness:

[0142] By using RAG technology, when formulating repayment strategies, the model combines the customer's latest information, continuously adapts to the customer's latest changes, and generates a repayment strategy that is closest to the customer's current situation.

[0143] 4. Large model frame langchain4j:

[0144] It innovatively provides a Java real-time large model framework langchain4j (langchain forjava), which makes up for the lack of Java technology stack in the AI ​​field. This framework is jointly developed with a team in the United States and is currently open to the public on GitHub. It has received more and more forks and contributions from developers.

[0145] An embodiment of the present application also provides a computer-readable storage medium, which can be set in a media resource aggregation device to store at least one instruction or at least one program for implementing the at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the media resource aggregation method provided by the above method embodiment.

[0146] In this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0147] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the media resource aggregation method provided in the method embodiment.

[0148] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0149] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of the present application are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0150] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0151] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0152] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0153] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A debt collection and repayment reminder method based on a large model and multi-modality, characterized in that: The following steps are involved: S1. Obtain the customer's repayment information, consumption information and activity information; S2, analyzing the customer information obtained in step S1 through the repayment assessment model, and using RAG technology to combine the customer information to obtain the customer's repayment reminder strategy; S3. Generate corresponding reminder content according to the repayment reminder strategy, and send it to the customer at the corresponding time node in combination with the corresponding channel in the repayment strategy.

2. The large-model multi-modal debt collection and repayment reminder method according to claim 1 is characterized in that: The step S1 is specifically as follows: S1-1. Obtain the latest loan information of customers from the financial system every half an hour; obtain the consumption information, activity information and business information of customers from Internet vendors; S1-2. Use the vector model EmbeddingModel in langchain4j to vectorize customer information based on the neural network algorithm embedding; S1-3. Use the MilvusEmbeddingStore tool of langchai4j to persistently store the vectorized data in step S1-2 in the open source vector database Milvus, and locate customer information based on the retrieval algorithm of the Milvus vector database.

3. The large-model multi-modal debt collection and repayment reminder method according to claim 2 is characterized in that: In the step S1-2, the customer information text is vectorized. In the text vectorization process, the data in the text is represented as vectors according to two categories of words and sentences with complete expression meanings, specifically: (1) When performing word vectorization, a single word is converted into a numerical vector: (1-1) Assign a unique binary vector to each word, where only one position is 1 and the rest are 0; (1-2) Map each word to a real vector that is semantically related; (2) When performing sentence vectorization, the entire sentence is converted into a numerical vector: (2-1) Average the word vectors in the sentence or perform weighted average based on word frequency; Generate sentence representation by recursively processing each word in the sentence through a recurrent neural network; Use the convolutional layers of a convolutional neural network to capture local features in a sentence and then generate a sentence representation; Through the self-attention mechanism, self-attention calculation is performed on each word in the sentence to generate sentence representation.

4. The large-model multi-modal debt collection and repayment reminder method according to claim 2 is characterized in that: In the step S1-3, specifically: When performing retrieval algorithms, Milvus supports similarity measurement methods. When creating an index, Milvus builds the corresponding index structure and performs query operations based on the specified measurement type.

5. The large-model multi-modal debt collection and repayment reminder method according to claim 2 is characterized in that: In the step S2, specifically: (1) Set reminder assistant prompt words; (2) Upload successful debt collection cases and upload a preset amount of historical customer debt information, consumption information, and social media information as a stored knowledge base, which will be used as model embedding when formulating repayment strategies later; (3) After completing the above steps, publish and generate the model GPT-CUI adapted for repayment reminders; (4) Generation of repayment reminder strategy: Obtain the customer list that needs to be reminded to repay from the collection system, and use langchain4j to query the latest customer information in Milvus.

6. The large-model multimodal debt collection and repayment reminder method according to claim 5 is characterized in that: In the step (4), specifically: A. First, make an encoder for the customer and the associated information in Milvus: A1. Capture the global information in the sequence by performing attention operation on the input sequence itself; A2. Implement a feedforward neural network through two linear transformations and a ReLU activation function; A3. Each sub-layer is followed by a residual connection and layer normalization; B. Decode the natural language processing results: mask the multi-head self-attention to prevent the decoder from seeing future information in the process of generating sequences, and perform attention operations through the encoder output and decoder input; C. Generate repayment reminder strategy: Langchain4j parses the data results, formats json, generates the final repayment reminder strategy, and stores it in the PostgreSQL of the business system for easy use next time.

7. The large-model multi-modal debt collection and repayment reminder method according to claim 1 is characterized in that: In the step S3, specifically: (1) Dynamic decision-making based on COT: (1-1) Summarize the customer's repayment information and social media information; (1-2) Find the base model based on the customer summary information and repayment strategy; (1-3) Use the base model to generate reminder content in conjunction with the customer's information summary; (2) Multimodal content generation: (2-1) Generative Adversarial Network: The generator generates data, and the discriminator tries to distinguish between real data and generated data; Initialize the generator G and the discriminator D; For each batch of data x, the generator G generates a set of data G(x); The discriminator D classifies the real data x and the generated data G(x); Update G and D using adversarial training; (2-2) Converter architecture: Initialize the encoder and decoder; Encode the input data and obtain the encoder output; Decode the encoder output using the decoder to generate an output sequence; (2-3) Use GANs to generate images and convert them into text descriptions through transformers; use the generated text descriptions to generate images again, forming a cycle; adjust model parameters to optimize the generation results.

8. A debt collection and repayment reminder system based on a large model and multi-modality, characterized in that: include: Information collection module, used to obtain customers' repayment information, consumption information and activity information; The repayment prediction module is used to analyze the acquired customer information through the repayment assessment model, and use RAG technology to combine the user's latest information to obtain the customer's repayment reminder strategy; The content generation module is used to generate corresponding reminder content according to the repayment reminder strategy, and send it to customers at the corresponding time node in combination with the corresponding channel in the repayment strategy.

Images