Method and device for determining collection mode, equipment and medium
By constructing a collection method determination model based on MDP and using offline reinforcement learning algorithms to train personalized collection methods, the problem of low accuracy of collection methods in existing technologies is solved, and more efficient default handling results are achieved.
Patent Information
- Application Number
- CN202411079841.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-12
AI Technical Summary
In existing technologies, the rules for determining bank collection methods lack personalization, resulting in poor collection effectiveness and low accuracy, which in turn affects the bank's revenue from default disposal.
A collection method based on Markov Decision Process (MDP) is adopted to determine the model. By constructing a state space, action space, policy model and reward function, the model is trained using an offline reinforcement learning algorithm. With maximizing policy returns as the optimization objective, a personalized collection method is generated.
It improved the accuracy of collection methods, reduced the default rate, increased the revenue from default handling business, and enhanced the collection effect.
Smart Images

Figure CN118941380A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence, and specifically to a method, device, equipment and medium for determining a collection method. Background Art
[0002] In the bank's credit business, there is a risk of bad debts caused by default, overdue repayment and non-repayment of borrowers, which will cause losses to the bank. In order to reduce the losses caused by customer default, the bank will adopt certain collection methods to urge customers to repay as soon as possible.
[0003] In the prior art, professionals usually formulate unified rules for determining collection methods. Specifically, the corresponding collection method is determined according to the overdue time and number of overdue times of the loan. Using the same collection method determination rules for different customers has the problem of low accuracy of the determined collection method, which in turn leads to poor collection results when the bank collects debts according to the determined collection method. Summary of the invention
[0004] The embodiments of the present application provide a method, device, equipment and medium for determining a collection method, which are used to improve the accuracy of determining a collection method.
[0005] In a first aspect, a method for determining a collection method is provided, the method comprising:
[0006] Obtain target object information of the target object and the target default stage of the target object;
[0007] The target object information and the target default stage are input into the collection method determination model to obtain the target collection method corresponding to the target object, wherein the collection method determination model is obtained based on the Markov decision process MDP of default disposal, and the target value corresponding to the target collection method is the largest, and the target value is related to the repayment amount and the repayment extension period.
[0008] In a possible embodiment, the collection method determination model is trained in the following manner:
[0009] Construct MDP;
[0010] Based on historical data, a training sample set is constructed, wherein each training sample includes: the sample default stage, sample object information and sample collection method corresponding to each moment of the corresponding sample object in the default process, and the default process represents the default process from the default start state to the default end state;
[0011] Based on the training sample set and MDP, the collection method determination model is obtained with maximizing the strategy benefit as the optimization goal.
[0012] Through the above method, the accumulated historical data is used to construct the training sample set of the Markov decision model, which ensures the reliability of the training sample set and thus ensures the accuracy of the model.
[0013] In a possible embodiment, constructing an MDP includes:
[0014] Construct the state space of MDP based on the information of each candidate object and each candidate default stage;
[0015] Construct the behavior space of MDP based on each candidate collection method;
[0016] Construct the MDP policy model, reward function, and policy benefits.
[0017] Through the above method, the entire cycle of credit default disposal is modeled as a Markov decision process, with the combination of the default stage and object information of the default object as the state space. It describes the state of the default object at each moment in the process from the initial stage of default to the final repayment or the end stage of default with the formation of bad debts, so as to explore the connection between the default objects in various default states and improve the accuracy of the model.
[0018] In a possible embodiment, the strategy model is a model whose input is candidate object information and candidate default stage, and whose output is the probability of each candidate collection method;
[0019] The reward function is related to the candidate information, candidate default stage and candidate collection method at each moment in the default process;
[0020] The strategy return is the expectation of the total return of the default process, which is determined based on the reward function and the decay coefficient.
[0021] In a possible embodiment, based on the training sample set and the MDP, a collection method determination model is obtained with maximizing the strategy benefit as the optimization goal, including:
[0022] An offline reinforcement learning algorithm is used to train the MDP strategy model based on the training sample set and to maximize the strategy benefit as the optimization goal, so as to obtain the collection method determination model.
[0023] Through the above method, the offline reinforcement learning training model is used to improve data utilization efficiency, improve the generalization ability of the model, apply it to complex environments, and solve the distribution shift problem. And with maximizing the strategy return as the optimization goal, the return on default disposal business is increased and the bad debt rate is reduced.
[0024] In a possible embodiment, based on the training sample set and the MDP, with maximizing the strategic benefit as the optimization goal, after obtaining the collection method determination model, the following is further included:
[0025] Conduct rationality check and grayscale test on the collection method determination model.
[0026] Through the above method, the reliability of the collection method determination model is improved.
[0027] In a second aspect, the present application provides a device for determining a collection method, the device comprising:
[0028] An acquisition module, used to acquire target object information of a target object and a target default stage of the target object;
[0029] The determination module is used to input the target object information and the target default stage into the collection method determination model to obtain the target collection method corresponding to the target object, wherein the collection method determination model is obtained based on the Markov decision process MDP of default disposal, the target value corresponding to the target collection method is the largest, and the target value is related to the repayment amount and the repayment extension period.
[0030] In a possible embodiment, the device further includes a training module, and the training module is used to:
[0031] Construct MDP;
[0032] Based on historical data, a training sample set is constructed, wherein each training sample includes: the sample default stage, sample object information and sample collection method corresponding to each moment of the corresponding sample object in the default process, and the default process represents the default process from the default start state to the default end state;
[0033] Based on the training sample set and MDP, the collection method determination model is obtained with maximizing the strategy benefit as the optimization goal.
[0034] In a possible embodiment, when constructing the MDP, the training module is further used to:
[0035] Construct the state space of MDP based on the information of each candidate object and each candidate default stage;
[0036] Construct the behavior space of MDP based on each candidate collection method;
[0037] Construct the MDP policy model, reward function, and policy benefits.
[0038] In a possible embodiment, the strategy model is a model whose input is candidate object information and candidate default stage, and whose output is the probability of each candidate collection method;
[0039] The reward function is related to the candidate information, candidate default stage and candidate collection method at each moment in the default process;
[0040] The strategy return is the expectation of the total return of the default process, which is determined based on the reward function and the decay coefficient.
[0041] In a possible embodiment, based on the training sample set and the MDP, when the collection method determination model is obtained with maximizing the strategy benefit as the optimization goal, the training module is further used to:
[0042] An offline reinforcement learning algorithm is used to train the MDP strategy model based on the training sample set and to maximize the strategy benefit as the optimization goal, so as to obtain the collection method determination model.
[0043] In a possible embodiment, based on the training sample set and the MDP, with maximizing the strategic benefit as the optimization goal, after obtaining the collection method determination model, the training module is further used to:
[0044] Conduct rationality check and grayscale test on the collection method determination model.
[0045] In a third aspect, the present application provides an electronic device, including:
[0046] A memory for storing program instructions;
[0047] The processor is used to call the program instructions stored in the memory, and execute the steps included in any one of the methods in the first aspect according to the obtained program instructions.
[0048] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes any one of the methods described in the first aspect.
[0049] In a fifth aspect, the present application provides a computer program product, comprising: a computer program code, when the computer program code is run on a computer, the computer executes any one of the methods described in the first aspect.
[0050] In an embodiment of the present application, the target object information of the target object and the target default stage of the target object are obtained, and then the target object information and the target default stage are input into the collection method determination model to obtain the target collection method corresponding to the target object, wherein the collection method determination model is obtained based on the Markov decision process MDP of default disposal, and the target value corresponding to the target collection method is the largest, and the target value is related to the repayment amount and the repayment extension time. In this way, the object personalization is taken into account, and different target disposal rules are adopted for different target objects, which improves the accuracy of the collection method determination, and the evolution relationship between each default stage is considered through MDP, which further improves the accuracy of the collection method determination, and outputs the target collection method with the largest target value. The loss of the target collection method is the smallest, which increases the income of the default disposal business, reduces the bad debt rate, and has a good default disposal effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;
[0052] Figure 2 A schematic diagram of a flow chart of obtaining a collection method determination model in an embodiment of the present application;
[0053] Figure 3 A schematic diagram of the default stage state and state transition in an embodiment of the present application;
[0054] Figure 4 A flowchart of a method for determining a collection method in an embodiment of the present application;
[0055] Figure 5 This is a schematic diagram of the architecture of the method for determining the collection method in the embodiment of the present application;
[0056] Figure 6 This is a schematic diagram of the structure of a device for determining a collection method in an embodiment of the present application;
[0057] Figure 7 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0058] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.
[0059] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.
[0060] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description. It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.
[0061] In the technical solution of this application, the acquisition, transmission, storage, and use of data are in compliance with the requirements of relevant national laws and regulations.
[0062] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:
[0063] Markov Decision Process (MDP): A mathematical model of sequential decision making, used to simulate the random strategies and rewards that can be achieved by an agent in an environment where the system state has Markov properties. MDP is built based on a set of interacting objects, namely the agent and the environment, and its elements include state, action, strategy and reward. In the simulation of MDP, the agent will perceive the current system state, take actions on the environment according to the strategy, thereby changing the state of the environment and receiving rewards. The accumulation of rewards over time is called rewards.
[0064] Before introducing the method for determining the collection method provided in the embodiment of the present application, in order to facilitate understanding, the technical background of the embodiment of the present application is first introduced in detail below.
[0065] At present, in the bank's credit business, there is a risk of bad debts caused by default, overdue repayment and non-repayment of borrowers, which will cause losses to the bank. In order to reduce the losses caused by customer default, the bank will adopt certain collection methods to urge customers to repay as soon as possible.
[0066] In the prior art, professionals usually formulate unified rules for determining collection methods. Specifically, the corresponding collection method is determined according to the overdue time and number of overdue times of the loan. Using the same collection method determination rules for different customers has the problem of low accuracy of the determined collection method, which in turn leads to poor collection results when the bank collects debts according to the determined collection method.
[0067] In view of this, in an embodiment of the present application, a method, device, equipment and medium for determining a collection method are provided. The target object information of the target object and the target default stage of the target object are obtained, and then the target object information and the target default stage are input into the collection method determination model to obtain the target collection method corresponding to the target object, wherein the collection method determination model is obtained based on the Markov decision process MDP of default disposal, and the target value corresponding to the target collection method is the largest, and the target value is related to the repayment amount and the repayment extension time. In this way, object personalization is taken into account, and different target disposal rules are adopted for different target objects, which improves the accuracy of the collection method determination, and the evolution relationship between each default stage is considered through MDP, which further improves the accuracy of the collection method determination, and outputs the target collection method with the largest target value, which improves the income of the default disposal business, reduces the bad debt rate, and has a good default disposal effect.
[0068] The preferred implementation modes of the present application are described in detail below with reference to the accompanying drawings.
[0069] See also Figure 1, which is a schematic diagram of possible application scenarios in the embodiments of the present application. The application scenario diagram includes a server 110 and terminal devices 120 (including terminal devices 1201, 1202, ..., 120n).
[0070] The server 110 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal device 120 and the server 110 may be directly or indirectly connected via wired or wireless communication, which is not limited in this application.
[0071] The terminal device 120 includes but is not limited to mobile phones, tablet computers, laptops, desktop computers, e-book readers, intelligent voice interaction devices, smart home appliances, car terminals and other devices; various software can be installed on the terminal device, such as applications and applets.
[0072] It should be noted that Figure 1 The figure is only an example. In fact, the number of terminal devices 120 and servers 110 is not limited and is not specifically limited in the embodiments of the present application.
[0073] In an embodiment of the present application, any one or all of the server 110 and the terminal device 120 may be configured with a trained collection method determination model, so that the trained collection method determination model can be used to determine the target collection method for the target object.
[0074] Taking the example of the terminal device 120 using a trained collection method determination model to determine the target collection method of the target object, the collection method determination model used by the terminal device 120 may be obtained through its own training, or may be deployed to the terminal device 120 after other devices (such as the server 110 or other terminal devices) complete the training of the collection method determination model.
[0075] In the following description of this application, the training process and application process of the collection method determination model will be schematically explained from the perspective of the processing device. According to different application scenarios of this application, the processing device may specifically correspond to Figure 1 This application does not make any specific restrictions on any of the server 110 and terminal device 120 shown in the figure.
[0076] In addition, in the embodiment of the present application, according to actual processing needs, training to obtain a collection method determination model can be a periodic process, and the training sample set can be periodically regenerated to train the collection method determination model.
[0077] See also Figure 2 As shown, it is a flow chart of obtaining a collection method determination model in an embodiment of the present application. Figure 2 , the specific operations performed by the model to determine the collection method are described in detail:
[0078] Step 20: Construct the MDP.
[0079] In the embodiment of the present application, the MDP includes a state space, an action space, a strategy model, a reward function, and a strategy benefit.
[0080] In the embodiment of the present application, a state space of the MDP is constructed based on information of each candidate object and each candidate default stage, and the state space consists of two parts: object information state and default stage state.
[0081] Among them, the default stage status is s t Indicates that t represents the current Markov moment, which is an integer counting from 0. The unit of t can be day. The default stage state is a fixed state determined by the credit business.
[0082] For example, see Figure 3 As shown, it is a schematic diagram of the default stage status and state transition in the embodiment of the present application, wherein the default stage status includes: overdue for 1 to 7 days, overdue for 8 to 30 days, overdue for 31 to 60 days, overdue for 61 to 90 days, overdue for 91 to 180 days, overdue for more than 180 days, overdue for 1 to 7 days is the default starting state, and according to the repayment situation of the object, it enters the next default stage or stays in the current default stage, normal (recovering the penalty for breach of contract) and bad debt are the two default end states, and the final profit can be calculated in the default end state.
[0083] Among them, the object information status is represented by u t Indicates that t represents the current Markov moment, which is an integer counting from 0. The unit of t can be day. The object information status includes: basic characteristics such as customer gender, age, occupation, and statistical characteristics such as customer income, expenditure, and various loan information in the past period of time.
[0084] In the embodiment of the present application, the behavior space of the MDP is constructed based on various preset candidate collection methods.
[0085] The behavior space is the set of collection methods adopted by the bank on the defaulting party, represented by A. Let a be one of the candidate collection methods, a tIt represents the collection method adopted by the bank at time t in the Markov process.
[0086] Exemplarily, the candidate collection methods are shown in Table 1, which is not limited in the embodiments of the present application:
[0087] Table 1
[0088] Serial number Candidate default collection methods 1 SMS collection 2 Telephone collection 3 Negotiation-Personalized Installment 4 Negotiation-Extension 5 Third-party collection agencies 6 Sue in court 7 Overdue debt package transfer
[0089] In the embodiment of the present application, the strategy model is a model whose input is candidate object information and candidate default stage, and whose output is the probability of each candidate collection method. The strategy of the Markov decision process is the conditional probability distribution of taking different actions according to the current state. The state consists of two parts: the default stage s and the object information u, so the strategy model is expressed as: θ (a|s,u)=P[a|s,u,θ].
[0090] Among them, θ is a learnable parameter, π θ is a function that is fitted using a neural network.
[0091] In the embodiment of the present application, the reward function is related to the candidate object information, the candidate default stage and the candidate collection method at each moment in the default process.
[0092] Among them, the reward function at a certain moment in the default process can be expressed as: R t = r(s t ,u t ,a t ), s t , is the candidate default stage at a certain moment, u t is the candidate object information at a certain moment, a t The candidate collection methods to be taken at a certain moment.
[0093] In the embodiment of the present application, the strategy benefit is the expectation of the total benefit of the default process. The total benefit of the default process is determined based on the reward function and the decay coefficient.
[0094] In the embodiment of the present application, in the default end state, the final benefit can be calculated, measured by the logarithm of the default amount, the normal state is positive, and the bad debt state is zero. At the same time, the longer the time from the default start state to the default end state, the lower the benefit will be. The total benefit of the default process can be expressed as:
[0095] Among them, H is the total duration of the default process, γ t is the attenuation coefficient at time t, R tis the reward value of the attenuation coefficient at time t, the attenuation coefficient γ∈(0,1], the closer γ is to 0, the higher the importance of the repayment extension time, and vice versa. If γ=1, the repayment extension time has no effect on the income.
[0096] In the embodiment of the present application, the strategy benefit of strategy π can be expressed as:
[0097]
[0098] For example, a training sample with a default process duration of H can be expressed as: τ = (s0, u0, a0, s1, u1, a1, ..., s H ,u H ,a H ), the probability distribution of the sequence strategy of the default process can be expressed as: Then the expected return of the default process can be expressed as:
[0099]
[0100] In this way, the entire cycle of credit default disposal is modeled as a Markov decision process, with the combination of the default stage and object information of the default object as the state space, describing the state of the default object at each moment in the process from the initial stage of default to the final repayment or the end stage of default with the formation of bad debts, so as to explore the connection between the default objects in various default states and improve the accuracy of the model.
[0101] Step 21: Construct a training sample set based on historical data.
[0102] Among them, each training sample includes: the sample default stage, sample object information and sample collection method corresponding to the corresponding sample object at each moment in the default process. The default process is characterized from the default start state to the default end state. The each moment in the default process can be the moment when the collection method is adopted each time, or it can be each moment with a preset time interval, for example, each moment is the first day, the second day...the nth day, which is not limited in the embodiments of the present application.
[0103] In the embodiment of the present application, historical data is processed to construct a training sample set.
[0104] Among them, historical data includes business data, default object information and overdue behavior data, default handling records and object repayment performance, etc., which are not limited in the embodiments of the present application.
[0105] For example, a training sample can be expressed as: τ = (s0,u0,a0,s1,u1,a1,…,s H ,u H ,a H), H is the total duration of the default process of the sample object.
[0106] In addition, it is worth noting that each training sample also includes: the reward value corresponding to the corresponding sample object at each moment in the default process, which is not limited in the embodiment of the present application. A training sample can be expressed as: τ = (s0, u0, a0, R0, s1, u1, a1, R1, …, s H ,u H ,a H ,R H ), which can further improve the accuracy of the model.
[0107] In this way, the training sample set of the Markov decision model is constructed using the accumulated historical data, which ensures the reliability of the training sample set and thus ensures the accuracy of the model.
[0108] Step 22: Based on the training sample set and MDP, the collection method determination model is obtained with maximizing the strategy benefit as the optimization goal.
[0109] In the embodiment of the present application, an offline reinforcement learning algorithm is used to train the MDP strategy model based on a training sample set with maximizing the strategy benefit as the optimization goal to obtain a collection method determination model.
[0110] The offline reinforcement learning algorithm may be an algorithm with good learning efficiency and stability, such as Implicit Q-Learning (IQL) or Conservative Q-learning (CQL).
[0111] In the embodiment of the present application, based on the principle of strategy benefits and importance sampling, the strategy to be optimized π θ The revenue can be expressed as: Among them, π D (a t |s t ,ut) is the conditional probability of the collection method of a training sample at the Markov decision time t, π θ (a t |s t ,ut) is the conditional probability of the collection method of the strategy to be optimized at the Markov decision time t. An offline reinforcement learning algorithm is used to maximize the strategy benefit J(π θ ) is taken as the optimization objective, and the collection method determination model is obtained through learning.
[0112] In this way, the offline reinforcement learning training model improves data utilization efficiency, improves the generalization ability of the model, is applicable to complex environments, and solves the distribution shift problem. And with maximizing the strategy return as the optimization goal, it increases the return on default disposal business and reduces the bad debt rate.
[0113] Furthermore, after obtaining the collection method determination model, in order to improve the reliability of the collection method determination model, the embodiment of the present application can also perform rationality verification and grayscale testing on the collection method determination model.
[0114] Based on the above embodiments, see Figure 4 FIG. 1 is a flow chart of a method for determining a collection method in an embodiment of the present application, which specifically includes:
[0115] Step 40: Obtain target object information of the target object and the target default stage of the target object.
[0116] Among them, the target object information includes: basic characteristics such as customer gender, age, occupation and statistical characteristics such as customer's income, expenditure, various loan information in the past period of time; the target default stages include: overdue for 1 to 7 days, overdue for 8 to 30 days, overdue for 31 to 60 days, overdue for 61 to 90 days, overdue for 91 to 180 days, overdue for more than 180 days, etc.
[0117] Step 41: Input the target object information and the target default stage into the collection method determination model to obtain the target collection method corresponding to the target object.
[0118] Among them, the collection method determination model is obtained based on the Markov decision process MDP of default disposal. The target value corresponding to the target collection method is the largest, that is, the target collection method is used to collect the target object, and the return generated is the largest. The target value is related to the repayment amount and the repayment extension period. The shorter the repayment extension period, the larger the target value. Among them, the repayment amount is related to the object repayment amount and the cost generated by the collection method.
[0119] In an embodiment of the present application, the target object information and the target default stage are used as input parameters and input into a collection method determination model to obtain the result output by the collection method determination model, thereby obtaining the target collection method corresponding to the target object.
[0120] Based on the above embodiments, see Figure 5 FIG. 1 is a schematic diagram of the architecture of the method for determining the collection method in an embodiment of the present application, which specifically includes:
[0121] First, the model training unit processes the historical data to obtain the training sample set of the MDP. Then, based on the offline reinforcement learning algorithm, the constructed MDP is trained with the maximization of strategy benefits as the optimization goal to obtain the default disposal determination model. Secondly, the service deployment unit deploys the default disposal determination model, inputs the target object information and target default stage of the target object into the collection method determination model, and obtains the target collection method corresponding to the target object output by the default disposal determination model.
[0122] Based on the same inventive concept, the present application embodiment provides a device for determining a collection method, please refer to Figure 6 , the device comprises:
[0123] An acquisition module 601 is used to acquire target object information of a target object and a target default stage of the target object;
[0124] Determination module 602 is used to input the target object information and the target default stage into the collection method determination model to obtain the target collection method corresponding to the target object, wherein the collection method determination model is obtained based on the Markov decision process MDP of default disposal, and the target value corresponding to the target collection method is the largest, and the target value is related to the repayment amount and the repayment extension period.
[0125] In a possible embodiment, the apparatus further includes a training module 603, and the training module 603 is used to:
[0126] Construct MDP;
[0127] Based on historical data, a training sample set is constructed, wherein each training sample includes: the sample default stage, sample object information and sample collection method corresponding to each moment of the corresponding sample object in the default process, and the default process represents the default process from the default start state to the default end state;
[0128] Based on the training sample set and MDP, the collection method determination model is obtained with maximizing the strategy benefit as the optimization goal.
[0129] In a possible embodiment, when constructing the MDP, the training module 603 is further used to:
[0130] Construct the state space of MDP based on the information of each candidate object and each candidate default stage;
[0131] Construct the behavior space of MDP based on each candidate collection method;
[0132] Construct the MDP policy model, reward function, and policy benefits.
[0133] In a possible embodiment, the strategy model is a model whose input is candidate object information and candidate default stage, and whose output is the probability of each candidate collection method;
[0134] The reward function is related to the candidate information, candidate default stage and candidate collection method at each moment in the default process;
[0135] The strategy return is the expectation of the total return of the default process, which is determined based on the reward function and the decay coefficient.
[0136] In a possible embodiment, based on the training sample set and the MDP, with maximizing the strategic benefit as the optimization goal, when obtaining the collection method determination model, the training module 603 is further used to:
[0137] An offline reinforcement learning algorithm is used to train the MDP strategy model based on the training sample set and to maximize the strategy benefit as the optimization goal, so as to obtain the collection method determination model.
[0138] In a possible embodiment, based on the training sample set and the MDP, with maximizing the strategic benefit as the optimization goal, after obtaining the collection method determination model, the training module 603 is further used to:
[0139] Conduct rationality check and grayscale test on the collection method determination model.
[0140] Based on the same inventive concept, the embodiment of the present application provides an electronic device, which can realize the function of determining the collection method discussed above, please refer to Figure 7 , the device includes a processor 701 and a memory 702.
[0141] The processor 701 calls the program instructions stored in the memory 702, and the processor 701 is used to execute any method for determining the collection method in the embodiments of the present application according to the obtained program instructions.
[0142] Based on the same inventive concept, the embodiment of the present application provides a computer-readable storage medium, and the computer program product includes: computer program code, when the computer program code is executed on a computer, the computer executes any of the methods for determining the collection method discussed above. Since the principle of solving the problem by the above-mentioned computer-readable storage medium is similar to the method for determining the collection method, the implementation of the above-mentioned computer-readable storage medium can refer to the implementation of the method, and the repeated parts will not be repeated.
[0143] Based on the same inventive concept, the embodiment of the present application also provides a computer program product, which includes: computer program code, when the computer program code is run on a computer, the computer executes any of the collection method determination methods discussed above. Since the principle of solving the problem by the above computer program product is similar to that of the collection method determination method, the implementation of the above computer program product can refer to the implementation of the method, and the repeated parts will not be repeated.
[0144] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0145] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0146] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of user-operated steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0148] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A method for determining a collection method. It is characterized in that include: Obtain target object information of a target object and a target default stage of the target object; The target object information and the target default stage are input into the collection method determination model to obtain the target collection method corresponding to the target object, wherein the collection method determination model is obtained based on the Markov decision process MDP of default disposal, the target value corresponding to the target collection method is the largest, and the target value is related to the repayment amount and the repayment extension period.
2. The method according to claim 1, It is characterized in that The collection method determination model is trained in the following way: constructing the MDP; Based on historical data, a training sample set is constructed, wherein each training sample includes: a sample default stage, sample object information, and a sample collection method corresponding to each moment of the default process of the corresponding sample object, wherein the default process represents a process from a default start state to a default end state; Based on the training sample set and the MDP, the collection method determination model is obtained with maximizing the strategy benefit as the optimization goal.
3. The method according to claim 2, It is characterized in that The constructing of the MDP comprises: Constructing the state space of the MDP based on the information of each candidate object and each candidate default stage; Constructing the behavior space of the MDP based on each candidate collection method; Construct the policy model, reward function, and policy benefit of the MDP.
4. The method according to claim 3, It is characterized in that The strategy model is a model whose input is the candidate object information and the candidate default stage, and whose output is the probability of each candidate collection method; The reward function is related to the candidate object information at each moment in the default process, the candidate default stage and the candidate collection method; The strategy return is the expectation of the total return of the default process, where the total return of the default process is determined based on the reward function and the decay coefficient.
5. The method according to claim 2, It is characterized in that The method of obtaining the collection method determination model based on the training sample set and the MDP and taking maximizing the strategy benefit as the optimization goal includes: An offline reinforcement learning algorithm is used to train the MDP strategy model based on the training sample set and to maximize the strategy benefit as the optimization goal, so as to obtain the collection method determination model.
6. The method according to claim 2, It is characterized in that After obtaining the collection method determination model based on the training sample set and the MDP with maximizing the strategy benefit as the optimization goal, the method further includes: The collection method determination model is subjected to rationality check and grayscale test.
7. A device for determining a collection method, It is characterized in that include: An acquisition module, used to acquire target object information of a target object and a target default stage of the target object; A determination module is used to input the target object information and the target default stage into a collection method determination model to obtain a target collection method corresponding to the target object, wherein the collection method determination model is obtained based on a Markov decision process MDP of default handling, the target value corresponding to the target collection method is the largest, and the target value is related to the repayment amount and the repayment extension period.
8. The device according to claim 7, It is characterized in that Also included is a training module, the training module is used to: constructing the MDP; Based on historical data, a training sample set is constructed, wherein each training sample includes: a sample default stage, sample object information, and a sample collection method corresponding to each moment of the default process of the corresponding sample object, wherein the default process represents a process from a default start state to a default end state; Based on the training sample set and the MDP, the collection method determination model is obtained with maximizing the strategy benefit as the optimization goal.
9. The device according to claim 8, It is characterized in that When constructing the MDP, the training module is further used to: Constructing the state space of the MDP based on the information of each candidate object and each candidate default stage; Constructing the behavior space of the MDP based on each candidate collection method; Construct the policy model, reward function, and policy benefit of the MDP.
10. The device according to claim 9, It is characterized in that The strategy model is a model whose input is the candidate object information and the candidate default stage, and whose output is the probability of each candidate collection method; The reward function is related to the candidate object information at each moment in the default process, the candidate default stage and the candidate collection method; The strategy return is the expectation of the total return of the default process, where the total return of the default process is determined based on the reward function and the decay coefficient.
11. The device according to claim 8, It is characterized in that When the collection method determination model is obtained based on the training sample set and the MDP with maximizing the strategy benefit as the optimization goal, the training module is further used to: An offline reinforcement learning algorithm is used to train the MDP strategy model based on the training sample set and to maximize the strategy benefit as the optimization goal, so as to obtain the collection method determination model.
12. The device according to claim 8, It is characterized in that After obtaining the collection method determination model based on the training sample set and the MDP and taking maximizing the strategy benefit as the optimization goal, the training module is further used to: The collection method determination model is subjected to rationality check and grayscale test.
13. An electronic device, It is characterized in that include: A memory for storing program instructions; A processor is used to call the program instructions stored in the memory, and execute the steps included in any one of claims 1-6 according to the obtained program instructions.
14. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method according to any one of claims 1 to 6.
15. A computer program product, It is characterized in that The computer program product comprises: a computer program code, and when the computer program code is run on a computer, the computer is enabled to execute the method according to any one of claims 1 to 6.
