Loss customer risk control method and system based on dynamic reward shaping, and medium

Abstract

The invention discloses a risk control method and system for lost customers based on dynamic reward shaping and a medium, and relates to the technical field of financial intelligent prediction, and the method comprises the steps: generating multiple groups of virtual lost customer data; for each group of virtual lost client data, extracting a first state of the virtual lost client data; determining a first action from a preset lightweight action space according to the first state, and calculating a dynamic reward value for executing the first action in the first state according to a preset multi-target weight; if it is detected that the preset weight mutation is smaller than or equal to the preset change threshold value, training a preset Q value table according to the dynamic reward value until the Q value converges, and obtaining a target Q value table; and inputting the actual lost customer data into the target Q value table to obtain a target risk control strategy. According to the invention, the risk management and control efficiency of the financial institution on the multi-mode lost customer is improved, the collection cost is reduced, and the dynamic balance of collection income, cost control and customer relationship maintenance is realized.

Description

Technical Field

[0001] This application relates to the field of financial intelligent prediction technology, and particularly to a risk control method, system and medium for lost-contact customers based on dynamic reward shaping. Background Art

[0002] With the digital transformation of financial credit business, the risk management of loan customers' loss of contact has become increasingly complex. Traditional collection strategies mainly rely on fixed rules and manual experience, and use the linear weighted method to simplify the multi-objective optimization problem into a single static function, which is difficult to meet the real-time switching needs of target priorities in dynamically changing scenarios. For example, when facing the rapid loss of assets of high-risk customers, a stop-loss mechanism needs to be quickly activated; while for high-quality customers with short-term turnover difficulties but long-term cooperation potential, more gentle collection measures need to be taken to maintain good customer relationships. However, existing methods often cannot effectively balance these conflicting goals, resulting in improper resource allocation or missing key response windows.

[0003] In addition, there is a significant contradiction between data sensitivity and algorithm generalization ability in the existing technology. Due to privacy protection reasons, it has become extremely difficult to obtain real business data for model training and technical verification, which limits the iteration speed and optimization effect of the algorithm. The reward function designed based on expert experience lacks theoretical guidance, resulting in a significant decline in the generalization performance of the strategy when facing unknown loss-of-contact patterns. Especially when dealing with the behavior of changing from intermittent loss of contact to systematic transfer mode, the traditional fixed-weight method is prone to problems of strategy lag and resource misallocation due to its inability to capture the dynamic coupling effect between the asset loss rate and the response rate. In response to the above bottlenecks, most existing studies focus on the optimization of single objectives and fail to effectively solve the problem of constructing a coordination mechanism among collection revenue, cost control and customer relationship maintenance, resulting in low risk control efficiency of financial institutions for multi-mode lost-contact customers and high collection costs. Summary of the Invention

[0004] The main purpose of this application is to provide a risk control method, system and medium for lost-contact customers based on dynamic reward shaping, specifically including a risk control method, system and computer-readable storage medium for lost-contact customers based on dynamic reward shaping. The aim is to improve the risk control efficiency of financial institutions for multi-mode lost-contact customers, reduce collection costs, and achieve a dynamic balance among collection revenue, cost control and customer relationship maintenance.

[0005] To achieve the above object, this application proposes a risk control method for lost-contact customers based on dynamic reward shaping, including:

[0006] Generating multiple groups of virtual lost-contact customer data, where the virtual lost-contact customer data is divided into multiple loss-of-contact modes;

[0007] For each set of virtual lost - contact customer data, extract the first state of the virtual lost - contact customer data;

[0008] Determine the first action from a preset lightweight action space according to the first state, calculate the dynamic reward value of executing the first action in the first state and the weight mutation of the multi - objective weights according to the preset multi - objective weights;

[0009] If it is detected that the weight mutation is less than or equal to a preset change threshold, train a preset Q - value table according to the dynamic reward value until the Q - value converges to obtain the target Q - value table;

[0010] Input the actual lost - contact customer data into the target Q - value table to obtain the Q - values corresponding to each first action respectively, and determine the first action with the largest Q - value as the target risk control strategy.

[0011] In one embodiment, the virtual lost - contact customer data at least includes first - mode data, second - mode data, and third - mode data. Each mode of data respectively includes a customer response rate, an asset loss rate, and a social feature. The steps of generating multiple sets of virtual lost - contact customer data include:

[0012] Construct the first - mode data, where the change function of the customer response rate of the first - mode data is a periodic fluctuation function, the threshold of the asset loss rate is in a first preset range, and the social feature is that the proportion of temporary addresses is higher than the temporary - address proportion threshold;

[0013] Construct the second - mode data, where the change function of the customer response rate of the second - mode data is a linear decay function, the threshold of the asset loss rate is in a second preset range, the social feature is that the proportion of invalid contacts is higher than the invalid - contact proportion threshold, and the second preset range is greater than the first preset range;

[0014] Construct the third - mode data, where the change function of the customer response rate of the third - mode data is a step function, the threshold of the asset loss rate is in a third preset range, the social feature is that the number cancellation rate is higher than the number - cancellation rate threshold, the third preset range is greater than the first preset range, and the third preset range is less than the second preset range.

[0015] In one embodiment, the lost - contact modes include the first mode, the second mode, and the third mode. The first state includes the lost - contact mode. The steps of determining the first action from a preset lightweight action space according to the first state include:

[0016] Determine the initial exploration rate according to the lost - contact mode of the first state;

[0017] For the first mode and the second mode, use an exponential decay function to adjust the initial exploration rate;

[0018] For the third mode, use an inverse - proportion decay function to adjust the initial exploration rate;

[0019] Randomly select a lightweight action from the preset lightweight action space with the adjusted initial exploration rate as the probability;

[0020] If the selection is successful, determine the selected lightweight action as the first action;

[0021] If the selection fails, determine the action with the largest Q-value in the first state in the preset Q-value table as the first action.

[0022] In one embodiment, before the step of determining the first action from the preset lightweight action space according to the first state, it includes:

[0023] Map the continuous action values in the preset initial action space to discrete action values to obtain the first action space;

[0024] For the high-frequency actions in the first action space, if the customer response rate in the first state is less than the preset response threshold, reduce the action frequency of the high-frequency actions to obtain the second action space;

[0025] Determine the second action space as the lightweight action space.

[0026] In one embodiment, the multi-objective weights include the collection revenue weight, the cost constraint weight, and the relationship maintenance weight. The first state also includes the asset loss rate. The step of calculating the dynamic reward value for executing the first action in the first state according to the preset multi-objective weights further includes:

[0027] Adjust the parameters of the preset exponential function according to the comparison result between the asset loss rate and the first preset high-risk threshold;

[0028] Calculate the collection revenue weight according to the adjusted exponential function;

[0029] Calculate the cost constraint weight according to the ratio of the disposal time in the first state to the disposal cycle threshold, where the longer the disposal time, the higher the cost constraint weight;

[0030] Determine the collection revenue value for executing the first action in the first state;

[0031] Calculate the relationship maintenance weight according to the collection revenue value, where the larger the collection revenue value, the higher the relationship maintenance weight;

[0032] Determine the cost constraint value and the relationship maintenance value for executing the first action in the first state;

[0033] Perform weighted calculation on the collection revenue value, the cost constraint value, and the relationship maintenance value according to each weight to obtain the dynamic reward value.

[0034] In one embodiment, the first state further includes the customer response rate. After the steps of determining the cost constraint value and the relationship maintenance value for performing the first action in the first state, the following steps are included:

[0035] If it is detected that the asset loss rate exceeds the second preset high-risk threshold, increase the collection revenue weight and reduce the cost constraint weight;

[0036] If the customer response rate is higher than the second preset response threshold, increase the relationship maintenance weight;

[0037] If the disposal time is greater than half of the preset disposal time threshold, linearly decay the cost constraint weight;

[0038] Perform weighted calculations on the collection revenue value, the cost constraint value, and the relationship maintenance value according to each updated weight to obtain a dynamic reward value.

[0039] In one embodiment, if it is detected that the weight mutation is less than or equal to the preset change threshold, the steps of training the preset Q-value table according to the dynamic reward value until the Q-value converges to obtain the target Q-value table include:

[0040] Calculate the weight mutations of all multi-objective weights at adjacent time steps;

[0041] If the absolute values of all weight mutations are less than the preset change threshold, determine the second state obtained after performing the first action in the first state;

[0042] In the Q-value table, search for all actions associated with the second state and the Q-values corresponding to the actions;

[0043] Determine the maximum Q-value from all the Q-values;

[0044] Calculate the Q-value increment according to the dynamic reward value and the maximum Q-value;

[0045] Add the Q-value increment to the Q-value corresponding to the mapping pair of the first state to obtain the updated Q-value;

[0046] Use the updated Q-value to update the Q-value corresponding to the mapping pair of the first state;

[0047] If the preset termination condition is satisfied and the Q-value converges, stop adjusting the Q-value table to obtain the target Q-value table.

[0048] In addition, to achieve the above object, the present application also proposes a risk control system for lost-contact customers based on dynamic reward shaping. The risk control system for lost-contact customers based on dynamic reward shaping includes:

[0049] A data acquisition module, configured to generate multiple groups of virtual lost-contact customer data, where the virtual lost-contact customer data is divided into multiple lost-contact modes;

[0050] An extraction status module, configured to extract a first status of virtual lost customer data for each group of virtual lost customer data;

[0051] A calculation reward module, configured to determine a first action from a preset lightweight action space according to the first status, calculate a dynamic reward value for executing the first action in the first status according to a preset multi-objective weight, and a weight mutation of the multi-objective weight;

[0052] A weight mutation detection module, configured to, if it detects that the weight mutation is less than a preset change threshold, adjust a preset Q-value table according to the dynamic reward value until the Q-value converges to obtain a target Q-value table;

[0053] A determination strategy module, configured to input actual lost customer data into the target Q-value table to obtain Q-values corresponding to respective first actions, and determine the first action with the maximum Q-value as a target risk control strategy.

[0054] In addition, to achieve the above object, the present application further provides a lost customer risk control device based on dynamic reward shaping. The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. The computer program is configured to implement the steps of the lost customer risk control method based on dynamic reward shaping as described above.

[0055] In addition, to achieve the above object, the present application further provides a medium, which is a computer-readable storage medium. A computer program is stored on the medium. When the computer program is executed by a processor, the steps of the lost customer risk control method based on dynamic reward shaping as described above are implemented.

[0056] In addition, to achieve the above object, the present application further provides a product, which is a computer program product. The computer program product includes a computer program. When the computer program is executed by a processor, the steps of the lost customer risk control method based on dynamic reward shaping as described above are implemented.

[0057] One or more technical solutions proposed by the present application have at least the following technical effects:

[0058] It is difficult to obtain the real customer privacy data in this application. By generating multiple groups of virtual lost-contact customer data, the data acquisition and calculation costs are reduced. Among them, the virtual lost-contact customer data is divided into multiple lost-contact modes, significantly reducing the dependence on real customer data and improving the speed of algorithm iteration. For each group of virtual lost-contact customer data, the first state of the virtual lost-contact customer data is extracted. According to the first state, the first action is determined from the preset lightweight action space, and the dynamic reward value of executing the first action in the first state is calculated according to the preset multi-objective weights, ensuring the adoption of the most appropriate collection measures in different scenarios and effectively balancing the relationship among collection revenue, cost control, and customer relationship maintenance. If it is detected that the preset weight mutation is less than or equal to the preset change threshold, the preset Q-value table is trained according to the dynamic reward value until the Q-value converges, ensuring smooth transition and stable convergence during the policy adjustment process, reducing the policy oscillation caused by weight mutation, and improving the overall decision-making quality. The actual lost-contact customer data is input into the converged Q-value table to obtain the Q-values corresponding to each first action, and the first action with the largest Q-value is determined as the target risk control strategy, improving the risk control efficiency of financial institutions for multi-mode lost-contact customers, reducing the collection cost, and achieving the dynamic balance of collection revenue, cost control, and customer relationship maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application and, together with the specification, are used to explain the principles of the present application.

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0061] Figure 1 It is a schematic flowchart of the first embodiment of the method for controlling the risk of lost-contact customers based on dynamic reward shaping in the present application;

[0062] Figure 2 It is a comparison chart of the multi-method multi-mode MCI convergence curves of the method for controlling the risk of lost-contact customers based on dynamic reward shaping in the present application;

[0063] Figure 3 It is a comparison chart of the dynamic response delay curves of the method for controlling the risk of lost-contact customers based on dynamic reward shaping in the present application;

[0064] Figure 4 It is a three-dimensional action space distribution diagram of the method for controlling the risk of lost-contact customers based on dynamic reward shaping in the present application;

[0065] Figure 5Schematic diagram of the module structure of the out-of-contact customer risk control system based on dynamic reward shaping in the embodiments of the present application;

[0066] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the out-of-contact customer risk control method based on dynamic reward shaping in the embodiments of the present application.

[0067] The implementation, functional features, and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0068] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0069] To better understand the technical solutions of the present application, the following will be described in detail with reference to the drawings in the specification and specific implementation manners.

[0070] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device or terminal system capable of implementing the above functions. The following takes the system as an example to describe this embodiment and the following embodiments.

[0071] Based on this, this embodiment provides an out-of-contact customer risk control method based on dynamic reward shaping. Refer to Figure 1 , Figure 1 Schematic flowchart of the out-of-contact customer risk control method based on dynamic reward shaping in the present application. The out-of-contact customer risk control method based on dynamic reward shaping includes steps S10 to S50:

[0072] Step S10: Generate multiple groups of virtual out-of-contact customer data, where the virtual out-of-contact customer data is divided into multiple out-of-contact modes;

[0073] Step S20: For each group of virtual out-of-contact customer data, extract the first state of the virtual out-of-contact customer data;

[0074] Step S30: Determine the first action from the preset lightweight action space according to the first state, calculate the dynamic reward value of executing the first action in the first state, and the weight mutation of the multi-objective weights;

[0075] Step S40: If it is detected that the preset weight mutation is less than the preset change threshold, train the preset Q-value table according to the dynamic reward value until the Q-value converges to obtain the target Q-value table;

[0076] Step S50: Input the actual lost-contact customer data into the target Q-value table to obtain the Q-values corresponding to each first action, and determine the first action with the maximum Q-value as the target risk control strategy.

[0077] It should be noted that multiple groups of virtual lost-contact customer data are generated, and the virtual lost-contact customer data are classified according to three modes: hide-and-seek, absconding with funds, and fake disappearance. Each group of virtual lost-contact customer data includes key indicators such as customer response rate, asset loss rate, and social characteristics. The initial action space defines the set of all possible actions that can be taken, such as adjusting the collection frequency or initiating legal proceedings. The lightweight action space is a lightweight processing of the action set based on the real-time requirements of the financial scenario on the basis of the initial action space. The dynamic reward value is determined by calculating three sub-items, namely collection income, cost constraint, and relationship maintenance, and their weights. The weight mutation is the adjustment range of the weight per unit time. The Q-value table is a data structure that records the expected long-term rewards of state-action pairs and is used to make optimal decisions. The convergence of Q-values refers to making the values tend to be stable by continuously updating the state-action pairs in the Q-value table, indicating that the optimal strategy has been learned. When the actual lost-contact customer data is input into the converged Q-value table, the system calculates a Q-value for each possible action and selects the action with the highest Q-value as the target risk control strategy.

[0078] In this embodiment, first, a parametric virtual customer behavior simulator is used to generate multiple groups (such as 1000 groups) of virtual lost-contact customer data covering three lost-contact modes: hide-and-seek, absconding with funds, and fake disappearance. For each group of data, a first state including the response rate, asset loss rate, and social characteristics is extracted. Based on this information, a first action is selected from the preset lightweight action space. The selection rule can be random selection or selection based on a preset algorithm. After determining the first action, execute the action to change the first state of the customer to the second state.

[0079] A dynamic reward function is used to quantify the effect after executing the first action, obtaining a dynamic reward value. This reward function comprehensively considers factors such as the collection success rate, cost control, and customer relationship maintenance, and assigns weights to the above factors. These weights can be dynamically updated to ensure the comprehensiveness and flexibility of decision-making. According to the second state and the dynamic reward value, the Q-value corresponding to the first state-first action pair in the Q-value table is updated. After the update, it is judged whether the condition for terminating the current training round is reached (such as the customer repaying the debt, the debt being settled, the duration of being out of contact exceeding the threshold, or reaching the maximum number of iterations, etc.); if not, the next training round will continue, that is, the second action is selected from the lightweight action space according to the second state, and its dynamic reward value is calculated for updating the Q-value table, and so on until the termination condition is reached. If the termination condition is reached, the training of this group of data is terminated, and it is judged whether the Q-value converges (the Q-value tends to be stable after multiple rounds of training, that is, it converges). If it does not converge, the training of the next group of data will continue until the Q-value converges.

[0080] During the process of adjusting the Q-value table, it is detected whether the weight mutation is greater than the set change threshold. If it is greater, the update of the Q-value table is suspended. If it is less than or equal to the change threshold, the Q-value table can be updated. This process continues until the Q-value tends to be stable. Finally, the data of real customers who are out of contact is input into the optimized model, the Q-values of each potential action are output, and the action with the highest Q-value is selected as the final strategy. The entire process relies on the continuous iteration and optimization of the reinforcement learning algorithm to adapt to the complex financial risk environment.

[0081] In this embodiment, it is difficult to obtain real customer privacy data. By generating multiple groups of virtual customers who are out of contact, the data acquisition and calculation costs are reduced. Among them, the virtual customers who are out of contact data are divided into multiple out-of-contact modes, significantly reducing the dependence on real customer data and improving the speed of algorithm iteration; for each group of virtual customers who are out of contact data, the first state of the virtual customers who are out of contact data is extracted; according to the first state, the first action is determined from the preset lightweight action space, and the dynamic reward value of executing the first action in the first state is calculated according to the preset multi-objective weights, ensuring the most appropriate collection measures are taken in different scenarios, effectively balancing the relationship between collection revenue, cost control, and customer relationship maintenance; if it is detected that the preset weight mutation is less than or equal to the preset change threshold, the preset Q-value table is trained according to the dynamic reward value until the Q-value converges, ensuring a smooth transition and stable convergence during the strategy adjustment process, reducing the strategy oscillation caused by weight mutation, and improving the overall decision-making quality; the actual out-of-contact customer data is input into the converged Q-value table to obtain the Q-values corresponding to each first action, and the first action with the largest Q-value is determined as the target risk control strategy, improving the risk control efficiency of financial institutions for multi-mode out-of-contact customers, reducing the collection cost, and realizing the dynamic balance of collection revenue, cost control, and customer relationship maintenance.

[0082] In a feasible implementation, the steps in step S10 may include steps A10 to A30:

[0083] Step A10, construct the first mode data, where the change function of the customer response rate of the first mode data is a periodic fluctuation function, the threshold of the asset loss rate is in the first preset range, and the social feature is that the proportion of temporary addresses is higher than the temporary address proportion threshold;

[0084] Step A20, construct the second mode data, where the change function of the customer response rate of the second mode data is a linear decay function, the threshold of the asset loss rate is in the second preset range, the social feature is that the proportion of invalid contacts is higher than the invalid contact proportion threshold, and the second preset range is larger than the first preset range;

[0085] Step A30, construct the third mode data, where the change function of the customer response rate of the third mode data is a step function, the threshold of the asset loss rate is in the third preset range, the social feature is that the number cancellation rate is higher than the number cancellation rate threshold, the third preset range is larger than the first preset range, and the third preset range is smaller than the second preset range.

[0086] It should be noted that in this embodiment, the first mode data refers to the customer behavior simulation data in the hide-and-seek mode, where the customer response rate is represented by a periodic fluctuation function, the threshold of the asset loss rate is set in the first preset range of 5%-15% per month, and the social feature is that the proportion of temporary addresses is higher than the temporary address proportion threshold of 80%. The second mode data represents the absconding with funds mode, the customer response rate follows a linear decay function, the asset loss rate is in the second preset range of 25%-35% per month, and the social feature shows that the proportion of invalid contacts exceeds the invalid contact proportion threshold of 50%. The third mode data corresponds to the fake disappearance mode, its customer response rate is described by a step function, the asset loss rate falls in the third preset range of 15%-25% per month, and the social feature is characterized by a number cancellation rate greater than 90% of the number cancellation rate threshold. The data of these three modes together constitute the virtual lost contact customer dataset.

[0087] Exemplarily, referring to Table 1, Table 1 defines the key parameter rules for each mode, which are designed based on the analysis of historical cases of financial institutions and behavioral economics theory to ensure the statistical consistency between the simulation environment and the real scenario.

[0088] Table 1 Virtual Customer Behavior Parameter Rules

[0089]

[0090] Let \(R\) be the customer response rate and \(t\) be the independent variable of time. In the hide-and-seek mode, the periodic fluctuation of the customer response rate (sine function term) simulates the strategic behavior of customers delaying repayment by intermittently answering calls. The setting of its amplitude of 0.1 and baseline of 0.2 reflects the fluctuation range of the collection reach effect. The social feature that the proportion of temporary addresses is greater than 80% further corroborates the typical means of customers hiding their whereabouts in this mode. The response rate in the absconding with funds mode linearly decays over time (slope -0.02), depicting the decision-making process of customers gradually cutting off contact during the asset transfer process. Together with the asset loss rate threshold of greater than 10% per month, it can effectively identify high-risk escape behaviors. The fake disappearance mode distinguishes the short-term response window and the long-term out-of-touch state through a step function. The strong social feature with a number cancellation rate greater than 90% and the asset loss rate of 5-10% per month together constitute the discriminant basis for this mode.

[0091] The classification thresholds of the asset loss rate (5% per month, 10% per month) are determined based on the statistical distribution of the collateral value decay in historical cases, and can distinguish low-risk retained assets from high-risk transfer behaviors. The selection of social feature indicators (such as temporary addresses, invalid contacts) refers to the empirical rules in the collection practice to ensure the semantic consistency between the virtual customer behavior patterns and the real scenarios.

[0092] After constructing the virtual out-of-touch customer data, to achieve the dynamic perception and differential response to the out-of-touch behavior of loan customers, this application constructs a multi-dimensional state space, and its design follows the principles of observability, completeness, and pattern sensitivity. The first state of this embodiment is extracted based on the multi-dimensional state space. The state vector consists of three groups of key indicators, which can capture the heterogeneous characteristics and their evolution trends of customer behavior patterns in real time, as follows.

[0093]

[0094] Among them, the pattern identifier \(M\) t \(\in\{0,1,2\}\), which is determined by a preset rule (Table 1) and is used to classify customer behavior patterns in real time. 0, 1, and 2 respectively correspond to the hide-and-seek, absconding with funds, and fake disappearance modes. The classification basis includes social features (such as the proportion of temporary addresses), asset loss rate thresholds (5% per month, 10% per month), and the change trend of the response rate; the customer response rate is used to count the proportion of successful reaches within the \(t\) period, mainly to quantify the feedback intensity of customers to the collection actions. A high response rate (\(>0.5\)) indicates a potential negotiation possibility, and a low response rate (\(<0.2\)) requires triggering legal means; the formula for calculating asset loss is: where \(\Delta C\) is the change in the collateral value and \(\Delta t\) is the time interval (unit: days). When the asset loss rate is negative, it means that the asset has depreciated or been transferred. When per month, the system marks it as a high-risk event.

[0095] In this embodiment, by accurately modeling different types of disconnection modes, financial institutions can better understand and handle complex customer disconnection situations. The problem of difficult acquisition of real customer privacy data is solved by simulating data.

[0096] In a feasible implementation manner, the steps in step S30 may include steps B10 to B60:

[0097] Step B10, determining an initial exploration rate according to the disconnection mode in the first state;

[0098] Step B20, for the first mode and the second mode, adjusting the initial exploration rate by using an exponential decay function;

[0099] Step B30, for the third mode, adjusting the initial exploration rate by using an inverse proportional decay function;

[0100] Step B40, randomly selecting a lightweight action from a preset lightweight action space with the adjusted initial exploration rate as the probability;

[0101] Step B50, if the selection is successful, determining the selected lightweight action as the first action;

[0102] Step B60, if the selection fails, determining the action with the largest Q value in the first state in the preset Q value table as the first action.

[0103] It should be noted that the initial exploration rate refers to the probability of deciding to adopt a new strategy rather than using existing knowledge during the reinforcement learning process. For the first mode (hide-and-seek) and the second mode (absconding with funds), the initial exploration rate is adjusted by an exponential decay function, so that the exploration probability gradually decreases over time. For the third mode (false disappearance), an inverse proportional decay function is used for adjustment to ensure a certain exploration ability is maintained in the long term. The preset lightweight action space includes limited action options such as the intensity of legal deterrence (warning letters, lawyer letters, and litigation preparation) and the collection frequency. If the first action is not randomly determined from these actions, then turn to the Q value table and select the action with the largest Q value in a specific state as the first action.

[0104] In this embodiment, the mode to which the lost customer belongs is first determined according to the current state of the lost customer, and the corresponding initial exploration rate is set. For example, for the hide-and-seek and absconding with funds modes, an exponential decay function is used to adjust the initial exploration rate to accelerate the early convergence process. For the false disappearance mode, an inverse proportion decay function is used to adjust the exploration rate to maintain the possibility of long-term exploration. Next, according to the adjusted exploration rate, the first action is selected probabilistically from the preset lightweight action space. If the selection is successful, the selected lightweight action is determined as the first action; if the lightweight action cannot be successfully selected, the existing Q-value table is turned to, and the action with the highest Q-value in this state is selected as the final first action. This process combines the dual advantages of dynamically adjusting the exploration rate and utilizing existing knowledge, aiming to improve the decision-making efficiency and accuracy. The adjustment mechanism of the exploration rate helps the system widely try different strategies in the initial stage and gradually turn to the selection of the optimal strategy as experience accumulates.

[0105] Exemplarily, the calculation formula of the exploration rate can be expressed as follows:

[0106]

[0107] where M t ∈ {0, 1, 2}, 0, 1, and 2 correspond to the hide-and-seek, absconding with funds, and false disappearance modes respectively; the initial exploration rate of the hide-and-seek mode is set to 0.5, the initial exploration rate of the absconding with funds mode is set to 0.3, and the initial exploration rate of the false disappearance mode is set to 0.4, reflecting the priority of exploration requirements for different modes; for the decay coefficient, the hide-and-seek mode (0.015) and the absconding with funds mode (0.02) adopt exponential decay to accelerate early convergence; the false disappearance mode uses inverse proportion decay (denominator coefficient 0.01) to retain the long-term exploration ability.

[0108] In this embodiment, by dynamically adjusting the exploration rate, a balance between extensive exploration in the initial stage and precise utilization in the later stage is achieved, which helps to quickly find the optimal strategy for different types of lost customers.

[0109] In a feasible implementation manner, steps C10 to C30 may be included before the steps in step S30:

[0110] Step C10, mapping the continuous action values in the preset initial action space to discrete action values to obtain the first action space;

[0111] Step C20, for the high-frequency actions in the first action space, if the customer response rate of the first state is less than the preset response threshold, the action frequency of the high-frequency actions is reduced to obtain the second action space;

[0112] Step C30, determining the second action space as the lightweight action space.

[0113] It should be noted that the initial action space refers to the set of all possible actions set before optimizing the collection strategy. These actions include the frequency of collection, the severity of the collection language, whether to initiate legal proceedings, the strength of legal deterrence, debt restructuring plans and repayment plans, etc., and have both continuous and discrete values. Discrete action values are the result of simplifying the decision-making process by mapping continuous action values to a limited number of levels. High-frequency actions refer to collection actions with a high frequency in the initial action space. The preset response threshold is a standard used to judge the degree of customer response to collection actions. When the customer response rate in the first state is lower than this threshold, the frequency of high-frequency actions will be adjusted to avoid excessive collection that damages customer relationships. The lightweight action space is a set of actions that have undergone the above adjustments.

[0114] This embodiment first converts the continuous action values in the initial action space into discrete action values that are easy to handle. For example, the intensity of legal deterrence is mapped from a continuous interval to three levels: "warning letter", "lawyer's letter" and "litigation preparation". Then, if it is found that the customer's response rate in the first state is lower than the preset response threshold, the collection frequency is reduced, that is, the range of options for the execution frequency of the collection action is narrowed. This adjustment is to prevent the deterioration of customer relationships due to overly frequent collection activities. Ultimately, the new set of these adjusted actions is called a lightweight action space. Of course, it is also possible to reduce the collection frequency first and then discretize the continuous actions. This embodiment does not limit the order of the two.

[0115] For example, this embodiment is mathematically represented as follows:

[0116] 1) Discretize the legal deterrence intensity: Map the continuous value L∈[0,1] to three discrete actions L∈[0.3,0.6,0.9], representing "warning letter", "lawyer's letter" and "litigation preparation", respectively, to reduce the decision-making dimension;

[0117] 2) Dynamic grouping of collection frequency: When , the collection frequency F is selected from the high frequency group {5, 10, 15} times / month; when When , the collection frequency F switches to the low-frequency group {3, 6, 9} times / month to avoid deterioration of customer relationships due to invalid contact.

[0118] This embodiment simply embeds a lightweight computing module into the existing risk control system to achieve intelligent strategy upgrades, reducing computational complexity while improving strategy effectiveness and relevance. In this way, the system can better maintain customer relationships while ensuring collection effectiveness.

[0119] In a feasible implementation, the steps in step S30 may include steps D10 to D70:

[0120] Step D10: Adjust the parameters of the preset exponential function according to the comparison result between the asset loss rate and the first preset high-risk threshold;

[0121] Step D20: Calculate the collection revenue weight according to the adjusted exponential function;

[0122] Step D30: Calculate the cost constraint weight according to the ratio of the disposal time of the first state to the disposal cycle threshold, where the longer the disposal time, the higher the cost constraint weight;

[0123] Step D40: Determine the collection revenue value of performing the first action in the first state;

[0124] Step D50: Calculate the relationship maintenance weight according to the collection revenue value, where the larger the collection revenue value, the higher the relationship maintenance weight;

[0125] Step D60: Determine the cost constraint value and the relationship maintenance value of performing the first action in the first state;

[0126] Step D70: Perform weighted calculations on the collection revenue value, the cost constraint value, and the relationship maintenance value according to each weight to obtain the dynamic reward value.

[0127] It should be noted that the asset loss rate is an indicator to measure the speed of customer asset depreciation or transfer. The first preset high-risk threshold is a standard value used to evaluate whether the asset loss rate reaches the level that requires urgent handling. The collection revenue weight is calculated according to the adjusted exponential function and reflects the importance degree of the collection revenue in the current state. The cost constraint weight is calculated by taking the ratio of the disposal time to the disposal cycle threshold, reflecting that as the disposal time lengthens, cost control becomes more important. The relationship maintenance weight is calculated based on the collection revenue value, indicating that when the collection revenue is higher, the importance of maintaining the customer relationship also increases accordingly. The dynamic reward value is a comprehensive evaluation index obtained by performing weighted calculations on the collection revenue value, the cost constraint value, and the relationship maintenance value.

[0128] Exemplarily, this embodiment essentially uses a hierarchical dynamic reward function to calculate the dynamic reward value of performing the first action in the first state, and realizes the adaptive adjustment of multi-objective weights through the dynamic reward shaping mechanism. The total reward function is defined as:

[0129] R total (t) = α(t)R collect + β(t)R cost + γ(t)R relation

[0130] Where R collect 、R cost and R relationThey respectively represent the collection income value, the cost constraint value, and the relationship maintenance value; α(t), β(t), and γ(t) are the collection income weight, the cost constraint weight, and the relationship maintenance weight, and satisfy α(t) + β(t) + γ(t) = 1.

[0131] The design of each value and weight rule is as follows:

[0132] 1) Collection income item:

[0133]

[0134] Among them, C actual is the actual recovered amount, and C exoected is the expected recoverable amount based on the customer's asset valuation (such as the value of the collateral); the exponential decay term e -0.1t is used to encourage quick response, where t represents the time interval (unit: days) from the occurrence of loss of contact to taking action. The decay coefficient k = 0.0098 / day is determined by fitting historical data (for the recovered amount and response time of 1000 historical loss of contact cases, the exponential fitting formula: C actual = C expected ·e -kt , in e -kt , the unit of t is days, and the goodness of fit R 2 = 0.87), and the corresponding daily recovery amount decay rate of this value is approximately 0.98%, and the difference from the collateral market depreciation rate (9.5% / month, approximately 0.318% per day) is balanced by the adaptive mechanism of the dynamic weight α(t).

[0135] 2) Cost constraint item:

[0136] R cost = -(0.5F + 0.3L + 0.2I legal )

[0137] F is the collection frequency (times / month), L ∈ [0, 1] is the legal deterrence intensity (0 means only sending a lawyer's letter, 1 means enforcement), and I legal ∈ {0, 1} is the legal litigation start flag; the coefficients 0.5, 0.3, and 0.2 respectively represent the labor cost of a single collection, the marginal cost of legal deterrence, and the fixed cost of litigation, and their values are determined by the regression analysis of historical business data.

[0138] 3) Relationship maintenance item:

[0139]

[0140] is the customer response rate (percentage of successful contact times) within time period t, reflecting the customer's willingness to communicate; W∈[0,1] is the severity of the collection speech (0 is a gentle reminder, 1 is a high-pressure deterrent), and its original discrete value is normalized by minimum-maximum processing; the quadratic term W 2 This approach is used to mitigate the damage to long-term customer relationships caused by excessive debt collection. A response rate gain factor of 0.7 and a severity penalty factor of 0.3 were optimized using a grid search method to maximize the multi-objective coordination index (MCI).

[0141] 4) Collection income weight:

[0142]

[0143] is the asset loss rate (unit: % / month), defined as Where ΔC is the change in the value of the collateral, and Δt is the time interval (days). The asset loss rate needs to be normalized to the minimum and maximum values to eliminate the impact of dimensional differences on the exponential decay function.

[0144] 5) Cost constraint weight:

[0145]

[0146] T max The maximum disposal cycle (e.g., 90 days) is used, and the weight decays linearly with the disposal time, reflecting the trade-off between early cost control and later resource allocation.

[0147] 6) Relationship maintenance weight:

[0148]

[0149] is the normalized collection income (R collect Divided by the historical maximum recovery amount) to ensure that the focus is on customer relationship maintenance in high recovery scenarios.

[0150] This embodiment dynamically adjusts weights to achieve scenario-sensitive optimization of multiple objective weights and an effective balance between collection benefits, cost control, and customer relationship maintenance. This helps financial institutions more flexibly respond to different types of lost customers, improves collection efficiency, and reduces the risk of customer churn due to excessive collection, providing an efficient and low-cost risk management solution.

[0151] In a feasible implementation, the steps in step C60 may include steps E10 to E40:

[0152] Step E10: If it is detected that the asset loss rate exceeds the second preset high-risk threshold, the collection income weight is increased and the cost constraint weight is reduced;

[0153] Step E20: If the customer response rate is higher than the second preset response threshold, increase the relationship maintenance weight.

[0154] Step E30: If the handling time is greater than half of the preset handling time threshold, linearly decay the cost constraint weight.

[0155] Step E40: Perform weighted calculations on the collection revenue value, cost constraint value, and relationship maintenance value according to each updated weight to obtain a dynamic reward value.

[0156] It should be noted that the second preset high-risk threshold is another key indicator used to evaluate whether the asset loss rate has reached the emergency handling standard, and it can be the same as the first preset high-risk threshold value. The collection revenue weight is adjusted according to the comparison result between the asset loss rate and this threshold to prioritize responding to the stop-loss demand. The relationship maintenance weight is increased based on the comparison between the customer response rate and the second preset response threshold, reflecting the importance of customer communication willingness. The handling time refers to the time length from the occurrence of loss of contact to taking action; if it exceeds half of the preset handling time threshold, the cost constraint weight is gradually reduced, emphasizing more attention to cost control in the case of long-term unresolved situations. The dynamic reward value is a comprehensive evaluation indicator obtained by performing weighted calculations on the updated collection revenue value, cost constraint value, and relationship maintenance value.

[0157] In this embodiment, first, it is detected whether the asset loss rate exceeds the second preset high-risk threshold. If it does, the collection revenue weight is increased and the cost constraint weight is reduced to quickly respond to potential high-risk situations. Then, it is evaluated whether the customer response rate is higher than the second preset response threshold. If so, the relationship maintenance weight is increased, indicating that the importance of maintaining customer relationships has increased in the current state. Subsequently, it is checked whether the handling time is greater than half of the preset handling time threshold. If so, the cost constraint weight is linearly decayed to ensure that as the handling time extends, the system pays more attention to the effective control of costs. Finally, weighted calculations are performed on the collection revenue value, cost constraint value, and relationship maintenance value using each updated weight to obtain the dynamic reward value. This process optimizes the balance among multiple objectives by adjusting the weights in real time and improves the ability of the strategy to cope with complex market environments.

[0158] Exemplarily, through experiments, this embodiment can adopt the following specific parameters to achieve dynamic balance: If you want to prioritize stop-loss, when the asset loss rate > 10%, the collection revenue weight doubles and the cost constraint weight is reduced by 50%; if you want to maintain the relationship, when the customer response rate > 0.5, the relationship maintenance weight can be increased by 0.2 to inhibit high-pressure collection actions; if you want to control costs, in the later stage of handling (the handling time is greater than half of the preset handling time threshold), the cost constraint weight linearly decays to release resources for core objectives.

[0159] In this embodiment, by monitoring key indicators such as the asset loss rate, customer response rate, and disposal time in real time and dynamically adjusting the weights, an effective balance among collection revenue, cost control, and customer relationship maintenance is achieved.

[0160] In a feasible implementation manner, the steps in step S40 may include steps F10 to F90:

[0161] Step F10, calculate the weight mutation of all multi-objective weights at adjacent time steps;

[0162] Step F20, if the absolute values of all weight mutations are less than a preset change threshold, determine the second state obtained after performing the first action in the first state;

[0163] Step F30, in the Q-value table, find all actions associated with the second state and the Q-values corresponding to the actions;

[0164] Step F40, determine the maximum Q-value from all Q-values;

[0165] Step F50, calculate the Q-value increment according to the dynamic reward value and the maximum Q-value;

[0166] Step F60, add the Q-value increment to the Q-value corresponding to the mapping pair of the first state to obtain the updated Q-value;

[0167] Step F70, use the updated Q-value to update the Q-value corresponding to the mapping pair of the first state;

[0168] Step F80, if the preset termination condition is satisfied and the Q-values converge, stop adjusting the Q-value table to obtain the target Q-value table.

[0169] Step F90, among all weight mutations, if the absolute value of any weight mutation exceeds the preset change threshold, suspend the adjustment of the Q-value table and resume the adjustment after the system state stabilizes.

[0170] It should be noted that the multi-objective weights include the collection revenue weight, the cost constraint weight, and the relationship maintenance weight. The weight mutation measures the degree of change of these weights between adjacent time steps, and evaluates the speed and direction of strategy adjustment by calculating the change rate of the weights. The preset change threshold is a standard value used to determine whether the weight adjustment is small enough to ensure the stability of the system. The second state is the subsequent state after performing a specific action. The Q-value table records the expected return of performing a specific action in a specific state. The Q-value increment is the update amount calculated according to the dynamic reward value and the maximum Q-value, and is used to update the mapping pair in the Q-value table.

[0171] In this embodiment, first, the weight mutations of all multi-objective weights at adjacent time steps are calculated. This process involves continuously monitoring the weights of collection revenue, cost constraint, and relationship maintenance, and calculating their rates of change. If the absolute values of all weight mutations are less than the preset change threshold, it indicates that the current policy adjustment is tending to be stable. At this time, the Q value corresponding to the first state-first action pair in the Q value table is updated according to the second state and the dynamic reward value, and the second state obtained after executing the first action in the first state is determined; in the Q value table, all actions associated with the second state and the Q values corresponding to the actions are searched, and the maximum Q value is determined from all the Q values; the Q value increment is calculated according to the maximum Q value; the Q value increment is added to the Q value corresponding to the mapping pair of the first state to obtain the updated Q value; the Q value corresponding to the mapping pair of the first state is updated using the updated Q value;

[0172] After the update, it is judged whether the condition for terminating the current training round is reached (such as the customer repaying the loan, the debt being settled, the loss of contact duration exceeding the threshold, or reaching the maximum number of iterations, etc.); if not, the next training round will continue, that is, the second action is selected from the lightweight action space according to the second state, and its dynamic reward value is calculated for updating the Q value table; if the termination condition is reached, it is judged whether the Q value converges (the Q value tends to be stable after multiple rounds of training, which means convergence). However, if the absolute value of any weight mutation among all weight mutations exceeds the preset change threshold, the adjustment of the Q value table is suspended, and the adjustment is resumed after the system state stabilizes.

[0173] Exemplarily, the calculation formula for weight mutation can be as follows:

[0174]

[0175] where, is the weight mutation, Δt is the time step change, ω(t + 1) and ω(t) are the multi-objective weights (collection revenue weight, cost constraint weight, or relationship maintenance weight) at adjacent time steps. If the preset change threshold is 0.1, when a weight mutation is detected, the Q value update is suspended and the system state is waited to stabilize.

[0176] This embodiment further updates the hierarchical Q function, that is, a dynamic weight perception mechanism is embedded in the Q-learning (a reinforcement learning algorithm) framework, and the update rule of the state-action value function is defined:

[0177]

[0178] where, ω i (t) is the dynamic weight coefficient (α(t), β(t), γ(t)), corresponding to the collection revenue weight, cost constraint weight, and relationship maintenance weight respectively; R i Base reward term (Rcollect , R cost , R relation ), corresponding to the collection income value, cost constraint value, and relationship maintenance value respectively; the learning rate is η = 0.22 in the DRS (Dynamic Reward Shaping) method of this embodiment, and η = 0.18 for other methods, which controls the update step of the Q value. Through experimental verification, it can balance the convergence speed and stability; the discount factor is δ = 0.92 in the DRS method, and δ = 0.85 for other methods, which is used to measure the present value conversion rate of future rewards and emphasizes the long-term strategy benefits.

[0179] In this embodiment, by monitoring the weight change situation in real time, the smoothness and stability of the policy adjustment are ensured, and the policy oscillation caused by the sudden change of the weight is prevented, thereby improving the overall decision-making quality.

[0180] Exemplarily, this application verifies the effectiveness of the dynamic reward shaping mechanism in the loan loss of contact scenario through multiple groups of control experiments and visual analysis. The experiment focuses on evaluating the multi-objective coordination, policy stability, and real-time response ability, and compares with the traditional method.

[0181] 1. Experimental settings

[0182] First, a virtual customer generator is established, and 10,000 groups of lost contact customer data are generated based on the rules in Section 2.1, including the hide-and-seek mode (40%), the absconding with funds mode (30%), and the false disappearance mode (30%).

[0183] Then, set the asset loss rate thresholds, low risk (<5% / month), medium risk (5 - 10% / month), high risk (>10% / month).

[0184] Next, to verify the effectiveness of DRS, three types of comparison methods are introduced:

[0185] (1) Fixed weight method: Adopt a fixed weight combination (α = 0.55, β = 0.35, γ = 0.1 representing the common static configuration in the industry), which reflects the static strategy in traditional multi-objective optimization;

[0186] (2) Curriculum learning method: Adjust the weight distribution in stages (0.45, 0.45, 0.10) → (0.65, 0.25, 0.10) → (0.75, 0.20, 0.05), and construct a progressive optimization path based on the curriculum learning theory;

[0187] (3) DRS method: The dynamic reward shaping proposed in this paper realizes the adaptive adjustment of weight parameters through real-time risk perception (asset loss rate) and social feature feedback, and its dynamic weight adaptive rule.

[0188] Next, set the evaluation metrics:

[0189] 1) Multi-objective coordination index

[0190] To accurately quantify the collaborative decision-making efficiency of the dynamic reward shaping mechanism in multi-objective conflict scenarios, the multi-objective coordination index MCI (custom variable) constructed in this application adopts the following design principles:

[0191] First-principles modeling: directly retain the original dimensions of the collection revenue R collect , cost constraint R cost and relationship maintenance R relation , and adjust the magnitude difference between objectives in real time through dynamic weights α(t), β(t), γ(t) to avoid numerical distortion caused by extreme value interference in traditional normalization methods;

[0192] Risk-sensitive mechanism: apply the absolute value operation |R cost | to the cost item to ensure that the increase in cost will necessarily lead to an exponential decay, which conforms to the loss aversion principle in behavioral economics;

[0193] Strategy stability guarantee: the denominator α(t) + β(t) + γ(t) realizes weight self-normalization, suppressing the strategy oscillation caused by the mutation of a single objective weight. The specific formula of MCI is:

[0194]

[0195] The larger the MCI value, the better the comprehensive performance.

[0196] 2) Strategy volatility: The standard deviation of MCI in 10 independent experiments, which measures the algorithm stability.

[0197] 3) Response timeliness: The number of delay steps from detecting the asset loss rate to initiating legal actions.

[0198] 2. Experimental results and discussions

[0199] (1) Comparison of multi-objective coordination

[0200] Table 2 shows the average values and standard deviations under three types of modes after 10,000 rounds of training. The means of DRS in the hide-and-seek, absconding with funds, and fake disappearance modes reach 0.949, 0.781, and 0.568 respectively, which are 49%, 38%, and 60.9% higher than those of FW (Fixed Weighting), and 36%, 26%, and 41% higher than those of CL (Curriculum Learning). As shown in Table 2:

[0201] Table 2 Comparison of multi-objective coordination index (mean and standard deviation)

[0202]

[0203] Key findings: For the mode of absconding with funds: DRS preferentially stops losses through the weight doubling mechanism α(t)←2α(t), and the MCI increases most significantly (about 38% higher than the FW average); in terms of strategy volatility, the standard deviation of DRS in the hide-and-seek mode is reduced by about 55% compared to FW (FW standard deviation 0.031 → DRS standard deviation 0.014), verifying the inhibitory effect of dynamic weights on random interference.

[0204] (2) Analysis of strategy convergence

[0205] Figure 2 It shows that DRS exhibits significant advantages in all three modes after 4000 rounds of training: its curve rises rapidly and quickly stabilizes in the early stage of training. For example, in the mode of absconding with funds, DRS reaches the stable value in only about 1500 rounds (the curve shows a smooth plateau), while CL and FW require more than 3000 rounds and 4000 rounds respectively; in the hide-and-seek mode, the MCI curve of DRS always maintains the highest level and has the smallest fluctuations (the curve slope is steep and then flattens), verifying the precise adaptation of its dynamic weight mechanism to high-social-feature scenarios; in the fake disappearance mode, DRS can still maintain a relatively high MCI during the sharp decline stage (the curve stabilizes above 0.55 after 2000 rounds), reflecting the effectiveness of its decay factor strategy. In contrast, the curves of CL and FW generally have problems such as convergence delay and large fluctuations, especially in high-risk scenarios such as absconding with funds, where the gap is particularly obvious.

[0206] (3) Verification of dynamic response delay ability

[0207] Figure 3 The dynamic response delays of the DRS method and traditional methods are compared. The DRS method (blue curve) shows significant low-delay advantages during the training process. The overall delay level is reduced by about 65% compared to traditional methods (gray area), especially in the middle and late stages of training (4000 - 10000 rounds), and it stably maintains at about 5 days, while the baseline method has a delay as high as 12 - 15 days. This advantage stems from the dynamic reward shaping mechanism: by real-time perceiving the asset loss rate (triggering weight doubling when >30%) and customer response rate, DRS can dynamically adjust the weights of the objective function, quickly initiate legal actions after detecting high-risk events (with an average delay of only 5 days), respond 7 - 10 days earlier than traditional fixed strategies, and effectively avoid the loss of the value of collateral. Its standard deviation of fluctuations (1.8 days) is reduced by 50% compared to the baseline method (3.6 - 4.1 days), verifying the stability of the strategy.

[0208] (4) Decision space compression characteristics

[0209] Figure 4It shows the decision space compression characteristics of the DRS method in three dimensions: legal intensity - collection frequency - relationship maintenance. The DRS method discretizes legal intensity (L ∈ {0.3, 0.6, 0.9}) and dynamically groups frequencies (high - frequency group: 4 - 12 times / month, low - frequency group: 2 - 8 times / month), concentrating decision points in the efficient region (legal intensity of 0.6, frequency of 8 times, social maintenance of 0.7+). The three - dimensional distribution shows that 86% of the decision points are in the high - social - maintenance (warm - color area), indicating that DRS can balance collection intensity and customer relationship. Its decision entropy (1.28 bits) is reduced by 41% compared with traditional methods, eliminating ineffective intermediate states. At the same time, the compression of the action space reduces the single - step calculation time to 16.2 ms (35% less than the baseline of 25 ms), supporting financial institutions to complete real - time risk assessment and decision - making within 20 ms, meeting the timeliness requirements of high - concurrency scenarios.

[0210] In summary, DRS achieves scenario - sensitive optimization through dynamic weight adjustment (such as doubling in the mode of absconding with funds), its mode adaptability and stability are significantly better than traditional methods, and the lightweight design (35% reduction in calculation time) provides a low - cost upgrade path for financial institutions.

[0211] It should be noted that the above examples are only for understanding this application and do not constitute a limitation to the risk control method for lost - contact customers based on dynamic reward shaping of this application. More simple transformations in various forms based on this technical concept are within the protection scope of this application.

[0212] This application also provides a risk control system for lost - contact customers based on dynamic reward shaping. Please refer to Figure 5 , the risk control system for lost - contact customers based on dynamic reward shaping includes:

[0213] A data acquisition module 10, used to generate multiple groups of virtual lost - contact customer data, where the virtual lost - contact customer data is divided into multiple lost - contact modes; a status extraction module 20, used to extract the first status of the virtual lost - contact customer data for each group of virtual lost - contact customer data; a reward calculation module 30, used to determine a first action from a preset lightweight action space according to the first status, calculate the dynamic reward value of executing the first action in the first status according to a preset multi - objective weight, and the weight mutation of the multi - objective weight; a weight mutation detection module 40, used to adjust the preset Q - value table according to the dynamic reward value until the Q - value converges to obtain a target Q - value table if it is detected that the weight mutation is less than a preset change threshold; a policy determination module 50, used to input actual lost - contact customer data into the target Q - value table, obtain the Q - values corresponding to each first action, and determine the first action with the maximum Q - value as the target risk control policy.

[0214] The lost customer risk control system based on dynamic reward shaping provided by this application adopts the lost customer risk control method based on dynamic reward shaping in the above-mentioned embodiment, which can improve the risk control efficiency of financial institutions for multi-mode lost customers, reduce collection costs, and achieve a dynamic balance between collection income, cost control and customer relationship maintenance. Compared with the existing technology, the beneficial effects of the lost customer risk control system based on dynamic reward shaping provided by this application are the same as the beneficial effects of the lost customer risk control method based on dynamic reward shaping provided by the above-mentioned embodiment, and the other technical features of the lost customer risk control system based on dynamic reward shaping are the same as the features disclosed in the above-mentioned embodiment method, and will not be repeated here.

[0215] The present application provides a lost customer risk control device based on dynamic reward shaping. The lost customer risk control device based on dynamic reward shaping includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the lost customer risk control method based on dynamic reward shaping in the above-mentioned embodiment one.

[0216] Reference below Figure 6 , which shows a schematic structural diagram of a lost customer risk control device based on dynamic reward shaping suitable for implementing the embodiment of the present application. The lost customer risk control device based on dynamic reward shaping in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The lost customer risk control device based on dynamic reward shaping shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of this application.

[0217] like Figure 6As shown, the risk control device for lost customers based on dynamic reward shaping may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory 1002 or the program loaded from the storage device 1003 into the random access memory 1004. In the random access memory 1004, various programs and data required for the operation of the risk control device for lost customers based on dynamic reward shaping are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. The input / output interface 1006 is also connected to the bus. Generally, the following systems can be connected to the input / output interface 1006. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the risk control device for lost customers based on dynamic reward shaping to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a risk control device for lost customers based on dynamic reward shaping with various systems, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems can be alternatively implemented or had.

[0218] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device, or installed from the storage device 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing device 1001, the above functions defined in the method of the embodiments disclosed in the present application are executed.

[0219] The risk control device for lost customers based on dynamic reward shaping provided by this application adopts the method for risk control of lost customers based on dynamic reward shaping in the above-mentioned embodiment, which can improve the risk control efficiency of financial institutions for multi-mode lost customers, reduce the collection cost, and achieve the dynamic balance of collection revenue, cost control and customer relationship maintenance. Compared with the prior art, the beneficial effects of the risk control device for lost customers based on dynamic reward shaping provided by this application are the same as those of the method for risk control of lost customers based on dynamic reward shaping provided by the above-mentioned embodiment, and other technical features in the risk control device for lost customers based on dynamic reward shaping are the same as those disclosed in the method of the previous embodiment, and will not be elaborated here.

[0220] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0221] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0222] This application provides a medium, which is a computer-readable storage medium with computer-readable program instructions (i.e., computer programs) stored thereon. The computer-readable program instructions are used to execute the method for risk control of lost customers based on dynamic reward shaping in the above-mentioned embodiment.

[0223] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0224] The above computer-readable storage medium can be included in the risk control device for lost customers based on dynamic reward shaping; it can also exist alone without being assembled into the risk control device for lost customers based on dynamic reward shaping.

[0225] The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by the risk control device for lost customers based on dynamic reward shaping, the risk control device for lost customers based on dynamic reward shaping is caused to:

[0226] Generate multiple sets of virtual lost customer data, where the virtual lost customer data is divided into multiple lost modes; for each set of virtual lost customer data, extract the first state of the virtual lost customer data; determine the first action from a preset lightweight action space according to the first state, calculate the dynamic reward value for executing the first action in the first state and the weight mutation of the multi-objective weights according to the preset multi-objective weights; if it is detected that the weight mutation is less than or equal to the preset change threshold, then train the preset Q-value table according to the dynamic reward value until the Q-value converges to obtain the target Q-value table; input the actual lost customer data into the target Q-value table to obtain the Q-values corresponding to each first action, and determine the first action with the largest Q-value as the target risk control strategy.

[0227] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0228] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions labeled in the blocks may occur in a different order than that labeled in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0229] The modules described in the embodiments of this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0230] The readable storage medium provided by this application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned risk control method for lost-contact customers based on dynamic reward shaping, which can improve the risk control efficiency of financial institutions for multi-mode lost-contact customers, reduce collection costs, and achieve a dynamic balance among collection revenue, cost control, and customer relationship maintenance. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by this application are the same as those of the risk control method for lost-contact customers based on dynamic reward shaping provided by the above embodiments, and will not be elaborated here.

[0231] This application also provides a product, which is a computer program product, including a computer program. When the computer program is executed by a processor, it realizes the steps of the risk control method for lost-contact customers based on dynamic reward shaping as described above.

[0232] The computer program product provided by this application can improve the risk control efficiency of financial institutions for multi-mode lost-contact customers, reduce collection costs, and achieve a dynamic balance among collection revenue, cost control, and customer relationship maintenance. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as those of the risk control method for lost-contact customers based on dynamic reward shaping provided by the above embodiments, and will not be elaborated here.

[0233] The above are only partial embodiments of this application, and thus do not limit the patent scope of this application. Any equivalent structural transformation made under the technical concept of this application by using the content of the specification and drawings of this application, or any direct / indirect application in other related technical fields, is included in the patent protection scope of this application.

Claims

1. A risk control method for lost customers based on dynamic reward shaping, characterized in that, The risk control method for lost customers based on dynamic reward shaping includes: Generating multiple groups of virtual lost customer data, where the virtual lost customer data is divided into multiple lost contact modes; For each group of virtual lost customer data, extracting the first state of the virtual lost customer data; Determining a first action from a preset lightweight action space according to the first state, calculating the dynamic reward value of executing the first action in the first state according to a preset multi-objective weight, and the weight mutation of the multi-objective weight; If it is detected that the weight mutation is less than or equal to a preset change threshold, training a preset Q-value table according to the dynamic reward value until the Q-value converges to obtain a target Q-value table; Inputting the actual lost customer data into the target Q-value table to obtain the Q-values corresponding to each of the first actions, and determining the first action with the largest Q-value as the target risk control strategy.

2. The risk control method for lost customers based on dynamic reward shaping according to claim 1, wherein The virtual lost customer data at least includes first mode data, second mode data, and third mode data. Each mode data respectively includes a customer response rate, an asset loss rate, and a social feature. The step of generating multiple groups of virtual lost customer data includes: Constructing first mode data, where the change function of the customer response rate of the first mode data is a periodic fluctuation function, the threshold of the asset loss rate is a first preset range, and the social feature is that the proportion of temporary addresses is higher than the temporary address proportion threshold; Constructing second mode data, where the change function of the customer response rate of the second mode data is a linear decay function, the threshold of the asset loss rate is a second preset range, and the social feature is that the proportion of invalid contacts is higher than the invalid contact proportion threshold, and the second preset range is greater than the first preset range; Constructing third mode data, where the change function of the customer response rate of the third mode data is a step function, the threshold of the asset loss rate is a third preset range, and the social feature is that the number cancellation rate is higher than the number cancellation rate threshold, the third preset range is greater than the first preset range, and the third preset range is less than the second preset range.

3. The risk control method for lost customers based on dynamic reward shaping according to claim 2, wherein, The lost contact modes include a first mode, a second mode, and a third mode. The first state includes the lost contact mode. The step of determining a first action from a preset lightweight action space according to the first state includes: Determining an initial exploration rate according to the lost contact mode of the first state; For the first mode and the second mode, adjusting the initial exploration rate using an exponential decay function; For the third mode, adjusting the initial exploration rate using an inverse proportion decay function; Randomly selecting a lightweight action from a preset lightweight action space with the adjusted initial exploration rate as the probability; If the selection is successful, determining the selected lightweight action as the first action; If the selection fails, determining the action with the largest Q-value in the first state in the preset Q-value table as the first action.

4. The risk control method for lost customers based on dynamic reward shaping according to claim 1, wherein Before the step of determining a first action from a preset lightweight action space according to the first state: Mapping the continuous action values in a preset initial action space to discrete action values to obtain a first action space; For high-frequency actions in the first action space, if the customer response rate in the first state is less than the preset response threshold, reduce the action frequency of the high-frequency actions to obtain a second action space; Determine the second action space as the lightweight action space.

5. The risk control method for lost customers based on dynamic reward shaping according to claim 1, characterized in that, The multi-objective weights include a collection income weight, a cost constraint weight, and a relationship maintenance weight. The first state further includes an asset loss rate. The step of calculating the dynamic reward value for executing the first action in the first state according to the preset multi-objective weights further includes: Adjust the parameters of the preset exponential function according to the comparison result between the asset loss rate and the first preset high-risk threshold; Calculate the collection income weight according to the adjusted exponential function; Calculate the cost constraint weight according to the ratio of the handling time in the first state to the handling cycle threshold, where the longer the handling time, the higher the cost constraint weight; Determine the collection income value for executing the first action in the first state; Calculate the relationship maintenance weight according to the collection income value, where the greater the collection income value, the higher the relationship maintenance weight; Determine the cost constraint value and the relationship maintenance value for executing the first action in the first state; Perform weighted calculation on the collection income value, the cost constraint value, and the relationship maintenance value according to each of the weights to obtain the dynamic reward value.

6. The risk control method for lost customers based on dynamic reward shaping according to claim 5, characterized in that, The first state further includes a customer response rate. After the step of determining the cost constraint value and the relationship maintenance value for executing the first action in the first state, it includes: If it is detected that the asset loss rate exceeds the second preset high-risk threshold, increase the collection income weight and reduce the cost constraint weight; If the customer response rate is higher than the second preset response threshold, increase the relationship maintenance weight; If the handling time is greater than half of the preset handling time threshold, linearly decay the cost constraint weight; Perform weighted calculation on the collection income value, the cost constraint value, and the relationship maintenance value according to each updated weight to obtain the dynamic reward value.

7. The risk control method for lost customers based on dynamic reward shaping according to claim 1, wherein The step of, if it is detected that the weight mutation is less than or equal to the preset change threshold, training the preset Q-value table according to the dynamic reward value until the Q-value converges to obtain the target Q-value table includes: Calculate the weight mutations of all the multi-objective weights at adjacent time steps; If the absolute values of all weight mutations are less than the preset change threshold, determine the second state obtained after executing the first action in the first state; In the Q-value table, search for all actions associated with the second state and the Q-values corresponding to the actions; Determine the maximum Q-value from all the Q-values; Calculate the Q-value increment according to the dynamic reward value and the maximum Q-value; Add the Q-value increment to the Q-value corresponding to the mapping pair of the first state to obtain the updated Q-value; Use the updated Q-value to update the Q-value corresponding to the mapping pair of the first state; If the preset termination condition is satisfied and the Q-value converges, stop adjusting the Q-value table to obtain the target Q-value table.

8. The risk control method for lost customers based on dynamic reward shaping according to claim 7, wherein After the step of calculating the weight mutations of all the multi-objective weights at adjacent time steps, it includes: Among all the weight mutations, if the absolute value of any weight mutation exceeds a preset change threshold, the adjustment of the Q-value table is paused and resumed after the system state stabilizes.

9. A risk control system for lost customers based on dynamic reward shaping, characterized in that, The risk control system for lost customers based on dynamic reward shaping includes: A data acquisition module for generating multiple sets of virtual lost customer data, where the virtual lost customer data is divided into multiple lost contact modes; A status extraction module for extracting the first status of the virtual lost customer data for each set of virtual lost customer data; A reward calculation module for determining a first action from a preset lightweight action space according to the first status, calculating a dynamic reward value for executing the first action in the first status, and a weight mutation of the multi-objective weights; A weight mutation detection module for, if it detects that the weight mutation is less than a preset change threshold, adjusting a preset Q-value table according to the dynamic reward value until the Q-value converges to obtain a target Q-value table; A policy determination module for inputting actual lost customer data into the target Q-value table to obtain Q-values corresponding to the first actions respectively, and determining the first action with the maximum Q-value as the target risk control policy.

10. A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for risk control of lost customers based on dynamic reward shaping according to any one of claims 1 to 8 are implemented.

Images