Currency supply regulation and control strategy generation method and device, electronic equipment and storage medium

Abstract

The invention provides a currency supply regulation and control strategy generation method and device, electronic equipment and a storage medium, and belongs to the technical field of reinforcement learning, and the method comprises the steps: obtaining state data, behavior data and contribution degree, enabling the state data to represent the operation state of a banking industry financial institution in a preset geographic region in a current period of time, and enabling the behavior data to represent the contribution degree; the behavior data represents behavior actions of the banking industry financial institution under the current regulation and control strategy, and the contribution degree is the contribution degree of the behavior data to the total currency supply amount and the structure regulation and control target; and inputting the state data into a currency supply regulation and control strategy generation model to obtain currency supply regulation and control strategy content output by the currency supply regulation and control strategy generation model. According to the scheme, the currency supply regulation and control strategy is generated in real time according to the state of the financial institution of the banking industry by utilizing historical data, and a decision basis is provided for making and executing a currency policy; an offline reinforcement learning model is used, a scene of real-time interaction with the environment is avoided, potential risks and cost are remarkably reduced, and a financial scene is further fitted.

Description

Technical Field

[0001] The present invention relates to the field of reinforcement learning technology, and in particular to a method, device, electronic device and storage medium for generating a money supply regulation strategy. Background Art

[0002] It is very difficult to formulate regulatory strategies related to the money supply. It requires comprehensive consideration of various economic indicators at the social level, as well as a comprehensive understanding of how the strategy targets (mainly various financial institutions) have implemented the regulatory strategies over the past period of time. It also requires drawing experience from past long-term regulatory practices to establish expectations for adjustments.

[0003] The strategies for regulating the total amount and structure of money supply have traditionally relied on experience and quantitative analysis models in the economic and financial fields to assist decision-making. The modeling methods are also mainly based on the traditional method of regression analysis, which is difficult to meet the current dual decision-making goals of both total amount and structure.

[0004] Therefore, there is an urgent need for a tool to generate monetary supply regulation strategies to assist administrative departments in making monetary policy decisions to meet the current dual decision-making goals of both total volume and structure. Summary of the invention

[0005] The present invention provides a money supply regulation strategy generation method, device, electronic device and storage medium, which are used to solve the problem that the money supply regulation strategy generation method in the prior art cannot meet the current dual decision-making target requirements of both total amount and structure.

[0006] In a first aspect, the present invention provides a method for generating a money supply regulation strategy, comprising: Acquire status data, behavior data and contribution, wherein the status data represents the operating status of banking financial institutions in a preset geographical area within a current period of time, the behavior data represents the behavior actions of the aforementioned banking financial institutions under the current regulatory strategy, and the contribution degree is the contribution of the behavior data to the total money supply and structural regulatory targets; The state data is input into a money supply regulation strategy generation model to obtain the money supply regulation strategy content output by the money supply regulation strategy generation model, wherein the money supply regulation strategy generation model is an offline reinforcement learning model.

[0007] According to a method for generating a money supply regulation strategy provided by the present invention, the money supply regulation strategy generation model is constructed based on a conservative reinforcement learning CQL algorithm.

[0008] According to a money supply regulation strategy generation method provided by the present invention, the money supply regulation strategy generation model generates money supply regulation strategy content based on the following method: Initialize Q function and control strategy; Sampling the state data and the behavior data to obtain sampled data, wherein the distribution of the sampled data matches the distribution of the state data under a specific control strategy, wherein the specific control strategy is one of a set of control strategies for generating a data set distribution; Based on the state data and the behavior data, using the Q function to calculate a Q value; Based on the Q value and the Q value after adding the regularization term, updating the Q function using a preset learning rate, wherein the regularization term is calculated based on the contribution when taking the action action in the state state; Based on the updated Q function, the money supply regulation strategy is optimized, and the step of calculating the value based on the state data and the behavior data using the Q function is returned until the Q value converges or reaches a preset number of iterations, and the optimized result is determined to be the content of the money supply regulation strategy.

[0009] According to a method for generating a money supply regulation strategy provided by the present invention, the state data includes at least one of a risk indicator, a capital indicator, and a strategy indicator of a banking financial institution; Among them, the risk indicators represent the financial risk situation of the banking financial institutions themselves, the capital indicators represent the operation of the capital business of the banking financial institutions, and the strategy indicators represent the response degree of the banking financial institutions to the current money supply regulation strategy.

[0010] According to a method for generating a money supply regulation strategy provided by the present invention, the behavior data includes at least one of a conventional business behavior and a strategy response behavior; Among them, the routine business behavior refers to the daily business behavior of banking financial institutions, and the strategic response behavior refers to the business behavior of banking financial institutions under the guidance of the current money supply regulation strategy.

[0011] According to a method for generating a money supply regulation strategy provided by the present invention, before acquiring status data, wherein the status data represents the current operating status of a banking financial institution, the method further includes: Based on objective information of multiple banking financial institutions, clustering the behavior data of each banking financial institution to obtain a clustering result, wherein the objective information includes the type of institution, region, holding type and asset size of the banking financial institution; Based on the clustering result, the plurality of banking financial institutions are grouped to obtain a plurality of banking financial institution groups; In the step of obtaining status data, status data of each banking financial institution in the same banking financial institution group is obtained.

[0012] In a second aspect, the present invention further provides a device for generating a money supply regulation strategy, comprising: A data acquisition module, used to acquire status data, behavior data and contribution, wherein the status data represents the operating status of banking financial institutions in a preset geographical area within a current period of time, the behavior data represents the behavior actions of banking financial institutions under the current regulation strategy, and the contribution degree is the contribution of the behavior data to the total money supply and structural regulation targets; The strategy generation module is used to input the state data into the money supply regulation strategy generation model to obtain the money supply regulation strategy content output by the money supply regulation strategy generation model, and the money supply regulation strategy generation model is an offline reinforcement learning model.

[0013] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-described methods for generating a money supply regulation strategy are implemented.

[0014] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods for generating a money supply regulation strategy.

[0015] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, characterized in that when the computer program is executed by a processor, the steps of any of the above-mentioned methods for generating a money supply regulation strategy are implemented.

[0016] The money supply regulation strategy generation method, device, electronic device and storage medium provided by the present invention obtain state data, the state data represents the operating state of the bank; obtain behavior data, the behavior data is the behavior action of the banking financial institution under the monetary policy; obtain contribution, the contribution is the contribution of the behavior data to the total amount of money supply and structural regulation targets; input the state data into the money supply regulation strategy generation model, and obtain the money supply regulation strategy content output by the money supply regulation strategy generation model, the money supply regulation strategy generation model is an offline reinforcement learning model. This scheme uses the state data, behavior data and contribution within a period of history, and uses an offline reinforcement learning model to generate a money supply regulation strategy in real time for the state of the banking financial institution, providing a solution for monetary policy decision-making. The powerful computing power of the reinforcement learning model meets the current dual decision-making target requirements of both total amount and structure; the offline reinforcement learning model is used to avoid the scene of real-time interaction with the environment, significantly reduce potential risks and costs, and further fit the financial scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the method for generating a money supply regulation strategy provided by the present invention; Figure 2 is a schematic diagram of a Markov decision process in the prior art; Figure 3 It is a structural diagram of an AC frame in the prior art; Figure 4 It is a flow chart of the money supply regulation strategy generation model provided by the present invention; Figure 5 It is a structural schematic diagram of the money supply regulation strategy generation device provided by the present invention; Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] It should be noted that in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The orientation or position relationship indicated by the terms "upper", "lower" and the like is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or it can be a connection between two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0021] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0022] Combine the following Figure 1-Figure 6 The invention describes a method, device, electronic device and storage medium for generating a money supply regulation strategy provided by an embodiment of the invention.

[0023] Figure 1 It is a flow chart of the method for generating a money supply regulation strategy provided by the present invention, such as Figure 1 As shown, including but not limited to the following steps: S110, acquiring status data, behavior data and contribution, wherein the status data represents the operating status of banking financial institutions in a preset geographical area in the current period of time, the behavior data represents the behavior actions of the aforementioned banking financial institutions under the current regulation strategy, and the contribution degree is the contribution of the behavior data to the total money supply and structural regulation targets; S120, inputting the state data into a money supply regulation strategy generation model to obtain the money supply regulation strategy content output by the money supply regulation strategy generation model, wherein the money supply regulation strategy generation model is an offline reinforcement learning model.

[0024] Here, the operating status of a bank refers to its historical operating status within a preset time period, such as the operating status from February to March. Correspondingly, the behavioral actions under the current monetary policy refer to the behavioral actions under the monetary policy within the preset time period, and the contribution degree refers to the contribution degree to the strategic goal after the bank's real-time behavioral actions, thereby reflecting whether the bank's operating behavior is in line with the guidance direction of the strategic goal.

[0025] It should be noted that the execution subject of the money supply regulation strategy generation method provided in the embodiment of the present application can be a server, a computer device, such as a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, etc.

[0026] In order to make full use of existing historical data and avoid scenarios of real-time interaction with the environment, so as to further fit the actual use scenarios of the model, a branch of reinforcement learning, Offline Reinforcement Learning (Offline RL), is selected. Traditional reinforcement learning interacts with the environment, learns by collecting data and then discards it, while Offline RL does not perform any form of online interaction and exploration. Offline RL studies how to learn a near-optimal strategy from a static data set. Since it does not require any additional exploration and does not require real-time collection of user data, it can significantly reduce potential risks and costs, and is particularly suitable for the financial field. At the same time, Offline RL also has the advantages of high computational efficiency and low deployment difficulty. Here, the offline reinforcement learning model used is not limited.

[0027] The method for generating a money supply regulation strategy provided in an embodiment of the present invention utilizes historical data to generate a money supply regulation strategy in real time according to the status of a bank, thereby providing a solution for monetary policy decision-making. An offline reinforcement learning model is used to avoid scenarios of real-time interaction with the environment, significantly reducing potential risks and costs, and further adapting to financial scenarios.

[0028] In an optional embodiment, the money supply regulation strategy generation model is constructed based on a conservative reinforcement learning CQL algorithm.

[0029] In offline reinforcement learning, a round from a certain state to the terminal state is a Markov chain. A Markov chain only has states and state transition probabilities. Adding actions to the Markov chain constitutes a Markov decision process (MDP). MDP is a process of reinforcement learning that satisfies the Markov property.

[0030] The goal of offline reinforcement learning is to find the optimal strategy given an MDP. The strategy is the mapping from state to action, so that the final cumulative return expectation is maximized. MDP first assumes that the state in the real world can be described by a static probability distribution to describe the result of each action. Secondly, assuming that the state of the environment satisfies the properties of the Markov process, then the next state of the system has nothing to do with the previous state, but only with the current state information.

[0031] Figure 2 is a schematic diagram of the Markov decision process in the prior art, such as Figure 2 As shown, an MDP can be defined as a five-tuple { S , A , P sa , γ , R},in, S Represents a collection of states, i The status of the step is s i ; A represents a set of actions (actions), and the action of the 𝑖th step is a i ; P sa represents the state transition probability; γ represents the discount factor for future expectations, where γ ∈ [0,1]; R Represents the reward function, which is the state s and actions a The function of r ( s , a ).

[0032] Offline reinforcement learning generally solves the Q function by iterating the Bellman equation, and through continuous iteration, the state action value function finally converges to obtain the optimal strategy. In the Markov decision process, for this optimal strategy, the optimal value function and the optimal action value function can be found.

[0033] Entropy is a measure of the degree of randomness of a random variable. Specifically, if Xis a random variable, and its probability density function is P , then its entropy H for H ( X ) =Ex~p [ -logp(x) ], used in reinforcement learning H (π(·| s )) to represent the strategy π in state s The idea of ​​maximum entropy reinforcement learning (RL) is to make the strategy more random in addition to maximizing the cumulative reward. In this way, an entropy regularization term is added to the goal of reinforcement learning, which is defined as: ; Here α represents a regularization coefficient, which is used to control the importance of entropy. The entropy regularization term increases the degree of exploration of the reinforcement learning algorithm. The larger α is, the stronger the exploration is, which helps to accelerate subsequent strategy learning and reduce the possibility of the strategy falling into a poor local optimum.

[0034] Figure 3 It is a structural diagram of the AC frame in the prior art, such as Figure 3 As shown in the figure, the AC (Actor-Critic) framework is a reinforcement learning framework that considers both strategies and value functions. This method maintains two learners at the same time, where the Actor is the strategy function, responsible for learning strategies, controlling the specific behavior of the agent and interacting with the environment to generate actions; the Critic guides the Actor to update through the neural network fitting value function, and evaluates the quality of the Actor's actions. Compared with algorithms based on value functions, the AC algorithm draws on the policy gradient approach, enabling it to handle decision-making tasks with continuous or high-dimensional action spaces. Compared with traditional policy gradient algorithms, the AC algorithm can perform single-step updates instead of updates in units of trajectories.

[0035] The SAC (Soft Actor-Critic) deep reinforcement learning algorithm based on the maximum entropy principle and AC framework has the advantages of high stability and wide exploration space. The SAC algorithm uses a deep neural network to estimate the strategy and value function, and uses the stochastic gradient descent method to update these estimated values. It introduces a soft constraint, which is to maximize the expected return while minimizing the entropy of the strategy. This entropy regularization technique can make the strategy more exploratory, making it easier to avoid falling into the local optimal solution.

[0036] The SAC algorithm is very efficient and solves the reinforcement learning problem in discrete action spaces and continuous action spaces. The purpose of the SAC algorithm is to maximize the future cumulative reward value and entropy, making the strategy as random as possible, that is, the probability of each action output is as dispersed as possible, rather than concentrated on one action.

[0037] The objective function expression of the SAC algorithm is as follows: ; in, T represents the total number of time steps of interaction between the agent and the environment, p π Indicates that under strategy π( s t , a t ), H (·) represents the entropy value and α represents a hyperparameter whose purpose is to control the degree of randomness of the optimal policy and weigh the importance of entropy relative to reward.

[0038] Classic offline algorithms have a distribution transfer problem, that is, when the strategy sampled from the experience pool is very different from the strategy used for the current behavior, the algorithm will not work. For example, when the experience pool does not have a certain state-action pair, the offline reinforcement learning algorithm cannot learn the strategy for this action in that state. This problem is often called inference error. The sources of inference error are mainly the following: Model bias: When calculating the value function, the expected value of the reward function needs to be calculated. However, the data taken from the experience pool in each batch is limited, and the number of visits to each state-action pair is also limited, which leads to model bias. Data is missing. If some specific state-action pairs do not appear in the experience pool, the value of the state-action pair will be misestimated during strategy operation. Training mismatch: Even if there is sufficient data, errors may occur if the data distribution of the dataset is inconsistent with the distribution of the current strategy.

[0039] In order to solve the problem of extrapolation error, it is necessary to impose some restrictions on the current strategy to solve the problem of inference error caused by the difference between the data distribution in the data set and the current strategy distribution. The CQL (Conservative Q_Learning) algorithm is proposed based on this. CQL conservatively estimates the value of invisible actions outside the data set, that is, assigns a lower value to the action value function of the action, which can reduce the number of times invisible actions are executed and avoid overestimation of the value of invisible actions during state transfer.

[0040] Figure 4 It is a flow chart of the money supply regulation strategy generation model provided by the present invention, such as Figure 4 As shown in the figure, the main process of the model is: input the operating status of each bank (mainly asset-liability data, loan structure data, risk indicator data and other status data), the bank's actions under the current strategy tools (behavioral data), and the bank's contribution to the total money supply and structural regulation targets within a certain period of time (contribution). Through the SAC algorithm of offline reinforcement learning, data-driven is realized, and the recommended monetary policy tools for the bank in the current state are output, and the bank's subsequent actions are recorded to verify the model effect.

[0041] The goal of offline reinforcement learning is to obtain a better strategy through reinforcement learning algorithms from a certain set of data that has been collected without the agent interacting with the environment. This model uses CQL as the offline reinforcement learning algorithm to obtain the best results in offline reinforcement learning problems. CQL conservatively estimates the value of unseen actions outside the data set, that is, it assigns a lower value to the action value function of the action, which can reduce the number of times the unseen actions are executed and avoid overestimation of the value of unseen actions during state transfer.

[0042] Furthermore, the money supply regulation strategy generation model generates the money supply regulation strategy content based on the following method: Initialize Q function and control strategy; Sampling the state data and the behavior data to obtain sampled data, wherein the distribution of the sampled data matches the distribution of the state data under a specific control strategy, wherein the specific control strategy is one of a set of control strategies for generating a data set distribution; Based on the state data and the behavior data, using the Q function to calculate a Q value; Based on the Q value and the Q value after adding the regularization term, updating the Q function using a preset learning rate, wherein the regularization term is calculated based on the contribution when taking the action action in the state state; Based on the updated Q function, the money supply regulation strategy is optimized, and the step of calculating the value based on the state data and the behavior data using the Q function is returned until the Q value converges or reaches a preset number of iterations, and the optimized result is determined to be the content of the money supply regulation strategy.

[0043] Specifically, initialize the Q function and monetary policy; Sampling state data and behavior data, the distribution of sampled data matches the distribution of state data under the total money supply and structural regulation targets; Use the Q function to calculate the Q value of taking action action in state state; Based on the Q value and the Q value after adding the regularization term, updating the Q function using a preset learning rate, wherein the regularization term is calculated based on the contribution when taking the action action in the state state; Based on the updated Q function, the money supply regulation strategy is optimized, and the step of using the Q function to calculate the Q value of the action action taken under the state state is returned to, until the Q value converges or reaches a preset number of iterations, and the final result is used as the content of the money supply regulation strategy.

[0044] It should be understood that the introduction of the regularization term makes the Q-value estimation more conservative, thereby reducing the problems caused by data distribution mismatch in offline reinforcement learning and improving the stability and robustness of the strategy, especially when data is scarce or unevenly distributed.

[0045] Optionally, based on the input banking and financial institution data, the model automatically makes decisions and learns based on the offline reinforcement learning algorithm, and outputs targeted strategic recommendations (i.e., recommended strategy tools) for the current status of banking and financial institutions. In addition, the model simultaneously outputs the "estimated effect", which can be compared with the actual operating behavior of banking and financial institutions and the corresponding status changes, and can further verify the actual effect of the model.

[0046] In an optional embodiment, the status data includes at least one of risk indicators, capital indicators and strategy indicators of a banking financial institution; Among them, the risk indicators represent the financial risk situation of the banking financial institutions themselves, the capital indicators represent the operation of the capital business of the banking financial institutions, and the strategy indicators represent the response degree of the banking financial institutions to the current money supply regulation strategy.

[0047] Specifically, risk indicators are indicators that characterize the financial risk status of banking financial institutions themselves, and may include but are not limited to indicators such as asset-liability ratio, capital adequacy ratio (including Tier 1 and Tier 2 core capital adequacy ratios), non-performing loan ratio, provision ratio, and liquidity ratio; capital indicators are indicators that characterize the operation of capital business of banking financial institutions, and may include but are not limited to indicators such as excess reserve ratio, issuance certificate of deposit interest rate, and interbank lending rate; strategy indicators are indicators that characterize the degree of response of banking financial institutions to the current money supply regulation strategy, and may include but are not limited to the proportion of small and micro enterprise loans in total loans, the growth rate of small and micro enterprise loan balances, the growth rate of agricultural loan balances, and the average interest rate of agricultural loans.

[0048] In an optional embodiment, the behavior data includes at least one of regular business behavior and policy response behavior; Among them, the routine business behavior refers to the daily business behavior of banking financial institutions, and the strategic response behavior refers to the business behavior of banking financial institutions under the guidance of the current money supply regulation strategy.

[0049] Specifically, routine business operations refer to the results of the daily business operations of banking financial institutions reflected at the data level, which may include but are not limited to changes in the average interest rate of deposits, deposit increases, loan increases, changes in interbank borrowing and lending, etc.; strategic response operations refer to the results of the business operations of banking financial institutions in accordance with the guidance of the current money supply regulation strategy, which are reflected at the data level, which may include but are not limited to changes in loan interest rates for small and micro enterprises, changes in green loan balances, changes in loan interest rates for specialized and new enterprises, etc.

[0050] In an optional embodiment, before acquiring the status data, wherein the status data represents the current operating status of the bank, the method further includes: Based on objective information of multiple banking financial institutions, clustering the behavior data of each banking financial institution to obtain a clustering result, wherein the objective information includes the type of institution, region, holding type and asset size of the banking financial institution; Based on the clustering result, the plurality of banking financial institutions are grouped to obtain a plurality of banking financial institution groups; In the step of obtaining status data, status data of each banking financial institution in the same banking financial institution group is obtained.

[0051] Specifically, firstly, descriptive statistics are performed based on various historical data of banking financial institutions to obtain the minimum, maximum, variance and mean of the relevant data, and obtain the basic information of the sample data set; secondly, a high-dimensional dynamic clustering algorithm is used to cluster the operating behavior data of banking financial institutions in combination with the objective information of banking financial institutions (institutional type, region, holding type, asset size, etc.), and the clustering results are interpreted in order to dynamically group banking financial institutions and realize the automated active and passive strategy tool selection based on the grouping of different banking financial institutions.

[0052] Optionally, clustering may be performed based on the status data of the banking financial institution, or may be performed based on both the behavior data and the status data of the banking financial institution.

[0053] In summary, the money supply regulation strategy generation method provided by the present invention effectively uses historical data for self-learning and exploration, generates a recommended strategy tool in real time based on the status of banking financial institutions, explores which specific strategies or measures have a positive effect on improving the banking financial institutions' service to the macro-money supply total amount and structural regulation goals, and provides a solution for monetary policy decision-making departments; adopts the SAC algorithm in offline reinforcement learning, and by designing the input of state, action, and feedback, the algorithm can automatically make decisions and learn, and obtain the money supply regulation strategy content that is most suitable for the current status of banking financial institutions.

[0054] The following is a description of a money supply regulation strategy generation device provided in an embodiment of the present application. The money supply regulation strategy generation device described below and the money supply regulation strategy generation method described above can refer to each other.

[0055] Figure 5 is a schematic diagram of the structure of the money supply control strategy generation device provided by the present invention, such as Figure 5 As shown, the money supply regulation strategy generating device includes: The data acquisition module 510 is used to acquire status data, behavior data and contribution, wherein the status data represents the operating status of banking financial institutions in a preset geographical area within a current period of time, the behavior data represents the behavior actions of banking financial institutions under the current regulation strategy, and the contribution degree is the contribution of the behavior data to the total money supply and structural regulation targets; The strategy generation module 520 is used to input the state data into the money supply regulation strategy generation model to obtain the money supply regulation strategy content output by the money supply regulation strategy generation model, and the money supply regulation strategy generation model is an offline reinforcement learning model.

[0056] It should be noted that the money supply regulation strategy generation device provided in the embodiment of the present invention can execute the money supply regulation strategy generation method described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0057] Figure 6 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the money supply regulation strategy generation method, which includes: obtaining state data, behavior data and contribution, wherein the state data represents the operating state of the banking financial institution in the preset geographical area in the current period of time, the behavior data represents the behavior of the aforementioned banking financial institution under the current regulation strategy, and the contribution is the contribution of the behavior data to the total amount of money supply and structural regulation targets; inputting the state data, the behavior data and the contribution into the money supply regulation strategy generation model to obtain the money supply regulation strategy content output by the money supply regulation strategy generation model, and the money supply regulation strategy generation model is an offline reinforcement learning model.

[0058] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0059] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the money supply regulation strategy generation method provided in the above-mentioned embodiments, the method including: obtaining status data, behavior data and contribution degree, the status data representing the operating status of banking financial institutions in a preset geographical area during the current period of time, the behavior data representing the behavioral actions of the aforementioned banking financial institutions under the current regulation strategy, and the contribution degree being the contribution of the behavior data to the total amount of money supply and structural regulation targets; inputting the status data, the behavior data and the contribution degree into a money supply regulation strategy generation model to obtain the money supply regulation strategy content output by the money supply regulation strategy generation model, and the money supply regulation strategy generation model is an offline reinforcement learning model.

[0060] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the money supply regulation strategy generation method provided in the above-mentioned embodiments, the method comprising: obtaining status data, behavior data and contribution degree, the status data representing the operating status of banking financial institutions in a preset geographical area during a current period of time, the behavior data representing the behavioral actions of the aforementioned banking financial institutions under the current regulation strategy, and the contribution degree being the contribution degree of the behavior data to the total amount of money supply and structural regulation targets; inputting the status data, the behavior data and the contribution degree into a money supply regulation strategy generation model to obtain the money supply regulation strategy content output by the money supply regulation strategy generation model, the money supply regulation strategy generation model being an offline reinforcement learning model.

[0061] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0062] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a money supply regulation strategy, characterized in that: include: Acquire status data, behavior data and contribution, wherein the status data represents the operating status of banking financial institutions in a preset geographical area within a current period of time, the behavior data represents the behavior actions of the aforementioned banking financial institutions under the current regulatory strategy, and the contribution degree is the contribution of the behavior data to the total money supply and structural regulatory targets; The state data is input into a money supply regulation strategy generation model to obtain the money supply regulation strategy content output by the money supply regulation strategy generation model, wherein the money supply regulation strategy generation model is an offline reinforcement learning model.

2. The method for generating a money supply control strategy according to claim 1, characterized in that: The money supply regulation strategy generation model is constructed based on the conservative reinforcement learning CQL algorithm.

3. The method for generating a money supply control strategy according to claim 2, characterized in that: The money supply regulation strategy generation model generates the money supply regulation strategy content based on the following method: Initialize Q function and control strategy; Sampling the state data and the behavior data to obtain sampled data, wherein the distribution of the sampled data matches the distribution of the state data under a specific control strategy, wherein the specific control strategy is one of a set of control strategies for generating a data set distribution; Based on the state data and the behavior data, using the Q function to calculate a Q value; Based on the Q value and the Q value after adding the regularization term, updating the Q function using a preset learning rate, wherein the regularization term is calculated based on the contribution when taking the action action in the state state; Based on the updated Q function, the money supply regulation strategy is optimized, and the step of calculating the value based on the state data and the behavior data using the Q function is returned until the Q value converges or reaches a preset number of iterations, and the optimized result is determined to be the content of the money supply regulation strategy.

4. The method for generating a money supply control strategy according to any one of claims 1 to 3, characterized in that: The status data includes at least one of risk indicators, capital indicators and strategy indicators of banking financial institutions; Among them, the risk indicators represent the financial risk situation of the banking financial institutions themselves, the capital indicators represent the operation of the capital business of the banking financial institutions, and the strategy indicators represent the response degree of the banking financial institutions to the current money supply regulation strategy.

5. The method for generating a money supply control strategy according to any one of claims 1 to 3, characterized in that: The behavior data includes at least one of regular business behavior and strategic response behavior; Among them, the routine business behavior refers to the daily business behavior of banking financial institutions, and the strategic response behavior refers to the business behavior of banking financial institutions under the guidance of the current money supply regulation strategy.

6. The method for generating a money supply control strategy according to any one of claims 1 to 3, characterized in that: Before obtaining the status data, wherein the status data represents the current operating status of the banking financial institution, the method further includes: Based on objective information of multiple banking financial institutions, clustering the behavior data of each banking financial institution to obtain a clustering result, wherein the objective information includes the type of institution, region, holding type and asset size of the banking financial institution; Based on the clustering result, the plurality of banking financial institutions are grouped to obtain a plurality of banking financial institution groups; In the step of obtaining status data, status data of each banking financial institution in the same banking financial institution group is obtained.

7. A device for generating a money supply control strategy, characterized in that: include: A data acquisition module, used to acquire status data, behavior data and contribution, wherein the status data represents the operating status of banking financial institutions in a preset geographical area within a current period of time, the behavior data represents the behavior actions of banking financial institutions under the current regulation strategy, and the contribution degree is the contribution of the behavior data to the total money supply and structural regulation targets; The suggestion generation module is used to input the state data into the money supply regulation strategy generation model to obtain the money supply regulation strategy content output by the money supply regulation strategy generation model, and the money supply regulation strategy generation model is an offline reinforcement learning model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method for generating a money supply regulation strategy as claimed in any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for generating a money supply regulation strategy as claimed in any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for generating a money supply regulation strategy as claimed in any one of claims 1 to 6 are implemented.

Images