Banking outlet site selection method and equipment based on multi-agent reinforcement learning model

Abstract

The embodiment of the invention provides a bank outlet site selection method and device based on a multi-agent reinforcement learning model. The method comprises the steps that the positions of a plurality of bank outlets in a target area at the current moment are obtained to serve as state data; inputting the state data into a pre-trained multi-agent reinforcement learning model so as to perform reasoning on the state data, and determining action data of a plurality of bank outlets; and determining site selection information of the plurality of bank outlets according to the action data. According to the method, the bank outlet site selection efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a bank branch site selection method and device based on a multi-agent reinforcement learning model. Background Art

[0002] Bank branches are the windows for banks to provide external services, and their location directly affects the bank's profitability and development.

[0003] In the prior art, when selecting bank outlets in a region, staff conducts on-site surveys to obtain relevant data, processes and analyzes the obtained relevant data, and determines the locations of multiple bank outlets in the region through manual decision-making.

[0004] Manual site selection for bank branches requires a long cycle and high time cost, resulting in low site selection efficiency. Summary of the invention

[0005] The present application provides a bank branch site selection method and device based on a multi-agent reinforcement learning model, thereby solving the technical problem of low efficiency in bank branch site selection.

[0006] In a first aspect, the present application provides a bank branch location selection method based on a multi-agent reinforcement learning model, comprising:

[0007] Obtain the current locations of multiple bank branches in the target area as status data;

[0008] Inputting the state data into a pre-trained multi-agent reinforcement learning model to infer the state data and determine the action data of the plurality of bank branches;

[0009] The location information of the plurality of bank branches is determined according to the action data.

[0010] In a second aspect, the present application provides a bank branch location selection device based on a multi-agent reinforcement learning model, comprising:

[0011] An acquisition module, used to acquire the current positions of multiple bank outlets in a target area as status data;

[0012] A first determination module is used to input the state data into a pre-trained multi-agent reinforcement learning model to infer the state data and determine the action data of the multiple bank outlets;

[0013] The second determining module is used to determine the location information of the plurality of bank outlets according to the action data.

[0014] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0015] The memory stores computer-executable instructions;

[0016] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementation methods of the first aspect.

[0018] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0019] The bank branch site selection method and device based on the multi-agent reinforcement learning model provided by the present application obtains the current positions of multiple bank branches in the target area as state data, inputs the state data into the pre-trained multi-agent reinforcement learning model to reason about the state data, determines the action data of multiple bank branches, and determines the site selection information of multiple bank branches based on the action data. The application of the multi-agent reinforcement learning model automates the site selection of bank branches, reduces the process of manual decision-making, and thus improves the efficiency of the site selection of bank branches. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] Figure 1 A flowchart of the bank branch location selection method based on a multi-agent reinforcement learning model provided in this application;

[0022] Figure 2 A flowchart of the training process of the multi-agent reinforcement learning model provided in this application;

[0023] Figure 3 A schematic diagram of the locations of bank outlets provided for this application on a straight road segment and the locations to which they can be moved;

[0024] Figure 4 A schematic diagram of the locations of bank outlets provided in the T-shaped intersection and the collection of locations to which they can be moved provided by the present application;

[0025] Figure 5A schematic diagram of the locations of bank branches located at crossroads and the locations to which they can be moved provided for this application;

[0026] Figure 6 A schematic diagram of the structure of a bank branch location selection device based on a multi-agent reinforcement learning model provided in this application;

[0027] Figure 7 A schematic diagram of the structure of the electronic device provided in this application.

[0028] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0029] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] In addition, this application involves conducting big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and using artificial intelligence technology to make automated decisions, and making technical solutions that have a significant impact on personal rights and interests based on the results of automated decisions. The application provides users with corresponding operation entrances for them to choose to agree or reject the results of automated decisions; if the user chooses to reject, the expert decision-making process will be entered.

[0032] It should be noted that the bank branch site selection method and equipment based on the multi-agent reinforcement learning model provided in this application can be used in the field of artificial intelligence, and can also be used in any field other than artificial intelligence. The application field of the bank branch site selection method and equipment based on the multi-agent reinforcement learning model in this application is not limited.

[0033] The specific application scenario of this application is to select the location of bank outlets in a target area to reasonably allocate the location of the outlets. There are multiple bank outlets to be located in the target area, and the location information of the multiple bank outlets in the target area is finally determined. The location information includes the location information of the outlets and the coverage of the outlets.

[0034] To select bank branch locations in a target area using existing technologies, staff must first conduct on-site research in the target area to obtain relevant data; then process and analyze the acquired relevant data, and use manual experience to determine whether the location of each bank branch is accurate; finally, determine the location information of multiple bank branches in the target area.

[0035] With the development of society, more and more factors need to be considered when determining the location of bank branches, and the probability of misjudgment in manual analysis is increasing. As the time cost and labor cost of manual bank branch location selection are getting higher and higher, the efficiency of bank branch location selection is low and the location selection of branches is inaccurate.

[0036] The bank branch location selection method based on a multi-agent reinforcement learning model provided in this application aims to solve the above technical problems of the prior art.

[0037] The executor of the bank branch location selection method based on the multi-agent reinforcement learning model provided in this application can be a server with data processing capabilities, or any other device with data processing capabilities, and this application does not limit this.

[0038] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0039] Figure 1 A flow chart of the bank branch location selection method based on a multi-agent reinforcement learning model provided for this application is shown in FIG. Figure 1 As shown, the method includes:

[0040] S101. Acquire the current positions of multiple bank outlets in a target area as status data.

[0041] S102: Input the state data into a pre-trained multi-agent reinforcement learning model to infer the state data and determine the action data of multiple bank branches.

[0042] S103. Determine the location information of multiple bank branches based on the action data.

[0043] It should be noted that the target area may be any area in which a plurality of bank outlets are arranged. The number of bank outlets to be selected may be any integer greater than 1.

[0044] In some embodiments, the current locations of multiple bank branches may be locations of bank branches obtained by determining site selection information in a round before the current round when determining site selection information of bank branches in multiple rounds.

[0045] The above multi-agent reinforcement learning model can be represented by Figure 2 In the multi-agent reinforcement learning model, each bank branch can be regarded as an agent.

[0046] The action data output by the multi-agent reinforcement learning model may be determined according to the value of the objective function. The objective function may be related to the coverage area, the location and the preset influence function.

[0047] The above-mentioned action data may include movement data of moving the bank branch, such as leftward movement or rightward movement.

[0048] The site selection information of any bank branch includes the location of the bank branch and the coverage area of ​​the bank branch.

[0049] The bank branch location selection method based on the multi-agent reinforcement learning model provided in the embodiment of the present application obtains the current positions of multiple bank branches in the target area as state data, inputs the state data into the pre-trained multi-agent reinforcement learning model to reason about the state data, determines the action data of multiple bank branches, and determines the location information of multiple bank branches based on the action data. The application of the multi-agent reinforcement learning model enables the automation of bank branch location selection, thereby improving the efficiency of bank branch location selection.

[0050] In a specific implementation, the action data obtained in step S102 includes: target locations to which multiple bank outlets move. The target locations to which multiple bank outlets move indicate that the multiple bank outlets can achieve the most optimal location selection by moving to the corresponding target locations in the current state.

[0051] Step S103 includes the following sub-steps:

[0052] S1031. For each bank branch among the plurality of bank branches, move the bank branch from a current position to a corresponding target position, and determine a target coverage area of ​​the bank branch according to the target position.

[0053] S1032. Determine site selection information based on the target locations and target coverage areas of the plurality of bank branches.

[0054] It should be noted that, since the site selection information of a bank branch includes location information and coverage, after the location information of a bank branch is adjusted, the coverage corresponding to the branch needs to be updated in a timely manner.

[0055] In this embodiment, by moving multiple bank branches to corresponding target locations and timely updating the coverage areas of bank branches based on the updated bank branch locations, the optimal site selection information for bank branch location is obtained. The optimal site selection information makes the location of bank branches closer to areas with higher influence, so that bank branches set up according to the above site selection information can better serve users.

[0056] The following is a detailed description of the multi-agent reinforcement learning model used in the bank branch location selection method based on the multi-agent reinforcement learning model. The multi-agent reinforcement learning model uses a deep Q network, and the deep Q network uses two feedforward neural networks as target networks for model training. Figure 2 A flowchart of the training process of the multi-agent reinforcement learning model provided in this application is as follows: Figure 2 As shown, the training process includes:

[0057] S201. Construct a multi-agent reinforcement learning model with multiple bank branches in the target area as multiple agents.

[0058] It should be noted that since multiple bank outlets to be sited share the target area, the business conditions of multiple bank outlets affect each other. Since the location of any one of the bank outlets will affect the location of other bank outlets in the target area, the location selection problem of multiple bank outlets in the target area is a multi-agent reinforcement learning problem, and a multi-agent reinforcement learning model can be constructed with multiple bank outlets in the target area as multiple agents.

[0059] S202. Determine at least one influencing factor for bank branch site selection, and determine an influence function based on the at least one influencing factor.

[0060] In this step, in order to determine the training objectives of the constructed multi-agent reinforcement learning model, it is necessary to analyze the factors affecting the location of bank outlets, and determine at least one influencing factor that most affects the location of bank outlets for use in the analysis of the training objectives of the model, so that the calculation of the model is more accurate. In order to facilitate the construction of the model, the influence of at least one determined influencing factor on the location of bank outlets is calculated, and the influence of at least one determined influencing factor on the location of bank outlets is formulated to obtain an influence function based on at least one influencing factor. Specifically, the at least one determined influencing factor can be constructed as a multivariate linear regression model, and the obtained influence function can be expressed by the following formula (1):

[0061] (1);

[0062] in, Indicates the influence of at least one influencing factor on the location selection of bank branches; , , , …, Represent different influencing factors respectively; , , …, They respectively represent the impact factors corresponding to different impact factors. Different values ​​can be set according to the specific impact of each impact factor, and this application does not limit this; Indicates the number of influencing factors.

[0063] S203: Determine the objective function based on the coverage area, location and influence function of the bank outlets.

[0064] In the model, the determined objective function can be expressed as follows:

[0065] (2);

[0066] in, Indicates the location information of multiple bank branches in the target area. represents the objective function value; Indicates the number of bank branches in the target area; Indicates The coverage area of ​​each bank branch; Indicates The location of each bank branch, represents the position of any point in the target area. Indicates The distance from each bank branch to the arbitrary point; Indicates the impact of any point on the bank branch. It can be expressed by the following formula (3):

[0067] (3);

[0068] The training goal of the multi-agent reinforcement learning model is to minimize the value of the objective function so that more bank branches are distributed in areas with high influence.

[0069] S204. Input the state data and action data corresponding to multiple bank branches at multiple historical moments into the multi-agent reinforcement learning model, perform model training with the goal of minimizing the value of the objective function, and obtain a trained multi-agent reinforcement learning model.

[0070] It should be noted that the model training process includes multiple rounds of training. After each round of training, the locations of multiple bank outlets in the target area may change. The model records the status data and action data of multiple bank outlets in the experience pool after each round of training, forming status data and action data corresponding to multiple historical moments for subsequent iterative training of the model.

[0071] In this embodiment, a multi-agent reinforcement learning model is constructed with multiple bank branches as multiple agents, and based on the coverage area, location and influence function of the bank branches, an objective function is determined to set the goal of model training. The model trained by using the state data and action data corresponding to multiple historical moments of multiple bank branches is used for bank branch site selection, which eliminates the need for a large amount of manual field research, thereby improving the efficiency of bank branch site selection.

[0072] The parameters used in the objective function in step S203 are explained below. For the influence function in the objective function, the at least one influencing factor used, in some implementations of this embodiment, the at least one influencing factor may include at least one of the following: passenger flow information, traffic information, per capita income information, bank customer ratio information. The following describes the method of obtaining these influencing factor data. The following method is only the preferred method proposed by this application. Other methods of obtaining influencing factor data are also applicable to the technical solution of this application, and this application does not limit this.

[0073] For the pedestrian flow information, the mobile signaling data of mobile users in the target area can be obtained and analyzed to obtain the number of mobile users in the target area, which can be used as a reference value for the pedestrian flow information data.

[0074] Traffic information may include the location and number of bus stations, the location and number of subway stations, and road traffic congestion data in the target area. Data related to bus stations and subway stations can be obtained through the interface provided by the map application. Road traffic congestion data can be obtained by counting the traffic volume on the road and the speed data of vehicles on the road.

[0075] For per capita income information, the housing price data of each community in the target area can be obtained through the interface provided by the housing transaction application, and the residents' income reflected by the obtained housing price information can be used as the per capita income information.

[0076] The bank customer ratio information can be obtained through bank customer registration information and the number of mobile banking users in the target area.

[0077] In these implementations, the above influencing factors are several factors that have a greater impact on the location selection of bank branches. The values ​​of these influencing factors are used to calculate the influence function, and the influence function is used to calculate the objective function, so that the objective function can more accurately reflect the training objectives of the model.

[0078] For the coverage area of ​​bank outlets in the objective function, the division rule is: for the coverage area of ​​any bank outlet, the distance from any first position in the area to the bank outlet is less than or equal to the distance to other bank outlets. Based on the above division rule, the coverage area of ​​the ith bank outlet can be expressed by the following formula (4):

[0079] (4);

[0080] in, Indicates the target area; and The locations of two different bank branches in the target area; represents the position of any point in the target area. Indicates The distance from a bank branch to any point, Indicates The distance from a bank branch to any point.

[0081] In these implementations, other points in the target area closer to the branch location are divided into the coverage area of ​​the branch, so that the coverage areas of multiple bank branches are more clearly divided.

[0082] In some implementations of this embodiment, the multi-agent reinforcement learning model constructed in step S201 includes a state space, an action space, and a reward function. The state space includes the locations of multiple bank outlets; the action space includes the current location of each bank outlet during the site selection process and the set of locations to which it can move; the reward function is used to determine the reward after executing the corresponding action in the action space at any bank outlet, and the reward is used to guide the parameter tuning of the multi-agent reinforcement learning model, and the reward function is negatively correlated with the objective function.

[0083] In the model, the state space can be expressed as follows:

[0084] (5);

[0085] in, express The state space at time, Indicates Bank branches in The location at the moment, Indicates a period of time At different moments, multiple bank outlets correspond to one state data at each different moment. It should be noted that during the model training process, the setting moment is used to characterize the decision-making process of the model when selecting the locations of multiple bank outlets in the target area. The locations of bank outlets determined by the model at different moments are different.

[0086] For the action space, within the target area, bank branches are generally set up on both sides of the road. Specifically, the location settings of bank branches can include the following three situations: setting on a straight road section, setting at a T-shaped intersection, and setting at a cross intersection. In the model, all positions within the target area can be discretized. For example, the positions within the target area are defined in a gridding manner, and bank branches can be set at any grid point. It should be noted that the above position setting method is only a preferred position setting method provided by this application, and other methods of setting the position are also applicable to this application.

[0087] Based on the above three bank branch location settings, the current location of each bank branch and the set of locations to which it can be moved during the site selection process include:

[0088] If the bank branch is located on a straight road section, the bank branch moves leftward to a left position and rightward to a right position according to a first preset movement rule.

[0089] If the bank branch is located at a T-shaped intersection, the bank branch is moved to the left to a left position, to the right to a right position, and downward to a lower position according to a second preset movement rule.

[0090] If the bank branch is located at a crossroad, the bank branch is moved to the left to a left position, to the right to a right position, downward to a lower position, and upward to an upper position according to a third preset movement rule.

[0091] In the model, multiple bank branches are The action space at the moment can be expressed by the following formula (6):

[0092] (6);

[0093] in, express The action space at the moment, Indicates Bank branches in The action space at each moment, for each different moment, multiple bank branches correspond to one action data.

[0094] In order to intuitively represent the location of bank branches and the set of locations to which they can be moved during the site selection process, different numbers are used in the model to record the location of bank branches and the set of locations to which they can be moved.

[0095] Figure 3 A schematic diagram of the locations of bank outlets provided for this application on a straight road segment and the locations to which they can be moved, such as Figure 3 As shown, 1 represents the current location of the bank branch. The bank branch can move to the left to the left position 2 and to the right to the position 3 according to the first preset movement rule. The location of the branch and the set of positions to which it can move are represented in the model as {1, 2, 3}. It should be noted that the distance set by the first preset movement rule to move from position 1 to position 2 or to position 3 can be a fixed distance or a variable distance, and this application does not limit this.

[0096] Figure 4 A schematic diagram of the locations of bank outlets provided for this application at a T-shaped intersection and a collection of locations to which they can be moved, such as Figure 4 As shown, the bank branch is at the current position 1, and can move to the left to the left position 2, move to the right to the position 3, and move downward to the position 4 according to the second preset movement rule. The position of the branch and the set of positions to which it can move can be represented as {1, 2, 3, 4} in the model. The second preset rule is similar to the rule set by the first preset rule, and will not be repeated here.

[0097] Figure 5 A schematic diagram of the locations of bank outlets located at crossroads and the locations to which they can be moved provided for this application, such as Figure 5 As shown, the bank branch is at the current position 1, and can move to the left to the left position 2, to the right to the position 3, to the downward position 4, and to the upward position 5 according to the third preset movement rule. The position of the branch and the set of positions to which it can move can be represented as {1, 2, 3, 4, 5} in the model. The third preset rule is similar to the rule set by the first preset rule, and will not be repeated here.

[0098] As for the reward function, since the training goal of the model is to minimize the value of the objective function, if the bank branch performs an action in the action space and this action makes the objective function of the model gradually decrease, then the model should give the bank branch a reward greater than zero, indicating that the action performed by the bank branch has a positive effect on the objective function. In the model, the reward function can be expressed as follows:

[0099] (7);

[0100] in, express The reward function value at the moment; Indicates the number of bank branches in the target area; Indicates The coverage area of ​​each bank branch; Indicates The location of each bank branch, represents the position of any point in the target area. Indicates The distance from each bank branch to the arbitrary point; Indicates the impact of any point on the bank branch.

[0101] In these embodiments, through the above definitions, the state space, action space and reward function of the multi-agent reinforcement learning model for bank branch location selection are formulated to facilitate model training and model reasoning on a server with data processing capabilities.

[0102] In order to obtain the optimal parameters for the model, in some implementations of this embodiment, in step S204, the state data and action data corresponding to multiple bank branches at multiple historical moments are respectively input into the multi-agent reinforcement learning model, and the model is trained with the goal of minimizing the value of the objective function to obtain the trained multi-agent reinforcement learning model, including the following sub-steps:

[0103] First, input into the multi-agent reinforcement learning model The status and action data of multiple bank branches at all times are collected by the multi-agent reinforcement learning model based on Calculation of status data and action data at all times +1 Status data, action data, and reward value at the moment of time.

[0104] It should be noted that in The status data of multiple bank branches at this moment is The state data corresponding to the state space of multiple bank branches observed at all times, and the action data are the action data corresponding to the position movement action recently performed by the multiple bank branches.

[0105] Second, the multi-agent reinforcement learning model is based on +1 time state data, action data, reward value, output +1 moment's worth.

[0106] In the model, in order to distinguish historical state data, The state data at time +1 can be expressed by the following formula (8):

[0107] (8);

[0108] in, express +1 Status data of multiple bank branches at the moment, express +1 moment Status data of each bank branch.

[0109] The action data of multiple bank branches at time +1 can be expressed by the following formula (9):

[0110] (9);

[0111] in, express +1 Action data of multiple bank branches at the moment, express +1 moment The movement data of each bank branch.

[0112] The value of the joint action of multiple bank branches in the target area at time +1 can be expressed by the following formula (10):

[0113] (10);

[0114] in, express +1 The value of the action jointly executed by multiple bank branches at time, which is calculated using the action value function Q function of Q-Learning fitted by the deep neural network; Indicates The value of actions performed by each bank branch; are the parameters of the target network of the deep Q network.

[0115] Third, based on The value of the +1 moment is used to tune the parameters to minimize the value of the objective function.

[0116] The purpose of model training is to minimize the value of the objective function. Since the reward function is negatively correlated with the objective function, The reward function needs to be maximized to ensure consistency across multiple bank branches. The reward function is maximized using the following formula (11):

[0117] (11);

[0118] in, It means that the value of multiple bank branches jointly executing actions is maximized. It means that the value of the action performed by the nth bank branch is maximized.

[0119] In order to maintain the monotonicity of the value function, formula (11) also needs to satisfy the following constraints of formula (12):

[0120] (12);

[0121] Fourth, repeat the model input steps, value output, and parameter tuning steps until the preset conditions are met to obtain the trained multi-agent reinforcement learning model.

[0122] It should be noted that the model calculates After the status data and action data at the +1 moment are collected, the coverage areas of multiple bank branches need to be updated synchronously.

[0123] In this step, the preset condition may be that when the change curve of the loss function used by the model gradually becomes flat during multiple iterations of training, the model training is stopped. It may also be other preset conditions, which are not limited in this application.

[0124] During model training, the loss function can also be used to evaluate the difference between the model's predicted value and the true value to guide the model to tune parameters. The loss function for training a model using a deep Q network can be expressed as follows:

[0125] (13);

[0126] in, express +1 moment loss value, Indicates the number of samples sampled from the experience pool for each training. Indicates The reward function value of sampled data, represents the discount factor of the reward function, Indicates The maximum Q value of the sampled data.

[0127] Figure 6 The schematic diagram of the structure of the bank branch location selection device based on the multi-agent reinforcement learning model provided in this application is as follows: Figure 6 As shown, the bank branch location selection device 60 based on the multi-agent reinforcement learning model provided by the present application includes:

[0128] The acquisition module 601 is used to acquire the current positions of multiple bank outlets in the target area as status data.

[0129] The first determination module 602 is used to input the state data into a pre-trained multi-agent reinforcement learning model to infer the state data and determine the action data of multiple bank branches.

[0130] The second determination module 603 is used to determine the location information of multiple bank branches according to the action data.

[0131] In a specific embodiment, the action data determined by the first determination module 602 includes: the target positions to which multiple bank branches each move; the second determination module 603 is also used to move each bank branch among the multiple bank branches from the current position to the corresponding target position, and determine the target coverage area of ​​the bank branch according to the target position; determine the site selection information based on the target positions of the multiple bank branches and their respective target coverage areas.

[0132] In some implementations of the present embodiment, the device also includes a model building module, which is used to build a multi-agent reinforcement learning model with multiple bank branches in the target area as multiple agents; determine at least one influencing factor for the site selection of bank branches, and determine the influence function based on the at least one influencing factor; determine the objective function based on the coverage area, location and influence function of the bank branch; input the state data and action data corresponding to multiple bank branches at multiple historical moments into the multi-agent reinforcement learning model, and perform model training with the goal of minimizing the value of the objective function to obtain a trained multi-agent reinforcement learning model.

[0133] In some implementations of the present embodiment, the multi-agent reinforcement learning model constructed by the model construction module includes a state space, an action space, and a reward function; the state space includes the locations of multiple bank branches; the action space includes the current position of each bank branch during the site selection process and the set of positions to which it can be moved; the reward function is used to determine the reward after performing a corresponding action in the action space at any bank branch, and the reward is used to guide parameter tuning of the multi-agent reinforcement learning model, and the reward function is negatively correlated with the objective function.

[0134] In some implementations of this embodiment, the model building module is also used to input The status and action data of multiple bank branches at all times are collected by the multi-agent reinforcement learning model based on Calculation of status data and action data at all times +1 moment state data, action data, reward value; based on the multi-agent reinforcement learning model +1 time state data, action data, reward value, output +1 moment value; based on +1 moment to tune the parameters to minimize the value of the objective function; repeat the model input steps, value output and parameter tuning steps until the preset conditions are met to obtain the trained multi-agent reinforcement learning model.

[0135] In some implementations of this embodiment, the set of the current location and the locations to which each bank branch can move in the site selection process in the model building module includes:

[0136] If the bank branch is located on a straight road section, the bank branch moves leftward to a left position and rightward to a right position according to a first preset movement rule.

[0137] If the bank branch is located at a T-shaped intersection, the bank branch is moved to the left to a left position, to the right to a right position, and downward to a lower position according to a second preset movement rule.

[0138] If the bank branch is located at a crossroad, the bank branch is moved to the left to a left position, to the right to a right position, downward to a lower position, and upward to an upper position according to a third preset movement rule.

[0139] In some implementations of this embodiment, the model building module is used to determine at least one influencing factor of bank branch site selection, including at least one of the following factors: passenger flow information, traffic information, per capita income information, and bank customer ratio information.

[0140] In some implementations of this embodiment, the division rule of the coverage area of ​​bank outlets in the model building module is: for the coverage area of ​​any bank outlet, the distance from any first position in the area to the bank outlet location is less than or equal to the distance to other bank outlet locations.

[0141] In some implementations of this embodiment, the model building module uses a deep Q network for model training.

[0142] In some implementations of this embodiment, the deep Q network used by the model building module uses a loss function to evaluate the difference between the predicted value and the true value of the model to guide the model to perform parameter tuning.

[0143] Figure 7A schematic diagram of the structure of the electronic device provided in this application, such as Figure 7 As shown, the device 70 includes a processor 701 and a memory 702 communicatively connected to the processor; the memory 702 stores computer-executable instructions; the processor 701 executes the computer-executable instructions stored in the memory 702 to perform the above method.

[0144] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0145] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0146] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0147] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0148] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0149] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0150] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0151] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0152] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0153] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0154] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0155] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Unless otherwise specified, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. Unless otherwise specified, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory (RRAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), an enhanced dynamic random access memory (EDRAM), a high-bandwidth memory (HBM), a hybrid memory cube (HMC), etc.

[0156] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0157] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0158] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0159] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A bank branch location selection method based on a multi-agent reinforcement learning model, the method comprising: Obtain the current locations of multiple bank branches in the target area as status data; Inputting the state data into a pre-trained multi-agent reinforcement learning model to infer the state data and determine the action data of the plurality of bank branches; The location information of the plurality of bank branches is determined according to the action data.

2. The method according to claim 1, characterized in that The action data includes: target locations to which the plurality of bank outlets are moved; Determining the location information of the plurality of bank outlets according to the action data includes: For each of the plurality of bank outlets, move the bank outlet from the current position to a corresponding target position, and determine a target coverage area of ​​the bank outlet according to the target position; The site selection information is determined based on the respective target locations and respective target coverage areas of the multiple bank outlets.

3. The method according to claim 2, characterized in that The multi-agent reinforcement learning model is trained by the following training process: Constructing a multi-agent reinforcement learning model with the multiple bank outlets in the target area as multiple agents; Determine at least one influencing factor for bank branch location selection, and determine an influence function based on the at least one influencing factor; Determine the objective function based on the coverage area, location and influence function of the bank branches; The state data and action data corresponding to multiple historical moments of the multiple bank branches are respectively input into the multi-agent reinforcement learning model, and the model is trained with the goal of minimizing the value of the objective function to obtain a trained multi-agent reinforcement learning model.

4. The method according to claim 3, characterized in that The multi-agent reinforcement learning model includes a state space, an action space, and a reward function; The state space includes the locations of the plurality of bank branches; The action space includes the current location of each bank branch during the site selection process and the set of locations to which it can move; The reward function is used to determine the reward after executing the corresponding action in the action space at any bank branch, and the reward is used to guide the parameter tuning of the multi-agent reinforcement learning model. The reward function is negatively correlated with the objective function.

5. The method according to claim 4, characterized in that The state data and action data corresponding to the multiple bank branches at multiple historical moments are input into the multi-agent reinforcement learning model, and the model is trained with the goal of minimizing the value of the objective function to obtain the trained multi-agent reinforcement learning model, including: Inputting the state data and action data of the plurality of bank branches at time t into the multi-agent reinforcement learning model, and the multi-agent reinforcement learning model calculates the state data, action data and reward value at time t+1 according to the state data and action data at time t; The multi-agent reinforcement learning model outputs the value at time t+1 based on the state data, action data, and reward value at time t+1; Perform parameter tuning based on the value at time t+1 to minimize the value of the objective function; Repeat the model input steps, value output, and parameter tuning steps until the preset conditions are met to obtain a trained multi-agent reinforcement learning model.

6. The method according to any one of claims 4-5, characterized in that: The current location of each bank branch and the set of locations to which it can be moved during the site selection process include: If the bank branch is located on a straight road segment, the bank branch moves leftward to a left position and rightward to a right position according to a first preset movement rule; If the bank branch is located at a T-shaped intersection, the bank branch is moved leftward to a left position, rightward to a right position, and downward to a lower position according to a second preset movement rule; If the bank branch is located at a crossroads, the bank branch is moved leftward to a left position, rightward to a right position, downward to a lower position, and upward to an upper position according to a third preset movement rule.

7. The method according to any one of claims 3 to 5, characterized in that: The at least one influencing factor includes at least one of the following factors: passenger flow information, traffic information, per capita income information, and bank customer ratio information.

8. The method according to any one of claims 3 to 5, characterized in that: The division rules of the coverage areas of the bank outlets are as follows: For the coverage area of ​​any bank branch, the distance from any first position in the area to the bank branch location is less than or equal to the distance to other bank branch locations.

9. The method according to any one of claims 3 to 5, characterized in that: The multi-agent reinforcement learning model uses a deep Q network for model training.

10. The method according to claim 9, characterized in that The deep Q network uses a loss function to evaluate the difference between the model's predicted value and the true value to guide the model to perform parameter tuning.

11. A bank branch location selection device based on a multi-agent reinforcement learning model, comprising: An acquisition module, used to acquire the current positions of multiple bank outlets in a target area as status data; A first determination module, configured to input the state data into a pre-trained multi-agent reinforcement learning model to infer the state data and determine the action data of the plurality of bank outlets; The second determining module is used to determine the location information of the plurality of bank outlets according to the action data.

12. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 10 when executed by a processor.

14. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 10 when being executed by a processor.

Images