A method for site selection of urban public service facilities

Abstract

This invention provides a method for site selection of urban public service facilities, belonging to the field of data processing technology. Specifically, it includes: collecting facility data for a target area; modeling the urban public service facility site selection problem as a continuous maximum coverage problem with distance decay effects and capacity constraints, with the objective function being to maximize the satisfied demand; dividing the target area using a multi-scale grid and constructing a multi-scale spatial decision tree, which serves as the site selection environment for agents; having multiple agents control the site selection planning of multiple facilities, and setting the attributes of the agents; based on the objective function, having the multiple agents interact with the multi-scale spatial decision tree, using a proximal strategy optimization method to optimize the agent network parameters to obtain the optimal site selection strategy; and using the optimal site selection strategy to perform site selection planning in the multi-scale spatial decision tree to obtain a site selection scheme for urban public service facilities. This invention improves the efficiency and quality of site selection.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method for site selection of urban public service facilities. Background Technology

[0002] Currently, deep reinforcement learning methods have been introduced into facility location research in recent years. Deep reinforcement learning models the facility location problem as a Markov decision process, training the agent to learn the optimal location strategy through interaction with the environment, thus obtaining high-quality approximate solutions. Guided by given constraints and optimization objectives, it can adaptively explore optimal location locations without requiring manual design of additional search rules, possessing powerful adaptive search capabilities in the solution space. Commonly used algorithms include: Deep Q-Network algorithm, dominant actor-critic algorithm, and proximate policy optimization algorithm.

[0003] Given the difficulty of training agents through reinforcement learning, current reinforcement learning-based facility location work usually focuses on training a single agent and constructing a discrete location environment by providing candidate points for facilities or setting a fixed step size for facilities to move freely, in order to reduce the complexity of reinforcement learning training.

[0004] However, the current facility location method based on deep reinforcement learning has two shortcomings: (1) In actual location problem scenarios, geographical environment, population distribution, resource supply, demand patterns, etc. all exhibit obvious spatial heterogeneity. The single location strategy based on single agent reinforcement learning is obviously difficult to cope with complex non-homogeneous location decision scenarios, and the computing power of a single agent is obviously difficult to handle large-scale location problems involving a large number of facilities and demand points; (2) Compared with continuous location environment (where facilities can be placed at any point in the space), discrete location environment significantly compresses the feasible solution space, making it difficult to find higher quality facility location schemes.

[0005] It is evident that there is an urgent need for a site selection method for urban public service facilities that is both efficient and of high quality. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide a method for site selection of urban public service facilities, which at least partially solves the problems of poor site selection efficiency and quality in the prior art.

[0007] This invention provides a method for site selection of urban public service facilities, including:

[0008] Step 1: Collect facility data for the target area, wherein the facility data includes existing facility distribution data and facility service gap data;

[0009] Step 2: Model the problem of urban public service facility site selection as a continuous maximum coverage problem with distance decay effect and capacity constraint, with the objective function being to maximize the demand to be satisfied.

[0010] Step 3: Use a multi-scale grid to divide the target region and construct a multi-scale spatial decision tree, which will serve as the location environment for the agent.

[0011] Step 4: Allow multiple agents to control the site selection planning of multiple facilities respectively, and set the attributes of the agents, wherein the attributes include state space, action space and reward function;

[0012] Step 5: Based on the objective function, let the multi-agent interact with the multi-scale spatial decision tree, and use the near-end policy optimization method to optimize the agent network parameters to obtain the optimal addressing strategy.

[0013] Step 6: Use the optimal site selection strategy to perform site selection planning in a multi-scale spatial decision tree to obtain the site selection scheme for urban public service facilities.

[0014] According to a specific implementation of an embodiment of the present invention, step 2 specifically includes:

[0015] Step 2.1: Model the target region as a continuous space, containing elements... A set of demand points :

[0016] ;

[0017] in, Indicates the first One demand point, These are the coordinates of the demand point. It is demand;

[0018] Step 2.2, assuming there are in continuous space A collection of service facilities in a space to be planned :

[0019] ;

[0020] in, Indicates the first One service facility, These are the coordinates of the service facilities. The total amount of service that the facility can provide. The service scope of the facility;

[0021] Step 2.3: When the service area of ​​a facility covers multiple demand points, the service volume provided by the facility will be allocated to each demand point according to its weight.

[0022] ;

[0023] in, Indication facilities For demand points Service volume provided, weight , This indicates the distance between the demand point and the facility, and , It is the distance decay effect function;

[0024] Step 2.4, find The objective function is to select the location of each facility to maximize the demand that is met.

[0025] ;

[0026] in, This represents the set of spatial locations of all service facilities.

[0027] According to a specific implementation of an embodiment of the present invention, step 3 specifically includes:

[0028] Step 3.1: Divide the continuous space using grids of different scales to form a multi-scale spatial decision tree. Each tree node in the multi-scale spatial decision tree represents a spatial region and contains the attribute information of the region it represents. It is called a region node. Each time the subdivision is completed, each region node generates four child nodes. The side length of the region represented by the node is reduced by half and the area becomes one-quarter of the original.

[0029] Step 3.2: Based on this infinitely subdivisible multi-scale spatial decision tree, the continuous spatial region is infinitely subdivided to fully represent the entire continuous spatial region.

[0030] According to a specific implementation of an embodiment of the present invention, step 4 specifically includes:

[0031] Step 4.1: Each facility is controlled by an agent, and each agent is controlled by an Actor-Critic network. The agent interacts with the multi-scale spatial decision tree by outputting actions through the Actor network.

[0032] Step 4.2: Set the state space, action space, and reward function for each agent.

[0033] According to a specific implementation of an embodiment of the present invention, step 5 specifically includes:

[0034] Step 5.1: Based on the objective function, when the multi-agent interacts with the multi-scale spatial decision tree each time, the agent outputs the probability of going to the next level of four sub-region nodes based on the current level of region node information, and samples one sub-region node to move from. The reward is calculated based on the service provided by the agent to the demand point after the move.

[0035] Step 5.2: Represent the record of each interaction as [state, action, reward, next moment state]. Through multiple interactions, an interaction sequence can be formed to complete a facility site selection plan.

[0036] Step 5.3: For each agent, the Actor-Critic network parameters are updated using a proximal policy optimization method. The Actor-Critic network parameters are updated by learning the interaction sequence multiple times, and the stability of the multi-agent addressing policy learning is ensured by limiting the policy update magnitude, thereby optimizing the agent's addressing policy.

[0037] Step 5.4: Repeat steps 5.1 to 5.3 to obtain the optimal addressing strategy.

[0038] According to a specific implementation of an embodiment of the present invention, step 5.2 specifically includes:

[0039] Step 5.2.1: During each interaction, represent the state of each agent as follows:

[0040] ;

[0041] in, These are the spatial coordinates of the intelligent agent. It is the distance between the agent and the weighted center position of all demand points, calculated based on the demand. Indicates intelligent agent and The distance between each demand point express The demand quantity of each demand point in the current time step. Represents intelligent agents Compared with others The distance between agents;

[0042] Step 5.2.2: Set each agent to have four selectable actions. The selectable action is to choose one of the four sub-region nodes to deploy the facility. The action space of the agent is represented as follows:

[0043] ;

[0044] in, This represents an action that controls the agent in... and Movement in direction This represents the step size of the intelligent agent. The number of steps is halved as the agent makes sequential decisions in the multi-scale spatial decision tree.

[0045] Step 5.2.3, set the time step for each agent. The reward obtained is equal to the reward obtained from the decision at the current time step minus the reward obtained from the decision at the previous time step:

[0046] ;

[0047] Among them, the initial reward Calculations are performed based on the agent's initial position;

[0048] Step 5.2.4: During each interaction, the process of the agent moving from the current spatial location to another spatial location through action decision, as well as the changes in the amount of service provided by the demand point and the changes in the relative spatial locations between the agent and the demand point, are taken as the state of the next moment.

[0049] Step 5.2.5: Record the state, action, reward and next state after the move to form decision unit data. The decision unit sequence generated by the sequential decision-making process constitutes decision trajectory data, which is used as an interaction sequence.

[0050] According to a specific implementation of an embodiment of the present invention, step 5.3 specifically includes:

[0051] Step 5.3.1: Calculate the generalized dominance estimation function using a reverse recursive approach and time-series difference error. Evaluate the superiority or inferiority of actions based on the generalized dominance estimation function. The expression for the generalized dominance estimation function is:

[0052] ;

[0053] ;

[0054] in, Indicates timing difference error. It is a discount factor used to control the degree of emphasis on future rewards. It is a hyperparameter. Indicates an immediate reward. It is a state-value function. This represents the state value at time t+1. This represents the state value at time t;

[0055] Step 5.3.2: Construct the objective optimization function for each agent based on the generalized advantage estimation function.

[0056] ;

[0057] in, This means calculating the average value based on the agent trajectory data collected in the current batch, thereby obtaining the empirical expectation. Indicate the importance sampling ratio by using the old strategy Collect empirical data to update strategies , It is a hyperparameter used to limit the update magnitude of the policy. Indicates an action;

[0058] Step 5.3.3: Add an entropy regularization term to the objective optimization function.

[0059] ;

[0060] in, For hyperparameters, ;

[0061] Step 5.3.4, represent each agent as... ,in, Represents the policy network of agent i. Represents the value network of agent i. Indicates the policy network parameters, The parameters of the value network are represented by the orthogonal initialization method used to initialize the network weights. The learning rates for the policy network and the value network are set as follows: and And update the network parameters using the Adam optimizer;

[0062] Step 5.3.5, Set the total number of training rounds and the number of decision steps per round Set the number of training iterations for each batch of data. The network parameters are optimized through a gradient ascent strategy. Thus, the optimal addressing strategy is obtained. .

[0063] The urban public service facility site selection scheme in this embodiment of the invention includes: Step 1, collecting facility data of the target area, wherein the facility data includes existing facility distribution data and facility service gap data; Step 2, modeling the urban public service facility site selection problem as a continuous maximum coverage problem with distance decay effect and capacity constraint, the objective function being to maximize the demand to be satisfied; Step 3, dividing the target area using a multi-scale grid, constructing a multi-scale spatial decision tree, and using it as the site selection environment for the agents; Step 4, having multiple agents control the site selection planning of multiple facilities respectively, setting the attributes of the agents, wherein the attributes include state space, action space, and reward function; Step 5, based on the objective function, having multiple agents interact with the multi-scale spatial decision tree, using a proximal policy optimization method to optimize the agent network parameters, and obtaining the optimal site selection strategy; Step 6, using the optimal site selection strategy to perform site selection planning in the multi-scale spatial decision tree, obtaining the urban public service facility site selection scheme.

[0064] The beneficial effects of the embodiments of the present invention are as follows: By using a multi-scale grid to divide spatial regions, the present invention effectively solves the problem of facility location in continuous space. Based on multi-scale spatial partitioning and multi-agent near-end strategy optimization reinforcement learning, it achieves efficient and high-quality facility location planning, effectively overcoming the bottleneck of current facility location algorithms based on deep reinforcement learning that are difficult to solve the problem of facility location in continuous space. It also well couples the multi-agent system, enabling the algorithm to cope with complex non-homogeneous location decision scenarios through diverse combinations of location strategies, decomposing the huge computational load to multiple agents. It can provide a high-quality and high-performance solution for large-scale facility location planning, serving the practical application needs of urban public service facility location planning. Attached Figure Description

[0065] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 A flowchart illustrating a method for selecting locations for urban public service facilities, provided in an embodiment of the present invention;

[0067] Figure 2 This is a schematic diagram illustrating the spatial allocation of facility service volume according to an embodiment of the present invention;

[0068] Figure 3 This is a schematic diagram of the action space design of a facility intelligent agent provided in an embodiment of the present invention;

[0069] Figure 4 This is a schematic diagram illustrating the distribution of existing electric vehicle charging facilities and the charging service gap, provided as an embodiment of the present invention.

[0070] Figure 5 This is a schematic diagram illustrating the site selection results of newly added electric vehicle charging facilities and the filling of charging service gaps, provided as an embodiment of the present invention. Detailed Implementation

[0071] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0072] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0073] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this invention, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0074] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The illustrations only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0075] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0076] This invention provides a method for site selection of urban public service facilities, which can be applied to the facility deployment process in urban management scenarios.

[0077] See Figure 1 This is a flowchart illustrating a method for selecting locations for urban public service facilities provided in an embodiment of the present invention. Figure 1 As shown, the method mainly includes the following steps:

[0078] Step 1: Collect facility data for the target area, wherein the facility data includes existing facility distribution data and facility service gap data;

[0079] In practice, data on the study area can be collected, such as data on the distribution of existing facilities and data on facility service gaps, in order to facilitate subsequent operational procedures.

[0080] Step 2: Model the problem of urban public service facility site selection as a continuous maximum coverage problem with distance decay effect and capacity constraint, with the objective function being to maximize the demand to be satisfied.

[0081] In practical implementation, considering that urban space is a continuous physical space, modeling the urban facility site selection problem as a continuous site selection problem is more realistic and helps in the discovery of high-quality site selection solutions and refined facility site selection planning. In actual urban facility site selection scenarios, the main characteristics are limited service range, limited capacity, and service capacity decaying with distance, such as electric vehicle charging stations and express delivery service stations.

[0082] Therefore, based on the classic maximum coverage problem, the problem of site selection for urban public service facilities is modeled as a continuous maximum coverage problem with distance decay effect and capacity constraint.

[0083] The specific description is as follows: In continuous space, there exists a... A set of demand points :

[0084] ;

[0085] in, Indicates the first One demand point, These are the coordinates of the demand point. It refers to demand.

[0086] have A collection of service facilities in a space to be planned :

[0087] ;

[0088] in, Indicates the first One service facility, These are the coordinates of the service facilities. The total amount of service that the facility can provide. The service area of ​​the facility.

[0089] When a facility's service area covers multiple demand points, such as Figure 2 As shown, the service volume provided by this facility will be allocated to each demand point according to its weight:

[0090] ;

[0091] in, Indication facilities For demand points Service volume provided, weight , This indicates the distance between the demand point and the facility, and , It is a distance attenuation effect function, such as the Gaussian distribution function, the inverse distance weighting function, etc.

[0092] beg The location of each facility should be chosen to maximize the fulfillment of demand.

[0093] ;

[0094] in, Let be the objective function. This represents the set of spatial locations of all service facilities.

[0095] Step 3: Use a multi-scale grid to divide the target region and construct a multi-scale spatial decision tree, which will serve as the location environment for the agent.

[0096] In practice, continuous spatial regions are divided using grids of different scales, forming a multi-scale spatial decision tree. Each tree node represents a spatial region and contains attribute information about that region; it is called a region node. Each region node can be further subdivided to generate four child nodes. Each subdivision reduces the side length of the spatial region represented by the node by half and its area to one-quarter of its original size. This multi-scale spatial decision tree can be continuously subdivided to generate new child nodes. As the number of subdivisions increases... At this time, the side length and area of ​​the spatial region represented by the region node will approach 0, thus approximating a spatial point:

[0097] ;

[0098] in, Represents the coordinates of a point in space. It is equal to half the side length of the entire study area. Based on this infinitely subdivisible multi-scale spatial decision tree, the continuous spatial region can be infinitely subdivided, thus completely representing the entire continuous spatial region. This allows the agent to reach any point in the continuous spatial region by traversing the decision tree in a depth-first manner. This invention uses this multi-scale spatial decision tree as the location environment for the agent.

[0099] Step 4: Allow multiple agents to control the site selection planning of multiple facilities respectively, and set the attributes of the agents, wherein the attributes include state space, action space and reward function;

[0100] In practical implementation, each facility can be controlled by an agent, called a facility agent. Each facility agent is controlled by an Actor-Critic network, and the facility agent interacts with the environment (multi-scale spatial decision tree) by outputting actions through the Actor network (policy network). During each interaction, the facility agent outputs the probability of going to one of the four sub-region nodes in the next level based on the information of the current level's regional nodes, and samples one of these sub-region nodes to move to. The reward is calculated based on the facility agent's service performance to the demand point after the move. Through continuous interaction, the facility agent can perform a depth-first traversal of the multi-scale spatial decision tree, generating decision trajectory data of the facility agent's interaction with the environment. Each decision unit data can be represented as: [state, action, reward, next state], and a series of decision unit data constitutes the decision trajectory data.

[0101] Specifically, the state of each facility agent is represented as follows:

[0102] ;

[0103] in, These are the spatial coordinates of the intelligent agent. It is the distance between the agent and the weighted center position of all demand points, calculated based on the demand. Indicates intelligent agent and The distance between each demand point express The demand quantity of each demand point in the current time step. Represents intelligent agents Compared with others The distance between individual facility agents.

[0104] Each facility agent has four selectable actions: choose one of four candidate region nodes to deploy the facility, such as... Figure 3 As shown, the action space of the agent is represented as:

[0105] ;

[0106] in, This represents an action that controls the agent in... and Movement in direction This represents the step size of the intelligent agent. The number of steps is halved as the agent makes sequential decisions in the multi-scale spatial decision tree.

[0107] The state transition process of a facility agent (from the current state to the next state) can be described as: the process by which the agent moves from its current spatial location to another spatial location through action decisions, and this process is accompanied by changes in the amount of services provided by the demand point (i.e., changes in the demand of the demand point), as well as changes in the relative spatial locations between the facility agent and the demand point.

[0108] Each facility agent at time step The reward obtained is equal to the reward obtained from the decision at the current time step minus the reward obtained from the decision at the previous time step:

[0109] ;

[0110] Among them, the initial reward The reward calculation is based on the agent's initial position. This reward calculation method ensures that the reward received by the facility agent in a sequential decision-making sequence is always equal to the amount of service it provides, giving the reward value practical physical meaning. This facilitates intuitive observation and evaluation of the location selection of a single facility agent and the merits of multi-agent collaborative location selection schemes.

[0111] Step 5: Based on the objective function, let the multi-agent interact with the multi-scale spatial decision tree, and use the near-end policy optimization method to optimize the agent network parameters to obtain the optimal addressing strategy.

[0112] In practical implementation, the Actor network and Critic network (value network) in the facility agent update their network parameters through evaluation of decision trajectory data and multiple learning iterations, optimizing the facility agent's decision sequence (location strategy) in a multi-scale spatial decision tree, thereby finding the optimal decision sequence. This invention updates network parameters and optimizes the location strategy based on a proximal policy optimization algorithm. While ensuring the independence of the facility agent and fully exploring the potential of multi-agent collaborative location selection, it also maintains the steady optimization of the multi-agent collaborative location strategy, fully leveraging the advantages of multi-agent reinforcement learning collaborative location selection.

[0113] This invention optimizes policy parameters through gradient ascent. In order to find an optimal strategy In order to maximize expected return To maximize the overall reward obtained by the agent:

[0114] ;

[0115] in, This represents trajectory data obtained through the interaction between the intelligent agent and the environment. The policy of an agent is represented by its state. Choose action The probability of. It is the generalized advantage estimation function (GAE), used to evaluate the superiority or inferiority of taking a specific action in a given state compared to the average performance of the current policy:

[0116] ;

[0117] in, Indicates timing difference error. It is a discount factor used to control the degree of emphasis on future rewards. It is a hyperparameter. A larger value indicates that more timing differential steps are considered; it is typically set to 0.95. Indicates an immediate reward. It is the state value function. In practical applications, it is calculated using a reverse recursive method. : .

[0118] The objective function for each agent in the algorithm of this invention is:

[0119] ;

[0120] in, This means calculating the average value of the agent trajectory data collected in the current batch to obtain the empirical expectation. Indicate the importance sampling ratio by using the old strategy Collect empirical data to update strategies This allows the agent to learn from the same set of experience data multiple times during the training process, improving data utilization and learning efficiency. It is a hyperparameter, usually set to 0.2, used to limit the update magnitude of the policy.

[0121] To improve the exploration performance of the agent, entropy regularization is introduced by adding an entropy regularization term to the objective function:

[0122] ;

[0123] in, This is a hyperparameter; the larger the value, the stronger the agent's exploratory performance. .

[0124] This invention trains Several independent facility agents work together to complete the continuous location task. Each agent can be represented as... ,in Indicates policy network parameters, This represents the parameters of the value network. The network weights are initialized using an orthogonal initialization method to improve the training stability and performance of the agent. The learning rates for the policy network and the value network are set as follows: and Then update the network parameters using the Adam optimizer. Set the total number of training epochs. and the number of decision steps per round Set the number of training iterations for each batch of data. The specific process of the algorithm is shown in Table 1.

[0125] Table 1

[0126] ;

[0127] Step 6: Use the optimal site selection strategy to perform site selection planning in a multi-scale spatial decision tree to obtain the site selection scheme for urban public service facilities.

[0128] In practice, multiple facility agents learn the globally optimal collaborative site selection strategy through multiple rounds of training. They then use the optimal site selection strategy to perform site selection planning in a multi-scale spatial decision tree to obtain a site selection scheme for urban public service facilities.

[0129] The urban public service facility site selection method provided in this embodiment effectively solves the problem of facility site selection in continuous space by using multi-scale grid division of spatial regions. Based on multi-scale spatial division and multi-agent near-end strategy optimization reinforcement learning, it achieves efficient and high-quality facility site selection planning. It effectively overcomes the bottleneck of current facility site selection algorithms based on deep reinforcement learning, which are difficult to solve the problem of facility site selection in continuous space. It also well couples the multi-agent system, enabling the algorithm to cope with complex non-homogeneous site selection decision scenarios through diverse combinations of site selection strategies. It decomposes the huge computational load to multiple agents, providing a high-quality and high-performance solution for large-scale facility site selection planning and serving the practical application needs of urban public service facility site selection planning.

[0130] The method of the present invention will be further described below with reference to a specific embodiment. The specific process is as follows:

[0131] 1) Data Preparation. Collect data on existing charging stations in the central urban area of ​​City A, as well as vector data of City A's urban blocks and mobile phone signaling data of City A's population. Based on City A's mobile phone signaling data and the projected number of electric vehicles in City A by 2025, estimate the number of electric vehicles in the central urban area of ​​City A. In accordance with the policy documents provided by City A, set the service range of each existing charging station at 900m, capable of serving 112 electric vehicles, and calculate the current charging service gap, such as... Figure 4 As shown.

[0132] 2) Modeling the Facility Location Problem. The electric vehicle charging facility location problem is modeled as a continuous maximum coverage problem with distance decay effect and capacity constraints (refer to the technical solution). The goal is to find the locations of 130 charging stations to maximize the charging service provided.

[0133] 3) Construction of the agent's location environment. A multi-scale spatial decision tree is constructed by dividing a continuous spatial region using grids of different scales. Each tree node represents a spatial region and contains attribute information about that region; this node is called a region node. Each region node can be further subdivided into four child nodes. Each subdivision reduces the side length of the region represented by the node by half and its area to one-quarter of its original size. This multi-scale spatial decision tree serves as the agent's location environment.

[0134] 4) Facility Agent Attribute Settings. Each newly added charging station is controlled by an agent, whose internal structure is an artificial neural network with an Actor-Critic architecture. The state space, action space, reward function, and state transition function of each charging facility agent are set (refer to the technical solution). The reward value obtained by each agent is equal to the amount of charging service gap filled. The service range of each charging facility agent is set to 900m, and the number of electric vehicles it can serve is set to 112 (charging service volume). The charging service volume is allocated based on the urban area covered by the charging facility agent's service range.

[0135] 5) Agent Site Selection Strategy Learning. Each charging facility agent interacts with the site selection environment by performing actions and receives rewards. Each interaction is recorded as (current state, action, reward, next state). Multiple interactions form an interaction sequence, completing one facility site selection plan. The Actor-Critic artificial neural network is updated by repeatedly learning from the interaction sequence, thereby optimizing the agent's site selection strategy. For each agent, a proximal policy optimization method is used to update the network parameters, and the stability of multi-agent site selection strategy learning is ensured by limiting the policy update magnitude. The site selection strategies of 130 charging facility agents are simultaneously optimized through extensive training until these agents have the ability to collaboratively plan high-quality (high reward value) site selection schemes.

[0136] 6) Facility Site Selection Scheme Output. Using 130 trained charging facility agents, the system performs charging facility site selection planning in a site selection environment, determining the specific locations of 130 new charging stations in the central urban area of ​​City A. The site selection results are as follows: Figure 5 As shown.

[0137] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof.

[0138] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for site selection of urban public service facilities, characterized in that, include: Step 1: Collect facility data for the target area, wherein the facility data includes existing facility distribution data and facility service gap data; Step 2: Model the problem of urban public service facility site selection as a continuous maximum coverage problem with distance decay effect and capacity constraint, with the objective function being to maximize the demand to be satisfied. Step 2 specifically includes: Step 2.1: Model the target region as a continuous space, containing elements... A set of demand points : ； in, Indicates the first One demand point, These are the coordinates of the demand point. It is demand; Step 2.2, assuming there are in continuous space A collection of service facilities in a space to be planned : ； in, Indicates the first One service facility, These are the coordinates of the service facilities. The total amount of service that the facility can provide. The service scope of the facility; Step 2.3: When the service area of ​​a facility covers multiple demand points, the service volume provided by the facility will be allocated to each demand point according to its weight. ； in, Indication facilities For demand points Service volume provided, weight , This indicates the distance between the demand point and the facility, and , It is the distance decay effect function; Step 2.4, find The objective function is to select the location of each facility to maximize the demand that is met. ； in, This represents the set of spatial locations of all service facilities; Step 3: Use a multi-scale grid to divide the target region and construct a multi-scale spatial decision tree, which will serve as the location environment for the agent. Step 3 specifically includes: Step 3.1: Divide the continuous space using grids of different scales to form a multi-scale spatial decision tree. Each tree node in the multi-scale spatial decision tree represents a spatial region and contains the attribute information of the region it represents. It is called a region node. Each time the subdivision is completed, each region node generates four child nodes. The side length of the region represented by the node is reduced by half and the area becomes one-quarter of the original. Step 3.2: Based on this infinitely subdivisible multi-scale spatial decision tree, the continuous spatial region is infinitely subdivided to fully represent the entire continuous spatial region; Step 4: Allow multiple agents to control the site selection planning of multiple facilities respectively, and set the attributes of the agents, wherein the attributes include state space, action space and reward function; Step 5: Based on the objective function, let the multi-agent interact with the multi-scale spatial decision tree, and use the near-end policy optimization method to optimize the agent network parameters to obtain the optimal addressing strategy. Step 6: Use the optimal site selection strategy to perform site selection planning in a multi-scale spatial decision tree to obtain the site selection scheme for urban public service facilities.

2. The method according to claim 1, characterized in that, Step 4 specifically includes: Step 4.1: Each facility is controlled by an agent, and each agent is controlled by an Actor-Critic network. The agent interacts with the multi-scale spatial decision tree by outputting actions through the Actor network. Step 4.2: Set the state space, action space, and reward function for each agent.

3. The method according to claim 2, characterized in that, Step 5 specifically includes: Step 5.1: Based on the objective function, when the multi-agent interacts with the multi-scale spatial decision tree each time, the agent outputs the probability of going to the next level of four sub-region nodes based on the current level of region node information, and samples one sub-region node to move from. The reward is calculated based on the service provided by the agent to the demand point after the move. Step 5.2: Represent the record of each interaction as [state, action, reward, next moment state]. Through multiple interactions, an interaction sequence can be formed to complete a facility site selection plan. Step 5.3: For each agent, the Actor-Critic network parameters are updated using a proximal policy optimization method. The Actor-Critic network parameters are updated by learning the interaction sequence multiple times, and the stability of the multi-agent addressing policy learning is ensured by limiting the policy update magnitude, thereby optimizing the agent's addressing policy. Step 5.4: Repeat steps 5.1 to 5.3 to obtain the optimal addressing strategy.

4. The method according to claim 3, characterized in that, Step 5.2 specifically includes: Step 5.2.1: During each interaction, represent the state of each agent as follows: ； in, These are the spatial coordinates of the intelligent agent. It is the distance between the agent and the weighted center position of all demand points, calculated based on the demand. Indicates intelligent agent and The distance between each demand point express The demand quantity of each demand point in the current time step. Represents intelligent agents Compared with others The distance between agents; Step 5.2.2: Set each agent to have four selectable actions. The selectable action is to choose one of the four sub-region nodes to deploy the facility. The action space of the agent is represented as follows: ； in, This represents an action that controls the agent in... and Movement in direction This represents the step size of the intelligent agent. The number of steps is halved as the agent makes sequential decisions in the multi-scale spatial decision tree. Step 5.2.3, set the time step for each agent. The reward obtained is equal to the reward obtained from the decision at the current time step minus the reward obtained from the decision at the previous time step: ； Among them, the initial reward Calculations are performed based on the agent's initial position; Step 5.2.4: During each interaction, the process of the agent moving from the current spatial location to another spatial location through action decision, as well as the changes in the amount of service provided by the demand point and the changes in the relative spatial locations between the agent and the demand point, are taken as the state of the next moment. Step 5.2.5: Record the state, action, reward and next state after the move to form decision unit data. The decision unit sequence generated by the sequential decision-making process constitutes decision trajectory data, which is used as an interaction sequence.

5. The method according to claim 4, characterized in that, Step 5.3 specifically includes: Step 5.3.1: Calculate the generalized dominance estimation function using a reverse recursive approach and time-series difference error. Evaluate the superiority or inferiority of actions based on the generalized dominance estimation function. The expression for the generalized dominance estimation function is: ； ； in, Indicates timing difference error. It is a discount factor used to control the degree of emphasis on future rewards. It is a hyperparameter. Indicates an immediate reward. It is a state-value function. This represents the state value at time t+1. This represents the state value at time t; Step 5.3.2: Construct the objective optimization function for each agent based on the generalized advantage estimation function. ； in, This means calculating the average value based on the agent trajectory data collected in the current batch, thereby obtaining the empirical expectation. Indicate the importance sampling ratio by using the old strategy Collect empirical data to update strategies , It is a hyperparameter used to limit the update magnitude of the policy. Indicates an action; Step 5.3.3: Add an entropy regularization term to the objective optimization function. ； in, For hyperparameters, ; Step 5.3.4, represent each agent as... ,in, Represents the policy network of agent i. Represents the value network of agent i. Indicates policy network parameters, The parameters of the value network are represented by the orthogonal initialization method used to initialize the network weights. The learning rates for the policy network and the value network are set as follows: and And use the Adam optimizer to update the network parameters; Step 5.3.5, Set the total number of training rounds and the number of decision steps per round Set the number of training iterations for each batch of data. The network parameters are optimized through a gradient ascent strategy. Thus, the optimal addressing strategy is obtained. .

Images