Method and system for determining charging station guidance strategy in mixed cooperation and competition environment

Abstract

This invention provides a method and system for determining a charging station guidance strategy in a hybrid cooperative and competitive environment. The method models each charging station as an agent, with each agent's state space including the current operating state information of the corresponding charging station. Current charging demand information and the current operating state information of the distribution network are used as the global environmental state. A pre-trained multi-agent policy model is used to output a guidance strategy. Based on the guidance strategy, the coordinated optimization operation of the distribution network and the transportation system is achieved. Taking into account the charging demand of electric vehicles, the operating states of charging stations and the distribution network, and considering the hybrid cooperative and competitive relationships of charging stations, the multi-agent policy model constructed based on a heterogeneous hierarchical attention mechanism ensures that the generation of the guidance strategy fully considers user charging choices, distribution network operation, and the mutual influence between charging stations, thereby meeting the requirements of large-scale operation and improving the coordinated operation efficiency of the charging network and the electric transportation system.

Description

Technical Field

[0001] This invention relates to the field of decision optimization technology, specifically to a method and system for determining a charging station guidance strategy under a mixed cooperative and competitive environment. Background Technology

[0002] With the large-scale access of electric vehicles and the rapid deployment of charging infrastructure, charging stations in cities and regions are characterized by wide coverage, multiple nodes, and multiple operators. As a key hub connecting the transportation system and the power distribution network, the different charging guidance strategies of charging stations directly affect the charging choices and travel routes of electric vehicle users, while also changing the regional load distribution and the operating status of the power distribution network. Therefore, under large-scale operation conditions, charging guidance strategies are no longer a local problem of a single site or a single operator, but have evolved into a complex decision-making problem involving multiple charging stations, multiple operators, and the coupling constraints of the power-transportation system.

[0003] Current charging guidance strategies often make simplified assumptions about user behavior, which makes it difficult to meet the requirements of rationality, stability and system coordination under large-scale operation conditions, thus restricting the coordinated operation efficiency of charging networks and electric transportation systems. Summary of the Invention

[0004] To overcome the shortcomings of current charging guidance strategies, which fail to meet the requirements of rationality, stability, and system coordination under large-scale operation conditions and thus restrict the coordinated operation efficiency of charging networks and electric transportation systems, this invention provides a method for determining charging station guidance strategies in a hybrid cooperative and competitive environment, comprising: Obtain information on the current charging needs of electric vehicle users, the current operating status of each charging station, and the current operating status of the power distribution network; Each charging station is modeled as an agent in a multi-agent system, and the state space of each agent includes the current operating state information of the corresponding charging station. The current charging demand information and the current operating status information of the distribution network are used as the global environmental state. A pre-trained multi-agent policy model is used to output the guidance policy of each agent. Based on the guidance strategy of each of the intelligent agents, the operating parameters of each of the charging stations are adjusted to achieve coordinated and optimized operation of the power distribution network and the transportation system; The multi-agent policy model is constructed based on a heterogeneous hierarchical attention mechanism and trained with the optimization objective of achieving the optimal hybrid cooperation and competition relationship among charging stations.

[0005] Optionally, the multi-agent policy model includes a policy network and a value network; the pre-training process of the multi-agent policy model includes: Based on the obtained sample charging needs of electric vehicle users, sample operating status of each charging station, and sample operating status of the power distribution network, a heterogeneous hierarchical relationship structure is constructed for each of the intelligent agents. Through the policy network, based on the local association nodes in the heterogeneous hierarchical relationship structure of each agent, the state observations of charging stations of the same operator and different operators are fused through the local attention mechanism to generate a local embedded representation and output the corresponding candidate guidance policy. Based on the candidate guidance strategy of each agent, with the goal of minimizing the total charging cost, the system simulates electric vehicle users selecting target charging stations and calculates the reward value of each agent according to the reward function that considers the balance between mixed cooperation and competition. Through the value network, a global relationship structure is constructed with each charging station as a node. A hierarchical global attention mechanism is used to extract hybrid cooperation features and competitive relationship features. Based on the hybrid cooperation features and the competitive relationship features, gating fusion is performed to obtain the value estimate. Based on the reward value of each agent and the value estimate, combined with the loss function, the parameters of the value network are updated, and the network parameters shared by agents under the same operator are jointly optimized to obtain a trained multi-agent policy model.

[0006] Optionally, the reward function that considers the balance between mixed cooperation and competition includes: an intra-group cooperation term, an individual performance term, and an inter-group competition inhibition term; The intragroup cooperation items satisfy the following formula:

[0007] in, Represents the set of operators Total revenue in time period t; Let m be the charging revenue of charging station m during time period t; Let m be the waiting cost of charging station m during time period t; The individual operation item satisfies the following formula:

[0008] in, This represents the total revenue of charging station m during time period t; Let m be the charging revenue of charging station m during time period t; Let m be the waiting cost of charging station m during time period t; The inter-group competition inhibition term satisfies the following formula:

[0009] in, Including operator collection Other carrier collections Total revenue in time period t; For operators The charging revenue of charging station m within the area during time period t; For operators The waiting cost of charging station m within the time period t.

[0010] Optionally, the objective of minimizing the total charging cost satisfies the following formula:

[0011] in, This indicates the number of electric vehicle users during time period t. i If you choose to go to the charging station m The total cost incurred in completing the charging process. , , and These represent electric vehicle users during time period t. i If you choose to go to the charging station m The costs incurred in completing the charging process include driving time, waiting time, charging time, and charging fees, where n is the current traffic network node where the electric vehicle user is located.

[0012] Optionally, the gating fusion process based on the hybrid cooperation features and the competition relationship features satisfies the following formula:

[0013] in, For intelligent agents m The fusion representation vector; This represents the weighted embedding obtained by aggregating agent j with other agents m; This represents an attention-based linear embedding; is the gating coefficient.

[0014] Optionally, before modeling each charging station as an agent in a multi-agent system, the method further includes: Based on the operator relationships of each charging station, multiple operator sets are formed; charging stations within each operator set form cooperative relationships, while charging stations in different operator sets form competitive relationships; each charging station acts as an independent decision-making entity and makes autonomous decisions in a pre-defined discrete guidance strategy space.

[0015] Optionally, after adjusting the operating parameters of each charging station based on the guidance strategy of each of the intelligent agents, the method further includes: Under the guidance strategy of each intelligent agent, the charging load distribution of each charging station is obtained; Based on the charging load distribution of each charging station, an economic operation model for the power distribution network is constructed. Based on the aforementioned distribution network economic operation model, the optimal power flow that satisfies node power balance, voltage security constraints, and line capacity constraints is solved; based on the optimal power flow, the nodal marginal electricity price of each distribution network node connected to the charging station is generated. The marginal electricity price of each distribution network node to which each charging station is connected is used as a feedback signal to reflect the local power supply cost, and the current operating status information of the distribution network is updated.

[0016] Optionally, the charging demand information of the electric vehicle user includes at least one of the following: the current traffic network node where the electric vehicle is located, the time when the charging demand is generated, the remaining power of the vehicle, the vehicle battery capacity, and the expected charging amount; The operating status information of the charging station includes at least one of the following: the charging price of the charging station, the number and availability of charging piles, the number of vehicles charging in the station, and the number of vehicles waiting in the queue. The operating status information of the distribution network includes at least one of the following: voltage level of the charging station access node, active and reactive load of the node, power purchase cost of the distribution network, and marginal electricity price of the node.

[0017] On the other hand, the present invention also provides a system for determining a charging station guidance strategy in a hybrid cooperation and competition environment, comprising: The acquisition module is used to acquire information on the current charging needs of electric vehicle users, the current operating status of each charging station, and the current operating status of the power distribution network. The modeling module is used to model each charging station as an agent in a multi-agent system, and the state space of each agent includes the current operating state information of the corresponding charging station. The strategy generation module is used to take the current charging demand information and the current operating status information of the distribution network as the global environmental state, and use a pre-trained multi-agent strategy model to output the guidance strategy for each agent; wherein the multi-agent strategy model is constructed based on a heterogeneous hierarchical attention mechanism and is trained with the optimization objective of the optimal hybrid cooperation and competition relationship between charging stations. The collaborative optimization module is used to adjust the operating parameters of each charging station based on the guidance strategy of each of the intelligent agents, so as to achieve collaborative optimization of the power distribution network and the transportation system.

[0018] Optionally, the multi-agent policy model includes a policy network and a value network; it also includes a training module for: Based on the obtained sample charging needs of electric vehicle users, sample operating status of each charging station, and sample operating status of the power distribution network, a heterogeneous hierarchical relationship structure is constructed for each of the intelligent agents. Through the policy network, based on the local association nodes in the heterogeneous hierarchical relationship structure of each agent, the state observations of charging stations of the same operator and different operators are fused through the local attention mechanism to generate a local embedded representation and output the corresponding candidate guidance policy. Based on the candidate guidance strategy of each agent, with the goal of minimizing the total charging cost, the system simulates electric vehicle users selecting target charging stations and calculates the reward value of each agent according to the reward function that considers the balance between mixed cooperation and competition. Through the value network, a global relationship structure is constructed with each charging station as a node. A hierarchical global attention mechanism is used to extract hybrid cooperation features and competitive relationship features. Based on the hybrid cooperation features and the competitive relationship features, gating fusion is performed to obtain the value estimate. Based on the reward value of each agent and the value estimate, combined with the loss function, the parameters of the value network are updated, and the network parameters shared by agents under the same operator are jointly optimized to obtain a trained multi-agent policy model.

[0019] Optionally, the reward function that considers the balance between mixed cooperation and competition includes: an intra-group cooperation term, an individual performance term, and an inter-group competition inhibition term; The intragroup cooperation items satisfy the following formula:

[0020] in, Represents the set of operators In time period t Total revenue; Let m be the charging revenue of charging station m during time period t; Let m be the waiting cost of charging station m during time period t; The individual operation item satisfies the following formula:

[0021] in, This represents the total revenue of charging station m during time period t; Let m be the charging revenue of charging station m during time period t; Let m be the waiting cost of charging station m during time period t; The inter-group competition inhibition term satisfies the following formula:

[0022] in, Including operator collection Other carrier collections Total revenue in time period t; For operators The charging revenue of charging station m within the area during time period t; For operators The waiting cost of charging station m within the time period t.

[0023] Optionally, the objective of minimizing the total charging cost satisfies the following formula:

[0024] in, This indicates the number of electric vehicle users during time period t. i If you choose to go to the charging station m The total cost incurred in completing the charging process. , , and These represent electric vehicle users during time period t. i If you choose to go to the charging station m The costs incurred in completing the charging process include driving time, waiting time, charging time, and charging fees, where n is the current traffic network node where the electric vehicle user is located.

[0025] Optionally, the training module is used to implement the gating fusion process based on the hybrid cooperative features and the competitive relationship features through the following formula, satisfying the following formula:

[0026] in, For intelligent agents m The fusion representation vector; This represents the weighted embedding obtained by aggregating agent j with other agents m; This represents an attention-based linear embedding; is the gating coefficient.

[0027] Optionally, the modeling module is also used for: Based on the operator relationships of each charging station, multiple operator sets are formed; charging stations within each operator set form cooperative relationships, while charging stations in different operator sets form competitive relationships; each charging station acts as an independent decision-making entity and makes autonomous decisions in a pre-defined discrete guidance strategy space.

[0028] Optionally, the collaborative optimization module is also used for: Under the guidance strategy of each intelligent agent, the charging load distribution of each charging station is obtained; Based on the charging load distribution of each charging station, an economic operation model for the power distribution network is constructed. Based on the aforementioned distribution network economic operation model, the optimal power flow that satisfies node power balance, voltage security constraints, and line capacity constraints is solved; based on the optimal power flow, the nodal marginal electricity price of each distribution network node connected to the charging station is generated. The marginal electricity price of each distribution network node to which each charging station is connected is used as a feedback signal to reflect the local power supply cost, and the current operating status information of the distribution network is updated.

[0029] Optionally, the charging demand information of the electric vehicle user includes at least one of the following: the current traffic network node where the electric vehicle is located, the time when the charging demand is generated, the remaining power of the vehicle, the vehicle battery capacity, and the expected charging amount; The operating status information of the charging station includes at least one of the following: the charging price of the charging station, the number and availability of charging piles, the number of vehicles charging in the station, and the number of vehicles waiting in the queue. The operating status information of the distribution network includes at least one of the following: voltage level of the charging station access node, active and reactive load of the node, power purchase cost of the distribution network, and marginal electricity price of the node.

[0030] On the other hand, the present invention also provides a computer device, characterized in that it includes: one or more processors; The processor is used to store one or more programs; When the one or more programs are executed by the one or more processors, the method for determining the charging station guidance strategy in a hybrid cooperative and competitive environment described in any one of the above-mentioned methods is implemented.

[0031] On the other hand, the present invention also provides a computer-readable storage medium, characterized in that it stores a computer program thereon, which, when executed, implements the method for determining the charging station guidance strategy under a hybrid cooperation and competition environment as described in any one of the above.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention provides a method and system for determining a charging station guidance strategy in a hybrid cooperative and competitive environment. The method acquires current charging demand information of electric vehicle users, current operating status information of each charging station, and current operating status information of the power distribution network. Each charging station is modeled as an agent in a multi-agent system, with each agent's state space including the current operating status information of the corresponding charging station. The current charging demand information and the current operating status information of the power distribution network are used as the global environmental state. A pre-trained multi-agent strategy model is used to output the guidance strategy for each agent. Based on the guidance strategy of each agent, the operating parameters of each charging station are adjusted to achieve coordinated and optimized operation of the power distribution network and the transportation system. The multi-agent strategy model is constructed based on a heterogeneous hierarchical attention mechanism and trained with the optimal hybrid cooperative and competitive relationship between charging stations as the optimization objective. This invention comprehensively considers the charging demand of electric vehicles, the operating status of charging stations and power distribution networks, and constructs a multi-agent strategy model based on a heterogeneous hierarchical attention mechanism. This model can integrate the differences between different charging stations and take into account the mixed cooperative and competitive relationships between them. It ensures that the user's charging choice, power distribution network operation, and mutual influence between charging stations are fully considered during the generation of the guidance strategy. This satisfies the requirements for the rationality, stability, and system coordination of the charging guidance strategy under large-scale operation conditions, and improves the coordinated operation efficiency of the charging network and the power transportation system. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating the method for determining the charging station guidance strategy under a hybrid cooperation and competition environment according to the present invention. Figure 2 This is a schematic diagram of the dynamic pricing framework for charging stations in the power-transportation coupled environment of the present invention. Figure 3 This is a schematic diagram of the multi-agent deep reinforcement learning based on heterogeneous hierarchical attention mechanism of the present invention; Figure 4 This is a schematic diagram of the system architecture for determining the charging station guidance strategy under a hybrid cooperation and competition environment according to the present invention. Figure 5 This is a schematic diagram of the electronic device of the present invention. Detailed Implementation

[0034] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0035] Example 1: This invention provides a method for determining a charging station guidance strategy in a hybrid cooperative and competitive environment, such as... Figure 1 and Figure 2 As shown, it includes: Step 101: Obtain information on the current charging demand of electric vehicle users, the current operating status of each charging station, and the current operating status of the power distribution network; Step 102: Model each charging station as an agent in a multi-agent system. The state space of each agent includes the current operating state information of the corresponding charging station. Step 103: Using the current charging demand information and the current operating status information of the distribution network as the global environmental state, and using the pre-trained multi-agent policy model, output the guidance policy for each agent; wherein the multi-agent policy model is constructed based on a heterogeneous hierarchical attention mechanism and is trained with the optimal hybrid cooperation and competition relationship between charging stations as the optimization objective; Step 104: Based on the guidance strategy of each intelligent agent, adjust the operating parameters of each charging station to achieve coordinated and optimized operation of the power distribution network and transportation system.

[0036] The embodiments of this invention comprehensively consider the charging demand of electric vehicles, the operating status of charging stations and power distribution networks, and the multi-agent strategy model constructed based on a heterogeneous hierarchical attention mechanism. This model can take into account the differences between different charging stations and the mixed cooperative and competitive relationships between them, ensuring that the user's charging choice, power distribution network operation, and mutual influence between charging stations are fully considered during the generation of the guidance strategy. This satisfies the requirements for the rationality, stability, and system coordination of the charging guidance strategy under large-scale operation conditions, and improves the coordinated operation efficiency of the charging network and the power transportation system.

[0037] The guidance strategy to be determined in this embodiment of the invention is a charging guidance strategy, also known as a regulation guidance strategy. It generally refers to a set of controllable service parameters provided by charging stations to electric vehicle users to influence their choices of charging stations and charging time slots. Guidance strategies include, but are not limited to, one or more of the following: dynamic charging price, charging service level, charging power allocation priority, reservation queuing weight, navigation recommendation strength, and flexible adjustment of service time slots. Among these, the dynamic charging price, by autonomously adjusting within a discrete price range, directly affects the user's perceived total charging cost; other guidance parameters work synergistically from dimensions such as time, space, and service experience to jointly achieve flexible regulation of the user's charging behavior.

[0038] The charging demand information of electric vehicle (EV) users, the operating status information of charging stations, and the operating status information of the power distribution network in step 101 above can be used to construct the operating environment of the power-transportation coupled system. The charging demand information of EV users includes at least one of the following: the current traffic network node where the EV is located, the time when the charging demand is generated, the remaining battery power of the vehicle, the vehicle's battery capacity, and the expected charging amount. The operating status information of the charging station includes at least one of the following: the charging price of the charging station, the number and availability of charging piles, the number of vehicles currently charging at the station, and the number of vehicles waiting in the queue. The operating status information of the power distribution network includes at least one of the following: the voltage level of the node where the charging station is connected, the active and reactive loads of the node, the power purchase cost of the power distribution network, and the marginal electricity price of the node.

[0039] In actual operation, different charging stations typically belong to different operators. Each operator usually sets charging prices with the goal of maximizing its own profits. Charging stations exist in a competitive relationship, but also share power network and transportation resources to some extent, exhibiting a hybrid relationship of competition and cooperation. Therefore, this embodiment of the invention can further divide charging stations into multiple decision-making entities based on their operator affiliation, establishing a hybrid cooperation and competition model to determine the decision-making role of each charging station in the pricing process. Specifically, multiple operator sets can be formed based on the operator affiliation of each charging station; charging stations within each operator set form a cooperative relationship, while charging stations in different operator sets form a competitive relationship; each charging station, as an independent decision-making entity, makes autonomous decisions within a pre-defined discrete guidance strategy space. For example... Figure 2 Charging stations CS1 and CS2 shown belong to the same operator CSO. M 1. Charging station CS m and charging station CS m+1 Belonging to the same operator CSO M k Charging station CS M-1 and charging station CS M Belonging to the same operator CSO M K .

[0040] In one example, the guidance strategy may include charging service levels. Correspondingly, the discrete service strategy range includes multiple preset charging service levels, each level corresponding to different user guidance intensity or service rates, such as: high guidance (e.g., low price / high recommendation), medium guidance, and low guidance (e.g., high price / low recommendation).

[0041] In another example, the guidance strategy may include charging prices, and correspondingly, the discrete service strategy range may include discrete charging price ranges, within which each charging station can independently choose a charging price. Using this example as an illustration, current pricing strategies are mostly based on a single-site or single-operator perspective, making it difficult to characterize the mutual influence of pricing strategies among charging stations in a multi-operator environment. This can easily lead to local price fluctuations and disordered migration of user charging demand between stations, thereby exacerbating local traffic congestion or concentrated load on the power distribution network. However, this example considers both the competitive relationship between different charging station operators and the collaborative relationship within the same operator during the charging pricing process. This can characterize the mutual influence of charging station pricing behavior under multi-operator coexistence conditions, avoiding the problem of pricing strategy imbalance under a single-site or single-operator perspective, and improving the rationality and stability of pricing decisions. Furthermore, by modeling charging stations within a certain area as multiple decision-making entities and dynamically optimizing charging prices, it guides electric vehicle users to rationally choose charging stations, thereby improving the spatial and temporal distribution of charging load, avoiding localized load concentration at charging stations, and contributing to improving the overall operational efficiency of the charging network.

[0042] In one implementation, the guidance strategy determination process in 102 above can be modeled as a multi-agent Markov decision process, defining the state space, dynamic space, reward function and state transition of each charging station. In this modeling process, each charging station in the power-transportation coupling system is modeled as an agent.

[0043] For any charging station, its state space includes its operational state information, such as the state space... This includes at least the charging price for the previous period. Electricity purchase price in the previous period Number of electric vehicles entering the charging station during the current period Expected electric vehicle arrivals in the next period and average remaining battery power information of the arriving vehicle. Its action space is a set of optional charging prices (an example of a bootstrapping strategy), where the charging price... Limited to a preset price range, and represented in a discretized form, for example... , The preset lower limit of price, The system sets a preset price ceiling; its reward function is determined based on a hybrid revenue function, used to quantify the pricing effect of the charging station in the current time period. The hybrid revenue function will be explained later. The system state transition is used to determine the system operating state for the next time period based on the global state of the current decision cycle and the joint pricing actions of all charging stations. The changes in the system state include at least the update of the electric vehicle charging demand distribution and the changes in the charging station load state.

[0044] In one implementation, this invention proposes an efficient and scalable multi-agent reinforcement learning algorithm to jointly train the guidance strategies of each charging station. By selectively fusing state information from different charging stations, it achieves guidance strategy learning for each charging station in a hybrid cooperative-competitive environment. The following will combine... Figure 3 This will be explained. For example, a multi-agent policy model includes a policy network and a value network; the pre-training process of a multi-agent policy model may include the following steps: Based on the obtained sample charging needs of electric vehicle users, sample operating status of each charging station, and sample operating status of the power distribution network, a heterogeneous hierarchical relationship structure is constructed for each intelligent agent. Through the policy network, based on the local association nodes in the heterogeneous hierarchical relationship structure of each agent, the local attention mechanism is used to fuse the state observations of charging stations of the same operator and different operators, generate local embedded representations, and output the corresponding candidate guidance policies. Based on the candidate guidance strategy of each agent, with the goal of minimizing the total charging cost, the system simulates electric vehicle users selecting target charging stations and calculates the reward value of each agent according to the reward function that considers the balance between mixed cooperation and competition. A global relationship structure is constructed using a value network with each charging station as a node. A hierarchical global attention mechanism is used to extract hybrid cooperation features and competition features. Based on the hybrid cooperation features and competition features, gating fusion is performed to obtain value estimation. Based on the reward value and value estimate of each agent, combined with the loss function, the parameters of the value network are updated, and the network parameters shared by agents under the same operator are jointly optimized to obtain the trained multi-agent policy model.

[0045] The sample charging demand of electric vehicle users, the sample operating status of each charging station, and the sample operating status of the power distribution network obtained above can be used as sample data for the current training cycle or the current training step.

[0046] Based on the sample charging demands of electric vehicle users, the sample operating states of each charging station, and the sample operating states of the power distribution network, when constructing a heterogeneous hierarchical relationship structure for each agent, a heterogeneous hierarchical relationship structure can be constructed for each charging station based on the operator relationships to which the charging stations belong and the similarity of state observations between charging stations. In this structure, the representative charging station most similar to the current charging station's state is identified as the key association node, and other charging stations in the set are retained as supplementary association nodes, thus forming a heterogeneous hierarchical relationship graph containing at least two layers. For example, based on these sample data, the state observation vectors of each charging station can be extracted; then, according to the operator relationships to which the charging stations belong, all charging stations in the system can be divided into multiple operator sets. For any charging station m, within its operator set, calculate the state observation similarity between charging station m and all other charging stations within its operator set, based on cosine similarity, Euclidean distance, or a learnable similarity metric. Identify the representative charging station with the highest state observation similarity to charging station m (i.e., the most similar in state observation). The first layer of the heterogeneous hierarchical relationship structure is constructed by using these key nodes as the first layer of the key nodes, and the remaining charging stations in the set are used as supplementary nodes to form the second layer. Thus, a heterogeneous hierarchical relationship structure with at least two layers is constructed for charging station m. The first layer focuses on highly correlated collaborative charging stations, while the second layer retains global contextual information within the same operator group. In this example, the heterogeneous hierarchical relationship structure not only reflects the functional heterogeneity within the same operator due to differences in operating states, but also provides a hierarchical information aggregation path for the subsequent local attention mechanism: during the policy generation stage, the model prioritizes the state features of the key nodes in the first layer and combines them with the supplementary nodes in the second layer for context enhancement, thereby achieving refined modeling of the dynamic collaborative relationships within the cooperative group. For example, state observation similarity is used to characterize the similarity of different types of charging stations in spatial distribution, capacity characteristics, or access voltage levels. Figure 3 For example, charging stations include CS1 to CS8, where charging stations CS1 to CS4 belong to operator CSO 1, and charging stations CS5 to CS6 belong to operator CSO 2. Each charging station is modeled as an agent. For any agent m, the first layer of the heterogeneous hierarchical relationship structure obtained by this process contains agent m, the charging station j with the highest similarity to agent m is represented, and the remaining charging stations are contained in the second layer.

[0047] The above-mentioned method, which uses a policy network to fuse state observations of charging stations from the same operator and different operators based on local association nodes in the heterogeneous hierarchical relationship structure of each agent, and generates a local embedded representation and outputs the corresponding candidate guidance policy, may include the following steps: For each charging station, a local attention mechanism is constructed based on the first-layer node set of its heterogeneous hierarchical relationship structure. This mechanism weighted and fused the state observation information of the most relevant charging stations from the same operator and charging stations from different operators to generate a locally embedded representation for pricing decisions (here, we still take the guidance strategy including the charging price strategy as an example for illustration).

[0048] In the formula, For (local) embedding vectors; These are attention weights, used to quantize the embedding value vector of agent j when calculating the attention output of agent j. v j The relative importance of agent m. 、 、 These represent the query vector, key vector, and value vector obtained after linear projection, respectively. H is the kernel function. m Let T represent the set of nodes at the first level, and T represent the transpose operation.

[0049] 、 、 The specific definition is shown in the following formula:

[0050] In the formula, and Let m and j represent the observations of agents m and j, respectively. , , These are linear transformation matrices used to generate query, key, and value representations, respectively.

[0051] Kernel function The formula for calculation is:

[0052] In the formula, ELU() is the activation function, and x represents the input variable used to substitute the key vector. .

[0053] The local embedding representation is used as input to the policy network, which outputs the corresponding charging price action distribution. ,in This represents the action of agent m. This represents the policy function of agent m. These represent the parameters of the policy network of agent m. This indicates sampling from a probability distribution. This describes the embedded representation of the policy network for agent m, and uses this representation to generate the guidance policy for the current time period. Taking a charging price policy as an example, this example constructs a local attention mechanism based on the first-layer node set of a heterogeneous hierarchical relationship structure during the pricing policy generation process. It weighted and fused the state observation information of the most relevant charging stations from the same operator and charging stations from different operators to generate a local embedded representation for pricing decisions. This embedded representation is then used as input to the policy network, outputting the corresponding charging price decision. During this process, the actions of agent m... Actions with other intelligent agents This constitutes a combined action 'a', which includes the actions of 8 charging stations.

[0054] The reward function that considers the equilibrium of mixed cooperation and competition includes: intra-group cooperation, individual operation, and inter-group competition inhibition. Taking the guidance strategy including charging pricing as an example, the reward function can be a mixed reward function that integrates intra-group cooperation benefits, individual benefits, and inter-group competition inhibition for any charging station, used to characterize the pricing objective of the charging station in a mixed cooperation and competition environment.

[0055] In this example, the intra-group cooperation item, also known as the group-level revenue item, is used to characterize the total revenue generated by all charging stations belonging to the same charging station operator as the stated charging station during the current time period, and satisfies the following formula:

[0056] in, Represents the set of operators In time period t Total revenue; Let m be the charging revenue of charging station m during time period t; Let m be the waiting cost of charging station m during time period t; In this example, the individual operating item, also known as the individual revenue item, characterizes the individual revenue of the charging station during the current time period formed by the charging price and the corresponding charging demand, and satisfies the following formula:

[0057] in, This represents the total revenue of charging station m during time period t; Let m be the charging revenue of charging station m during time period t; Let m be the waiting cost of charging station m during time period t; In this example, the inter-group competition inhibition term, also known as the competition inhibition term, is used to characterize the total revenue generated by other charging station operators during the current time period. The impact on the pricing decisions of the charging stations, in order to curb irrational competitive behavior, satisfies the following formula:

[0058] in, Including operator collection Other carrier collections Total revenue in time period t; For operators The charging revenue of charging station m within the area during time period t; For operators The waiting cost of charging station m within the time period t.

[0059] The charging revenue of charging station m during time period t From charging price Charging needs and electricity purchase costs Together, they can be determined, for example, by satisfying the following formula: .

[0060] Among them, the cost of purchasing electricity is equal to the marginal electricity price at the node where the charging station is connected to the distribution network, for example... Figure 2 The marginal electricity price (DMLP) of the distribution network node connected to the charging station CS1 is: The marginal electricity price of the distribution network node to which the charging station CS2 is connected is Charging station CS m The marginal electricity price of the connected distribution network node is Charging station CS m+1 The marginal electricity price of the connected distribution network node is Charging station CS M-1 The marginal electricity price of the connected distribution network node is Charging station CS M The marginal electricity price of the connected distribution network node is Charging demand is determined by the charging price charged by electric vehicle users at various charging stations (e.g., Figure 2 The given charging price shown , , ..., Under the condition of minimizing the overall charging cost, the selection of charging stations is determined and is influenced by the charging pricing strategies of all charging stations.

[0061] In the hybrid reward function, these three terms can be dynamically weighted and combined, and the competition suppression term participates in the reward calculation in a negative form, so as to guide the strategy to improve collaboration and individual efficiency while suppressing vicious competition across operators.

[0062] Electric vehicle users' charging decisions are influenced by multiple factors, including charging price, travel time, waiting time, and charging duration. Users' route selection within the traffic network is coupled with their choice of charging stations, exhibiting significant spatiotemporal dynamics and uncertainty. Current pricing methods often make simplified assumptions about user behavior, failing to accurately reflect the comprehensive impact of price changes on user charging demand distribution and traffic flow evolution, leading to discrepancies between pricing results and actual operational conditions. This invention's embodiment considers multiple factors such as charging price, travel time, waiting time, and charging duration during the user decision-making process, enabling the formulation of guidance strategies that reflect user charging demand and traffic evolution. The above objective of minimizing total charging cost satisfies the following formula:

[0063] in, This indicates the number of electric vehicle users during time period t. i If you choose to go to the charging station m The total cost incurred in completing the charging process. , , and These represent electric vehicle users during time period t. i If you choose to go to the charging station m The costs incurred in completing the charging process include driving time, waiting time, charging time, and charging fees, where n is the current traffic network node where the electric vehicle user is located.

[0064] Electric vehicle users during time period t i If you choose to go to the charging station m The driving time cost incurred in completing the charging process Satisfy the following formula:

[0065] in, It is the time cost discount factor. Indicates road segment The required travel time is equal to the length of the road segment. Its speed The ratio, This represents the shortest path from the current traffic network node n of the electric vehicle to the charging station m, which belongs to the set of feasible paths. For example, for a given electric vehicle user i The charging request will be sent to the traffic network node where the user is currently located. n Considered as the starting point, located at node m The charging station is considered the destination, thus forming an origin-destination (OD) pair from the origin to the destination. Each OD pair can have multiple feasible paths. Connect the paths, where the travel time is shortest. Dijkstra's algorithm can be used for identification.

[0066] Electric vehicle users during time period t i If you choose to go to the charging station m The charging time cost incurred in completing the charging process Satisfy the following formula:

[0067] in, It is the time cost discount factor. This represents the battery capacity of electric vehicle user i. This represents the remaining battery capacity when electric vehicle user i sends a charging request during time period t. This represents the energy required for electric vehicle user i to reach charging station m during time period t. This indicates the charging power of charging station m.

[0068] Among them, electric vehicle users i Energy required to reach charging station m Vehicle unit energy consumption rate With the length of this road section d a The product of satisfies the following formula:

[0069] in, Indicates road segment The length of the road segment This represents the shortest path from the current traffic network node n of the electric vehicle to the charging station m, which belongs to the set of feasible paths. .

[0070] Electric vehicle users during time period t i If you choose to go to the charging station m Charging costs incurred during the charging process Satisfy the following formula:

[0071] in, This represents the battery capacity of electric vehicle user i. This represents the remaining battery capacity when electric vehicle user i sends a charging request during time period t. This represents the energy required for electric vehicle user i to reach charging station m during time period t. This indicates the charging price.

[0072] In one example, during the value assessment phase, a global relationship structure is constructed by treating all charging stations within the system as nodes. A hierarchical global attention mechanism is introduced into the value network to model the behavioral impact of charging stations from the same operator and those from different operators. Collaborative and competitive relationship features are extracted through attention aggregation. Subsequently, the embedded features from the same operator (i.e., homogeneous nodes) and different operators (i.e., heterogeneous nodes) are weighted and combined using a gating fusion mechanism. The gating weights are adaptively determined by the charging station's own local observation information to dynamically adjust the proportion of cooperative and competitive information in the value assessment. Based on the fused embedded representation and joint pricing actions, the long-term revenue of each charging station is estimated using the value network, and a training loss function is constructed. For multiple charging stations under the same operator, the value network parameters are shared, and the network parameters are updated through joint regression loss, thereby improving the stability of policy learning.

[0073] The specific implementation process of this example can be based on the following technical principles: For those M The system of charging stations, the policy network of the agents is represented as: The parameters are For example in Figure 3 In the middle, the parameters of the policy network m are The updated parameters are The parameters of the corresponding value network m are: The updated parameters are Since the reward function for each agent is defined as a hybrid reward that includes the overall profit of its affiliated operator CSO, the agent... m The rewards obtained are inevitably influenced by the behavioral choices of other agents, leading to a credit allocation problem. Inspired by the counterfactual baseline concept in the COMA (Counterfactual Multi-Agent Policy Gradient) algorithm, this problem is solved by comparing the actions of agents. m This problem is addressed by comparing the immediate reward gained from the current action (represented by a reward function) with the average expected reward of all possible actions of the same agent. A higher advantage value indicates a greater positive contribution of the current action to the agent's expected gain.

[0074] intelligent agent m Counterfactual baseline under its current strategy The calculation is as follows:

[0075] In the formula, Indicates the expectation. Indicating under the current joint action Q value; Represents intelligent agents m The set of all possible actions, This represents the action of agent m; This represents the actions of all other intelligent agents. Let m represent the policy network. For global observation, This indicates sampling from a probability distribution. Let m be the embedding representation of the policy network of agent m. This represents the action space of the intelligent agent m.

[0076] The value network then outputs a counterfactual baseline estimate for a given action. This baseline is then compared with the agent evaluated by the value network. m By comparing the joint action values, we can obtain the intelligent agent's... m The advantage function in the current environment is:

[0077] In the formula, It is the advantage function, which quantifies the agent. m In carrying out its current actions a The expected return relative to its average strategy.

[0078] Based on this, each agent maintains an independent policy network. To improve policy stability and promote thorough exploration during training, an entropy regularization term is introduced into the policy optimization objective:

[0079] In the formula, Let m be the policy loss function of agent m. Indicates the action Expectations are calculated according to strategy π. This represents the logarithmic probability of the current action under the agent's policy. Indicates the policy state Take action under the condition The probability of; The temperature coefficient is used to balance exploration and utilization by weighting the entropy term in the objective function. It is the dominance function.

[0080] In multi-agent policy gradient methods with counterfactual baselines, each agent network is updated using the policy gradient to maximize the expected cumulative discounted return. The gradient of the policy network is represented as:

[0081] In the formula, This indicates that the gradient is calculated with respect to the policy parameter θ.

[0082] During the value assessment phase, all charging stations within the system are treated as nodes to construct a global relationship structure. A hierarchical global attention mechanism is introduced into the value network to model the behavioral impact of charging stations from the same operator and charging stations from different operators. Collaborative relationship features and competitive relationship features are extracted through attention aggregation.

[0083] Embedded features from the same operator and different operators are weighted and combined using a gating fusion mechanism to obtain the intelligent agent. m The final embedding representation, which is the gating fusion process based on hybrid cooperative features and competitive features, satisfies the following formula:

[0084] In the formula, For intelligent agents m The fusion representation vector; This represents the weighted embedding obtained by linearly focusing on the most similar representative agent j and other agents m within the group, and then aggregating them. This represents an attention-based linear embedding, aggregated from adversary agents relative to their most similar adversaries; The gating coefficient is used in this formula. The gating fusion mechanism adaptively adjusts the fusion ratio of cooperative and competitive information based on the current operating status of the charging station. This ensures that the strategy generation process can respond to both group collaboration needs and cross-entity competitive pressures, thereby improving the adaptability and decision-making accuracy of the strategy model in complex environments.

[0085] The gating weights are adaptively determined by the charging station's own local observation information and are used to dynamically adjust the proportion of cooperative information and competitive information in the value assessment.

[0086] Based on the fused embedded representation and joint pricing actions, the long-term revenue of each charging station is utilized through the value network. Estimate the loss function and use it to construct the training loss function:

[0087] For multiple charging stations under the same operator that share value network parameters, updates are performed using joint regression loss:

[0088]

[0089] In the formula, Indicates the operator The joint loss function, This represents taking the expectation of the data in the experience replay pool D. These represent the state, action, reward, and next state, respectively. Indicating under the current joint action Q value, This indicates an embedded representation. This represents the Bellman objective value of agent m; r m For intelligent agents m The rewards received; Discount factor; Indicates the next time period Q value, This represents the embedding representation in the next time period. This indicates the action to be taken in the next time period.

[0090] In this implementation, a multi-agent deep reinforcement learning algorithm based on a heterogeneous hierarchical attention mechanism is adopted. Even when the scale of multiple charging stations expands, it can still effectively model the information of key related charging stations, avoid the computational complexity problem caused by the sharp increase in state dimension, and improve the applicability and scalability of the algorithm in large-scale charging networks.

[0091] Taking the guidance strategy, which includes the charging price strategy, as an example, based on the above training, in the actual operation of step 103, the charging price of each charging station is generated according to the pricing strategy obtained from the training in each time period, and the charging load distribution of the charging station is updated according to the charging selection behavior of electric vehicle users with the goal of minimizing the overall charging cost under the given charging price.

[0092] In this embodiment of the invention, based on the updated charging load information, the operating status of the distribution network is calculated, and an economic operation model of the distribution network is constructed. Under the conditions of satisfying node power balance, voltage constraints, and line capacity constraints, the marginal electricity price corresponding to each node is obtained. The electricity purchase cost of each charging station is determined by the marginal electricity price corresponding to its access node, and is used as feedback information input into the pricing decision process of the next decision cycle, forming a closed-loop interaction between the power system and the transportation system. For example, after step 104 above, the following steps may also be included: Under the guidance strategy of each agent, the charging load distribution of each charging station is obtained; Based on the charging load distribution of each charging station, an economic operation model of the power distribution network is constructed. Based on the economic operation model of the distribution network, the optimal power flow that satisfies the constraints of node power balance, voltage security, and line capacity is solved; based on the optimal power flow, the nodal marginal electricity price of each distribution network node connected to the charging station is generated. The marginal electricity price of each distribution network node to which each charging station is connected is used as a feedback signal to reflect the local power supply cost, and the current operating status information of the distribution network is updated.

[0093] As a crucial component of the distribution network, the spatiotemporal distribution of charging station load directly impacts node voltage levels, line load rates, and distribution network operating costs. However, traditional charging pricing methods often employ fixed electricity prices or simplified electricity purchase cost models, failing to adequately consider distribution network operational constraints and the dynamic changes in node marginal supply costs. This makes it difficult for charging prices to accurately reflect the operational pressure on the power system side, posing a risk of inducing local grid congestion and increased operating costs. This proposed approach fully considers node marginal electricity prices and charging load, enabling the formulated guidance strategy to accurately reflect the operational pressure on the power system side and reduce the risks of grid congestion and excessively high operating costs.

[0094] Through multi-cycle iterations, the guidance strategy for charging stations can be dynamically adjusted, gradually bringing the system's operating state to a stable range. This improves the overall operational efficiency of charging stations and promotes coordinated operation between the power system and the transportation system. The adjustment cycle for the guidance strategy can be, but is not limited to, once every 15 minutes.

[0095] In a competitive environment with multiple charging stations and multiple operators, the status information and operational data of charging stations are highly dynamic, highlighting information asymmetry among different operators and the potential for strategic adjustments or even information distortion during pricing decisions. Traditional solutions rely on rule-based or static game models for pricing, which depend on precise system modeling and the assumption of complete information. This approach struggles to maintain stability and effectiveness when faced with randomly arriving charging demand, fluctuating traffic conditions, and uncertainties in power system operation. However, in an environment with multiple charging stations and multiple operators, a multi-agent deep reinforcement learning framework models the process of determining charging station guidance strategies as a multi-agent collaborative decision-making problem. This allows for joint optimization of charging station guidance strategies in a hybrid cooperation-competition environment. When the guidance strategy includes charging prices, a multi-agent deep reinforcement learning algorithm based on a heterogeneous hierarchical attention mechanism is applied to the coupled power-transportation system. Considering user charging choices, power system operating costs, and the mutual influence between charging stations, dynamic optimization of charging prices improves charging station operational efficiency, reduces distribution network operating costs, and promotes coordinated operation between the power and transportation systems. In an operating environment with multiple charging stations and multiple operating entities, the number of charging stations and charging demand continue to grow. This enables collaborative modeling of entities, characterization of user behavior responses, and handling of power-transport coupling constraints. It can meet the requirements for reasonable charging prices, stability, and system coordination under large-scale operation conditions, and improve the collaborative operation efficiency of the charging network and the power transportation system.

[0096] Example 2: Based on the same inventive concept, this invention also provides a system for determining a charging station guidance strategy in a hybrid cooperative and competitive environment, as shown in the schematic diagram below. Figure 4 As shown, it includes: The acquisition module is used to acquire information on the current charging needs of electric vehicle users, the current operating status of each charging station, and the current operating status of the power distribution network. The modeling module is used to model each charging station as an agent in a multi-agent system. The state space of each agent includes the current operating state information of the corresponding charging station. The strategy generation module uses the current charging demand information and the current operating status information of the distribution network as the global environmental state, and uses a pre-trained multi-agent strategy model to output the guidance strategy for each agent. The multi-agent strategy model is built based on a heterogeneous hierarchical attention mechanism and is trained with the optimization objective of the optimal hybrid cooperation and competition relationship between charging stations. The collaborative optimization module is used to adjust the operating parameters of each charging station based on the guidance strategy of each agent, so as to achieve collaborative optimization of the power distribution network and transportation system.

[0097] In one specific implementation, the multi-agent policy model includes a policy network and a value network; it also includes a training module for: Based on the obtained sample charging needs of electric vehicle users, sample operating status of each charging station, and sample operating status of the power distribution network, a heterogeneous hierarchical relationship structure is constructed for each intelligent agent. Through the policy network, based on the local association nodes in the heterogeneous hierarchical relationship structure of each agent, the state observations of charging stations of the same operator and different operators are fused through the local attention mechanism to generate local embedded representations and output the corresponding candidate guidance policies. Based on the candidate guidance strategy of each agent, with the goal of minimizing the total charging cost, the system simulates electric vehicle users selecting target charging stations and calculates the reward value of each agent according to the reward function that considers the balance between mixed cooperation and competition. A global relationship structure is constructed using a value network with each charging station as a node. A hierarchical global attention mechanism is used to extract hybrid cooperation features and competition features. Based on the hybrid cooperation features and competition features, gating fusion is performed to obtain value estimation. Based on the reward value and value estimate of each agent, combined with the loss function, the parameters of the value network are updated, and the network parameters shared by agents under the same operator are jointly optimized to obtain the trained multi-agent policy model.

[0098] In one specific implementation, the reward function that considers the equilibrium of mixed cooperation and competition includes: intra-group cooperation term, individual performance term, and inter-group competition inhibition term; Intragroup collaboration items satisfy the following formula:

[0099] in, Represents the set of operators Total revenue in time period t; Let m be the charging revenue of charging station m during time period t; Let m be the waiting cost of charging station m during time period t; Individual running items satisfy the following formula:

[0100] in, This represents the total revenue of charging station m during time period t; Let m be the charging revenue of charging station m during time period t; Let m be the waiting cost of charging station m during time period t; The inter-group competition inhibition term satisfies the following formula:

[0101] in, Including operator collection Other carrier collections Total revenue in time period t; For operators The charging revenue of charging station m within the area during time period t; For operators The waiting cost of charging station m within the time period t.

[0102] In one specific implementation, the following formula is satisfied with the goal of minimizing the total charging cost:

[0103] in, This indicates the number of electric vehicle users during time period t. i If you choose to go to the charging station m The total cost incurred in completing the charging process. , , and These represent electric vehicle users during time period t. i If you choose to go to the charging station m The costs incurred in completing the charging process include driving time, waiting time, charging time, and charging fees, where n is the current traffic network node where the electric vehicle user is located.

[0104] In one specific implementation, the training module is used to perform gated fusion based on hybrid cooperative features and competitive relationship features through the following formula, satisfying the following formula:

[0105] in, For intelligent agents m The fusion representation vector; This represents the weighted embedding obtained by aggregating agent j with other agents m; This represents an attention-based linear embedding; is the gating coefficient.

[0106] In one specific implementation, the modeling module is also used for: Based on the operator relationships of each charging station, multiple operator sets are formed; charging stations within each operator set form cooperative relationships, while charging stations in different operator sets form competitive relationships; each charging station acts as an independent decision-making entity and makes autonomous decisions in a pre-defined discrete guidance strategy space.

[0107] In one specific implementation, the collaborative optimization module is also used for: Under the guidance strategy of each agent, the charging load distribution of each charging station is obtained; Based on the charging load distribution of each charging station, an economic operation model of the power distribution network is constructed. Based on the economic operation model of the distribution network, the optimal power flow that satisfies the constraints of node power balance, voltage security, and line capacity is solved; based on the optimal power flow, the nodal marginal electricity price of each distribution network node connected to the charging station is generated. The marginal electricity price of each distribution network node to which each charging station is connected is used as a feedback signal to reflect the local power supply cost, and the current operating status information of the distribution network is updated.

[0108] In one specific implementation, the charging demand information of electric vehicle users includes at least one of the following: the current traffic network node where the electric vehicle is located, the time when the charging demand is generated, the remaining power of the vehicle, the vehicle battery capacity, and the expected charging amount; The operational status information of a charging station includes at least one of the following: the charging price of the charging station, the number and availability of charging piles, the number of vehicles currently charging at the station, and the number of vehicles waiting in the queue. The operating status information of the distribution network includes at least one of the following: the voltage level of the charging station access node, the active and reactive loads of the node, the power purchase cost of the distribution network, and the marginal electricity price of the node.

[0109] Example 3: like Figure 5As shown, the present invention also provides an electronic device, which may be a computer device, a microcontroller device, a smart mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, processor, and transceiver component are connected via a bus; the memory can be used to store executable programs, and an exemplary executable program may include instructions; the processor is used to execute the instructions stored in the memory. The memory can also be used to store data, which can be accessed and / or modified when instructions are executed.

[0110] The processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and it is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the storage medium to implement the corresponding method flow or corresponding function, so as to realize the steps of the method for determining the charging station guidance strategy in a hybrid cooperation and competition environment in the above embodiments.

[0111] Example 4: Based on the same inventive concept, this invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory). This readable storage medium is a memory device within an electronic device used to store programs and data. It is understood that the storage medium here can include both built-in storage media within the electronic device and extended storage media supported by the electronic device. The storage medium provides storage space, which stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more executable programs (including program code). It should be noted that the storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. Loading and executing one or more instructions stored in the storage medium by the processor can implement the steps of the method for determining a charging station guidance strategy under a hybrid cooperative and competitive environment as described in the above embodiments.

[0112] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit its scope of protection. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that after reading the present invention, they can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the scope of protection of the claims pending approval.

Claims

1. A method for determining a charging station guidance strategy under a hybrid cooperation and competition environment, characterized in that, include: Obtain information on the current charging needs of electric vehicle users, the current operating status of each charging station, and the current operating status of the power distribution network; Each charging station is modeled as an agent in a multi-agent system, and the state space of each agent includes the current operating state information of the corresponding charging station. The current charging demand information and the current operating status information of the distribution network are used as the global environmental state. A pre-trained multi-agent policy model is used to output the guidance policy of each agent. Based on the guidance strategy of each of the intelligent agents, the operating parameters of each of the charging stations are adjusted to achieve coordinated and optimized operation of the power distribution network and the transportation system; The multi-agent policy model is constructed based on a heterogeneous hierarchical attention mechanism and trained with the optimization objective of achieving the optimal hybrid cooperation and competition relationship among charging stations.

2. The method as described in claim 1, characterized in that, The multi-agent policy model includes a policy network and a value network; The pre-training process of the multi-agent policy model includes: Based on the obtained sample charging needs of electric vehicle users, sample operating status of each charging station, and sample operating status of the power distribution network, a heterogeneous hierarchical relationship structure is constructed for each of the intelligent agents. Through the policy network, based on the local association nodes in the heterogeneous hierarchical relationship structure of each agent, the state observations of charging stations of the same operator and different operators are fused through the local attention mechanism to generate a local embedded representation and output the corresponding candidate guidance policy. Based on the candidate guidance strategy of each agent, with the goal of minimizing the total charging cost, the system simulates electric vehicle users selecting target charging stations and calculates the reward value of each agent according to the reward function that considers the balance between mixed cooperation and competition. Through the value network, a global relationship structure is constructed with each charging station as a node. A hierarchical global attention mechanism is used to extract hybrid cooperation features and competitive relationship features. Based on the hybrid cooperation features and the competitive relationship features, gating fusion is performed to obtain the value estimate. Based on the reward value of each agent and the value estimate, combined with the loss function, the parameters of the value network are updated, and the network parameters shared by agents under the same operator are jointly optimized to obtain a trained multi-agent policy model.

3. The method as described in claim 2, characterized in that, The reward function that considers the balance between mixed cooperation and competition includes: intragroup cooperation term, individual performance term, and intergroup competition inhibition term; The intragroup cooperation items satisfy the following formula: in, Represents the set of operators Total revenue in time period t; Let m be the charging revenue of charging station m during time period t; Let m be the waiting cost of charging station m during time period t; The individual operation item satisfies the following formula: in, This represents the total revenue of charging station m during time period t; Let m be the charging revenue of charging station m during time period t; Let m be the waiting cost of charging station m during time period t; The inter-group competition inhibition term satisfies the following formula: in, Including operator collection Other carrier collections Total revenue in time period t; For operators The charging revenue of charging station m within the area during time period t; For operators The waiting cost of charging station m within the time period t.

4. The method as described in claim 2 or 3, characterized in that, The objective of minimizing total charging cost satisfies the following formula: in, This indicates the number of electric vehicle users during time period t. i If you choose to go to the charging station m The total cost incurred in completing the charging process. , , and These represent electric vehicle users during time period t. i If you choose to go to the charging station m The costs incurred in completing the charging process include driving time, waiting time, charging time, and charging fees, where n is the current traffic network node where the electric vehicle user is located.

5. The method as described in claim 2, characterized in that, The gating fusion process based on the hybrid cooperation features and the competition relationship features satisfies the following formula: in, For intelligent agents m The fusion representation vector; This represents the weighted embedding obtained by aggregating agent j with other agents m; This represents an attention-based linear embedding; is the gating coefficient.

6. The method as described in claim 1, characterized in that, Before modeling each charging station as an agent in a multi-agent system, the process also includes: Based on the operator relationships of each charging station, multiple operator sets are formed; charging stations within each operator set form cooperative relationships, while charging stations in different operator sets form competitive relationships; each charging station acts as an independent decision-making entity and makes autonomous decisions in a pre-defined discrete guidance strategy space.

7. The method as described in claim 1, characterized in that, After adjusting the operating parameters of each charging station based on the guidance strategy of each of the intelligent agents, the method further includes: Under the guidance strategy of each intelligent agent, the charging load distribution of each charging station is obtained; Based on the charging load distribution of each charging station, an economic operation model for the power distribution network is constructed. Based on the aforementioned distribution network economic operation model, the optimal power flow that satisfies node power balance, voltage security constraints, and line capacity constraints is solved; based on the optimal power flow, the nodal marginal electricity price of each distribution network node connected to the charging station is generated. The marginal electricity price of each distribution network node to which each charging station is connected is used as a feedback signal to reflect the local power supply cost, and the current operating status information of the distribution network is updated.

8. The method as described in claim 1 or 7, characterized in that, The charging demand information of electric vehicle users includes at least one of the following: the current traffic network node where the electric vehicle is located, the time when the charging demand is generated, the remaining power of the vehicle, the vehicle battery capacity, and the expected charging amount; The operating status information of the charging station includes at least one of the following: the charging price of the charging station, the number and availability of charging piles, the number of vehicles charging in the station, and the number of vehicles waiting in the queue. The operating status information of the distribution network includes at least one of the following: voltage level of the charging station access node, active and reactive load of the node, power purchase cost of the distribution network, and marginal electricity price of the node.

9. A system for determining a charging station guidance strategy under a hybrid cooperation and competition environment, characterized in that, include: The acquisition module is used to acquire information on the current charging needs of electric vehicle users, the current operating status of each charging station, and the current operating status of the power distribution network. The modeling module is used to model each charging station as an agent in a multi-agent system, and the state space of each agent includes the current operating state information of the corresponding charging station. The strategy generation module is used to take the current charging demand information and the current operating status information of the distribution network as the global environmental state, and use a pre-trained multi-agent strategy model to output the guidance strategy for each agent. The multi-agent policy model is constructed based on a heterogeneous hierarchical attention mechanism and trained with the optimization objective of the optimal hybrid cooperation and competition relationship among charging stations. The collaborative optimization module is used to adjust the operating parameters of each charging station based on the guidance strategy of each of the intelligent agents, so as to achieve collaborative optimization of the power distribution network and the transportation system.

10. An electronic device, characterized in that, include: At least one processor and memory; The memory and processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the method for determining the charging station guidance strategy in a hybrid cooperative and competitive environment as described in any one of claims 1 to 8 is implemented.

Images