Multi-agent-based charging guiding method and device, product and storage medium

Abstract

The invention discloses a multi-agent-based charging guiding method and device, a product and a storage medium, and relates to the technical field of intelligent traffic and energy management. By constructing a multi-agent game model comprising a charging station agent and a user agent, each charging station can independently generate a regulation and control strategy based on local state data. And meanwhile, the regulation and control effect is predicted through a selection preference model of the user agent, and strategy network parameters are continuously optimized based on an actual execution result, so that the regulation and control decision of the charging station can adapt to dynamically changing charging requirements. Besides, each charging station can independently deploy the optimized strategy network to execute charging guidance, so that the calculation burden of unified processing and decision making of the central system is avoided, and the response speed of the charging guidance system to the charging demand change is effectively improved.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation and energy management technology, specifically to a charging guidance method, device, product, and storage medium based on multiple agents. Background Technology

[0002] With the rapid popularization of new energy vehicles, the scale of urban charging infrastructure construction is constantly expanding. In order to improve the utilization efficiency of charging resources, various charging operators have successively developed charging management systems, which guide users to choose suitable charging stations by collecting real-time data from charging stations, thereby alleviating charging congestion.

[0003] Currently, the mainstream charging guidance method mainly adopts a centralized scheduling approach. This method deploys a central control system to uniformly manage and schedule charging resources. Specifically, the central control system recommends the optimal charging station to users based on real-time status information uploaded by each charging station (such as charging pile availability and queue length) and pre-set scheduling rules. For example, a smart charging platform developed by one charging operator recommends the nearest available charging station to users by analyzing the congestion level of charging stations; another operator uses a price adjustment mechanism to increase charging prices during congested periods to divert charging demand.

[0004] However, this centralized scheduling method has limitations in practical applications. Due to the strong spatiotemporal dynamics of charging demand, pre-set fixed scheduling rules are difficult to adapt to complex and ever-changing real-world scenarios. The central control system needs to process data from all charging stations and make real-time decisions; as the charging network expands, the system's response speed struggles to meet actual demands. Summary of the Invention

[0005] In view of this, this application provides a charging guidance method, device, product, and storage medium based on multiple agents.

[0006] Firstly, this application provides a charging guidance method based on multiple agents, the method comprising: Obtain the operation data of the charging network, and construct a multi-agent game model based on the operation data. The multi-agent game model includes multiple charging station agents and multiple user agents. Extract the state data of each charging station intelligent agent from the operational data, and generate the state feature vector of each charging station intelligent agent based on the state data; The state feature vector is input into a preset policy network for decision calculation to obtain the control instructions of each charging station agent. Based on the control instructions, guidance information is generated, and based on the user agent's selection preference model, the response of each user agent to the guidance information is simulated to obtain the user distribution prediction results. Feedback data is collected based on the user distribution prediction results and actual execution effects, and performance evaluation values ​​are calculated based on the feedback data. The network parameters of the strategy network of each charging station agent are updated based on the performance evaluation values. The optimized network parameters are deployed to each charging station to enable each charging station to independently perform charging guidance.

[0007] By adopting the above technical solution and constructing a multi-agent game model that includes charging station agents and user agents, each charging station can independently generate control strategies based on local state data. Simultaneously, the control effect is predicted using a user agent's preference model, and the strategy network parameters are continuously optimized based on actual execution results, enabling the charging station's control decisions to adapt to dynamically changing charging demands. Furthermore, since each charging station can independently deploy the optimized strategy network to execute charging guidance, the computational burden of centralized system processing and decision-making is avoided, effectively improving the charging guidance system's response speed to changes in charging demand.

[0008] Optionally, constructing a multi-agent game model based on the operational data includes: Extract the facility configuration data and operational status data of the charging station, as well as the user's demand characteristics and behavioral data from the operational data; Based on the facility configuration data and operation status data of the charging stations, each charging station is modeled as a charging station intelligent agent, and the state space, action space and reward function of each charging station intelligent agent are determined. Based on the user's demand characteristics and behavior data, charging users are modeled as user intelligent agents, and selection preference models for each user intelligent agent are configured based on the behavior data. Based on the interaction relationship between the charging station agent and the user agent, the multi-agent game model is constructed.

[0009] Optionally, determining the state space, action space, and reward function of each charging station agent includes: Based on the facility configuration data and operation status data of the charging station, the state space of each charging station's intelligent agent is determined. The state space includes the number of charging piles, the current load status, the queuing status, and the price information. Based on the control capabilities of the charging station, the action space of each charging station's intelligent agent is determined, and the action space includes price adjustment actions and guidance and incentive actions. Based on the optimization objectives of charging guidance, a reward function is constructed for each charging station agent. The reward function includes a load balancing reward, a revenue reward, and a service quality penalty.

[0010] Optionally, configuring the selection preference model for each user agent based on the behavioral data includes: Extract users' historical selection records from the behavioral data and analyze the factors influencing users' choice of charging stations. A user utility function is constructed based on the aforementioned influencing factors, and the utility function includes distance factors, price factors, waiting time factors, and incentive factors. Based on the historical selection records, the weight parameters of each influencing factor in the utility function are determined. Based on the utility function, a probability model for the user's choice of charging station is established, which serves as the choice preference model for each user agent.

[0011] Optionally, the step of inputting the state feature vector into a preset policy network for decision calculation to obtain the control instructions for each charging station agent includes: Feature encoding is performed on the state feature vectors of each charging station's intelligent agent; The encoded state feature vector is input into a preset policy network, and forward propagation calculation is performed through the neural network layer of the policy network to obtain the action probability distribution of each charging station agent. Based on the aforementioned action probability distribution, determine the control actions of each charging station's intelligent agent; Based on the aforementioned regulatory actions, corresponding regulatory instructions are generated, including price adjustment instructions and incentive guidance instructions.

[0012] Optionally, the step of collecting feedback data based on the user distribution prediction results and actual execution results, and calculating a performance evaluation value based on the feedback data, includes: Based on the user distribution prediction results, obtain the expected state data of each charging station's intelligent agent; After each charging station executes the control command, the actual operating data of each charging station is collected. The actual operating data includes actual user distribution, load status and service quality indicators. By comparing the expected state data with the actual operating data, feedback data is generated. Based on the feedback data, the instantaneous reward value is calculated according to the reward function of each charging station agent; The instant reward values ​​are accumulated to obtain the performance evaluation value of each charging station agent.

[0013] Optionally, updating the network parameters of the strategy network for each charging station agent based on the performance evaluation value includes: The state feature vectors, control actions, and performance evaluation values ​​of each charging station's intelligent agent are used to construct training samples. The training samples are stored in the experience replay pool, and batch training data is sampled from the experience replay pool; The batch training data is input into the policy network of each charging station agent, and the policy gradient is calculated based on the performance evaluation value. Based on the policy gradient, the network parameters of the policy network of each charging station agent are updated using the gradient ascent algorithm.

[0014] A second aspect of this application provides a multi-agent-based charging-guided electronic device, the electronic device comprising: one or more processors and a memory; the memory being coupled to the one or more processors, the memory being used to store computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the multi-agent-based charging-guided electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0015] A third aspect of this application provides a computer program product containing instructions that, when run on a multi-agent-based charging-guided electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0016] A fourth aspect of this application provides a computer-readable storage medium including instructions that, when executed on a multi-agent-based charging-guided electronic device, cause the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0017] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: This application constructs a multi-agent game model comprising charging station agents and user agents, enabling each charging station to independently generate control strategies based on local state data. Simultaneously, it predicts the control effects using a user agent's preference model and continuously optimizes the strategy network parameters based on actual execution results, allowing the charging station's control decisions to adapt to dynamically changing charging demands. Furthermore, since each charging station can independently deploy the optimized strategy network to execute charging guidance, the computational burden of centralized system processing and decision-making is avoided, effectively improving the charging guidance system's response speed to changes in charging demand. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a multi-agent-based charging guidance method provided in an embodiment of this application; Figure 2 This is a diagram of a MADDPG multi-agent deep reinforcement learning architecture provided in an embodiment of this application; Figure 3 This is a schematic diagram of an exemplary multi-agent-based charging guidance method provided in an embodiment of this application; Figure 4 This is a system load rate comparison curve provided in an embodiment of this application; Figure 5 This is a schematic diagram of an exemplary hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0020] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0021] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0022] Please refer to Figure 1 A flowchart illustrating a multi-agent-based charging guidance method is presented. This method can be implemented using a computer program, a microcontroller, or run on a multi-agent-based charging guidance device. The computer program can be integrated into the computer device or run as a standalone utility application. Specifically, the method includes steps 10 to 60, as follows: Step 10: Obtain the operation data of the charging network, and construct a multi-agent game model based on the operation data. The multi-agent game model includes multiple charging station agents and multiple user agents.

[0023] In this embodiment of the application, the operational data refers to multi-dimensional real-time and historical information generated during the operation of the charging network, including operational data such as the facility configuration, load status, queuing situation, and price revenue of charging stations; supply-side data such as the real-time electricity price and load status of the power grid; and demand-side data such as the distribution of users' charging demand, historical selection records, and behavioral preferences. It also includes environmental data such as time context and traffic conditions. This data provides basic information support for the construction of multi-agent game models and strategy optimization.

[0024] The multi-agent game model refers to modeling N charging stations and M users in a charging network as autonomous agents, forming a partially observable stochastic game system. The charging station agents output price adjustment and incentive strategies based on local observation information, while the user agents make charging station selection decisions based on stochastic utility theory. In the process of maximizing their own interests, each agent optimizes through strategy interaction and learning, ultimately achieving multi-objective collaborative optimization of charging network load balancing, improved operational efficiency, and enhanced user satisfaction.

[0025] As an optional embodiment, the step of constructing a multi-agent game model based on operational data may further include the following steps: Step 101: Extract the facility configuration data and operational status data of the charging station, as well as the user's demand characteristics and behavior data from the operational data.

[0026] Specifically, the system collects facility configuration data for each charging station i by calling the data interface of the charging station operation and management platform. This includes static information such as the number of charging piles, charging power level, geographical coordinates, and service area coverage. This data determines the basic service capabilities of the charging station's intelligent agent. Simultaneously, the system acquires real-time operational status data for the charging station, including dynamic information such as current load status, queue length, real-time service price, and historical revenue data. This data is updated every minute through the charging station's IoT sensors and power monitoring system. For user demand characteristic data, the system analyzes the user's charging app registration information and device data to extract basic characteristic parameters such as the user's vehicle battery capacity, frequently used charging periods, and acceptable maximum driving distance. More importantly, the system establishes a user behavior database by mining users' historical charging behavior data. This database records the user's actual decision-making records when faced with a set of charging station choices over T historical time steps, including the status information of the alternative charging stations at each selection, the user's current location, battery status, and the final selection result. This behavioral data will be used to calibrate the stochastic utility function parameters of the user's intelligent agent. Through this comprehensive and systematic data extraction process, the system obtained the complete data support required to build a multi-agent game model under the MADDPG framework. This provides a reliable data foundation for subsequent accurate modeling of user choice preferences and charging station decision-making behavior, ensuring that the game model can truly reflect the actual operating characteristics of the charging network and the patterns of user behavior.

[0027] Step 102: Based on the facility configuration data and operation status data of the charging stations, model each charging station as a charging station agent, and determine the state space, action space and reward function of each charging station agent.

[0028] Specifically, the system creates an independent agent instance for each charging station i. This agent is equipped with an Actor network and an Critic network, enabling it to make autonomous decisions in partially observable environments. The Actor network outputs actions based on local observation vectors, while the Critic network, during the training phase, takes the joint observation vector and joint actions of all agents as input and outputs a Q-value to evaluate the quality of the joint actions. In the state space design, the system defines the charging station agent's observation information as a comprehensive state vector containing the station's load status, queue length, current price, grid electricity price, and time characteristics, allowing the agent to fully perceive key environmental factors influencing charging guidance decisions. In the action space construction, the system designs the charging station agent's behavioral output as continuous control variables for price adjustments and guidance incentives, allowing the agent to influence user choices by dynamically adjusting charging prices and providing user incentives. In designing the reward function, the system establishes a multi-objective reward mechanism that balances revenue maximization, load balancing, service quality, and grid stability. Revenue rewards incentivize charging stations to increase revenue, load balancing rewards guide them to maintain appropriate load levels, and queuing and grid overload penalties constrain them to avoid excessive congestion and grid shocks. Through this systematic agent modeling approach, each charging station possesses the ability to make autonomous decisions and continuously learn in complex environments, enabling it to promote the coordinated optimization of the entire charging network while pursuing its own interests.

[0029] Step 103: Based on user demand characteristic data and behavioral data, model charging users as user intelligent agents, and configure the selection preference model of each user intelligent agent based on behavioral data.

[0030] Specifically, the system analyzes users' historical charging selection records to identify the main decision-making factors influencing their charging station choices, including key variables such as charging price, driving distance, waiting time, and incentive offers. Based on stochastic utility theory, the system constructs a choice preference model for the user agent, modeling user selection behavior as a utility-maximizing decision-making process. The user utility function comprehensively considers the weights of each decision factor's impact on user satisfaction. The system uses historical selection data to calibrate the parameters of the user preference model and estimates the sensitivity coefficients of different user groups to factors such as price, distance, and time through statistical analysis methods, ensuring that the model accurately reflects users' true preference characteristics. Building on this, the system establishes a probabilistic user selection mechanism, converting users' utility evaluations of each charging station into a selection probability distribution, enabling the user agent to simulate the random selection behavior of real users. The system also implements an online user preference update function, dynamically adjusting the preference model parameters by continuously collecting new user selection data to maintain the model's accuracy and adaptability. By using this user agent modeling method based on behavioral data, the system can accurately predict users' responses to adjustments in charging station strategies, providing reliable user behavior feedback for optimizing charging guidance strategies, ensuring that the guidance strategies are accepted by users and achieve the expected load diversion effect.

[0031] Step 104: Based on the interaction between the charging station agent and the user agent, construct a multi-agent game model.

[0032] Specifically, the system first defines the interaction mechanism between agents. Charging station agents influence user agents' choices by adjusting prices and incentive strategies. User agents, based on their preference models, respond to the charging station strategies and make charging station choices. The user's choice, in turn, affects the charging station's load status and revenue, thus forming a complete policy interaction loop. The system models the entire interaction process as a partially observable stochastic game, where each charging station agent can only observe local environmental information and needs to make policy decisions and learn optimizations under incomplete information. In the technical implementation of the game model, the system uses a multi-agent deep deterministic policy gradient algorithm as the solution framework. Through a learning architecture of centralized training and distributed execution, each charging station agent can utilize global information for policy learning during the training phase, while only needing to make independent decisions based on local observation information during the execution phase. The system designs stability mechanisms such as experience replay and target network update to ensure the convergence of the multi-agent learning process and the effectiveness of the policies. By constructing this multi-agent game model, the system achieves coordinated optimization among charging stations and accurate modeling of user choices, enabling the entire charging network to gradually reach an equilibrium state through distributed learning. This satisfies users' personalized charging needs while achieving dynamic balance of network load and continuous improvement of overall system efficiency.

[0033] As an optional embodiment, the step of determining the state space, action space, and reward function of each charging station agent may further include the following steps: Step 201: Based on the facility configuration data and operation status data of the charging station, determine the state space of each charging station agent. The state space includes the current load status, queuing status, price information, and time information.

[0034] Specifically, the system designs the observation vector of the charging station agent i as si(t) = [Li(t), Qi(t), Pi(t), Pgrid(t), τ(t)], where Li(t) represents the current load state, calculated by the ratio of the current charging power Pi_current(t) to the total installed capacity Pi_max, with a value range of [0,1]. Qi(t) is the current queue length, which is counted in real time by vehicle detection sensors to determine the number of vehicles waiting to charge. Pi(t) is the current service price, reflecting the charging station's pricing strategy. Pgrid(t) is the real-time electricity price, affecting the charging station's operating costs. τ(t) is the time context feature vector, containing time information such as hour and day of the week, used to predict changes in user demand. Through this state space design, the charging station agent can perceive key environmental factors affecting user choices and its own revenue, providing a complete environmental information foundation for policy learning.

[0035] Step 202: Based on the control capabilities of the charging station, determine the action space of each charging station's intelligent agent. The action space includes price adjustment actions and guidance incentive actions.

[0036] Specifically, the system defines the action of the charging station agent i as ai(t) = [ΔPi(t), Ii(t)], where ΔPi(t) represents the price adjustment relative to the base price, continuously ranging from [-0.3 to +0.5 yuan per kilowatt-hour]. Negative values ​​indicate price reductions to attract users, while positive values ​​indicate price increases to divert users. Ii(t) represents the intensity of user-oriented incentives, continuously ranging from [0 to 10 yuan] in coupon amounts. Higher incentives are provided to attract users when the charging station load is low, and lower incentives are provided to divert users when the load is high. The system calculates the actual service price using Pi(t) = Pbase + ΔPi(t), where Pbase is the base service price, and designs diverse incentive forms including cash coupons and points rewards. Through this continuous action space design, the charging station agent can flexibly adjust its strategy according to environmental conditions, achieving precise control of user traffic.

[0037] Step 203: Based on the optimization objectives of charging guidance, construct the reward function for each charging station agent. The reward function includes load balancing reward, revenue reward, and service quality penalty.

[0038] Specifically, the system constructs a multi-objective weighted reward mechanism: Ri(t) = ω1·Revi(t) + ω2·Ei(t) -ω3·Qi(t)² - ω4·Pnl(t), where Revi(t) is the revenue reward term, calculated as the product of service price and charging volume minus cost. Ei(t) is the load balancing reward term, calculated using 1 - |Li(t) - L*|, where L* is the target load rate of 0.8. Qi(t)² is a secondary penalty term for queue length, strengthening the suppression of severe congestion. Pnl(t) is a grid overload penalty term, ensuring grid safety. The weighting coefficients ω1=1.0, ω2=0.5, ω3=0.1, ω4=2.0 are determined through training with historical data. Through this multi-objective reward function design, the charging station agent can pursue its own revenue while also considering load balancing and service quality, achieving coordinated optimization of the charging network.

[0039] As an optional embodiment, the step of configuring the selection preference model for each user agent based on behavioral data may further include the following steps: Step 301: Extract users' historical selection records from behavioral data and analyze the factors influencing users' choice of charging stations.

[0040] Specifically, the system extracts user j's historical charging station selection sequence through user APP usage records, charging order data, and location trajectory data, and records the environmental status information at the time of each selection, including the price P of the available charging stations. i Distance D ji Expected waiting time W ji and current battery level S j The system uses data mining techniques to analyze users' decision-making patterns when faced with multiple charging station options, identifying key influencing factors such as price sensitivity, distance convenience, time efficiency, and urgency of battery usage. Statistical analysis reveals significant differences in the importance different users place on each influencing factor, laying a data foundation for building a personalized user preference model.

[0041] Step 302: Construct a user utility function based on influencing factors, including distance factors, price factors, waiting time factors, and incentive factors.

[0042] Specifically, the system designs the utility function of user j with respect to charging station i as U ji = θ jp ·P i +θ jd ·Dji + θ jw ·W ji + θ js ·S j + ε ji , where θ jp θ represents the price sensitivity coefficient, reflecting how sensitive users are to charging prices. jd θ represents the distance sensitivity coefficient, reflecting the user's level of attention to driving distance. jw θ is the waiting time sensitivity coefficient, representing the user's tolerance for queuing. js ε is the power status sensitivity coefficient, reflecting the degree to which the current power level affects the selection. ji This is a random error term that captures random factors that the model cannot explain. Through this linear utility function design, the system can quantify the degree of influence of each factor on the user's choice decision.

[0043] Step 303: Based on historical selection records, calibrate the weight parameters of each influencing factor in the utility function.

[0044] Specifically, the system employs the maximum likelihood estimation method to estimate the personalized sensitivity parameter θ using users' historical selection data. j = {θ jp , θ jd , θ jw , θ js The system constructs a likelihood function and evaluates the rationality of the parameters by observing the degree of matching between the user's actual choices in historical scenarios and the model's predicted choices. For each user j, the system solves for the parameter combination that maximizes the likelihood function, obtaining the user's personalized sensitivity coefficient. To improve the stability of parameter estimation, the system also introduces a regularization term to prevent overfitting and ensures the generalization ability of the parameters through cross-validation. Through this parameter calibration process, the system establishes utility function parameters that reflect the individual preference characteristics of each user.

[0045] Step 304: Establish a charging station selection probability model for users based on the utility function, which serves as the selection preference model for each user agent.

[0046] Specifically, the system uses the Softmax function to represent the probability that user j chooses charging station i as follows: U jiLet $k$ be the utility value of user $j$ for charging station $i$, and the summation term covers all available charging stations $k$. This probabilistic model ensures that users tend to choose charging stations with higher utility while maintaining the randomness of selection to reflect the uncertainty of real-world decision-making. The system integrates this probabilistic model into the decision-making logic of the user agent. When the user agent needs to select a charging station, it calculates the utility value and selection probability of each charging station based on the current state, and then makes a random selection according to the probability distribution. Through this probabilistic selection mechanism based on the utility function, the user agent can simulate the selection behavior patterns of real users, providing a reliable simulation basis for evaluating the effectiveness of charging guidance strategies.

[0047] Step 20: Extract the state data of each charging station agent from the operation data, and generate the state feature vector of each charging station agent based on the state data.

[0048] Specifically, the system collects real-time operational status data of charging station i through the charging station's monitoring system, power grid interface, and business management platform, and constructs this data into a state feature vector, namely the aforementioned observation vector oi = [Li, Qi, Pi, Ce, T]. Here, Li is the station's load state vector, including two dimensions: current load rate and average vehicle charging amount. The current load rate is calculated using Li_rate = Pi_current / Pi_max, and the average vehicle charging amount is determined by historical statistical data to reflect typical charging demand. Qi is the station's queue length, calculated using vehicle detection sensors and the reservation system to count the number of vehicles currently waiting to charge. Pi is the station's current service price, reflecting the charging station's real-time pricing strategy. Ce is the real-time electricity price from the power grid, obtained from the power trading platform for the current electricity procurement cost. T is the time context feature vector, containing time-dimensional information such as hour, date type, and season, used to capture the temporal patterns of user demand. Through this state feature vector design, the charging station's intelligent agent can comprehensively perceive key environmental factors influencing user choices and its own operation.

[0049] Step 30: Input the state feature vector into the preset policy network for decision calculation to obtain the control instructions of each charging station agent.

[0050] Specifically, the system employs a deep neural network to construct a policy network π(ai|oi; θ). This network takes the state feature vector oi = [Li, Qi, Pi, Ce, T] of the charging station agent i as input and outputs the corresponding control command ai. The policy network adopts a three-layer fully connected structure. The input layer has a dimension of 5 (corresponding to the 5 dimensions of the state vector), the hidden layer uses 128 neurons and uses the ReLU activation function, and the output layer generates continuous control commands, including price adjustment amount and guiding incentive strength. Through a forward propagation calculation process, the system inputs the standardized state vector into the network. After nonlinear transformation in the hidden layer, the tanh activation function is used in the output layer to limit the price adjustment amount to the range of [-0.3, +0.5] yuan / kWh, and the sigmoid activation function is used to map the guiding incentive strength to the coupon amount range of [0, 10] yuan. The final control command ai = [ΔPi, Ii] represents the adjustment amount relative to the benchmark price and the incentive strength for users, respectively, providing intelligent decision support for the real-time control of the charging station.

[0051] As an optional embodiment, the step of inputting the state feature vector into a preset policy network for decision calculation to obtain the control instructions for each charging station agent may further include the following steps: Step 401: Encode the state feature vectors of each charging station agent.

[0052] Specifically, the system performs feature encoding on the state feature vectors of each charging station agent to enable the policy network to effectively handle different types of state information. The system standardizes and encodes each dimension of the state vector oi = [Li, Qi, Pi, Ce, T]. For the load state vector Li, the system performs min-max normalization on the load rate and average charging amount to the [0,1] interval. For the queue length Qi, a logarithmic transformation log(1+Qi) is used followed by standardization to reduce the influence of extreme values. For price information Pi and grid electricity price Ce, Z-score normalization is used to eliminate the influence of dimensions. For the time feature vector T, the system uses a periodic encoding method, encoding hourly information as [sin(2πh / 24), cos(2πh / 24)], weekday information as a one-hot vector, and seasonal information as sequential encoding. After feature encoding, the system obtains a dimension-expanded state representation vector o'i, which retains the semantic information of the original features while facilitating neural network processing.

[0053] Step 402: Input the encoded state feature vector into the preset policy network, and perform forward propagation calculation through the neural network layer of the policy network to obtain the action probability distribution of each charging station agent.

[0054] Specifically, the policy network adopts an Actor network architecture, consisting of an input layer, two hidden layers, and an output layer. The input layer receives the encoded state vector o'i. The first hidden layer contains 256 neurons and uses the ReLU activation function, while the second hidden layer contains 128 neurons, also using ReLU activation. The output layer is divided into two branches, corresponding to the price adjustment action and the incentive action, respectively. The price adjustment branch uses the tanh activation function to output the mean μp and standard deviation σp, constructing a Gaussian distribution N(μp,σp²) to represent the probability distribution of the price adjustment action. The incentive action branch uses the sigmoid activation function to output the parameters α and β of the beta distribution, constructing a Beta(α, β) distribution to represent the probability distribution of the incentive intensity. During the forward propagation process, the hidden layer outputs are calculated using h1 = ReLU(W1o'i + b1) and h2 = ReLU(W2h1 + b2), ultimately obtaining the action probability distribution π(ai|oi).

[0055] Step 403: Determine the control actions of each charging station's intelligent agent based on the action probability distribution.

[0056] Specifically, the system samples the price adjustment amount ΔPi from the Gaussian distribution N(μp, σp²) of the price adjustment action. The sampling process is ΔPi = μp + σp × ε, where ε ~ N(0,1) are random numbers from a standard normal distribution. The system truncates the sampling results to ensure that ΔPi ∈ [-0.3, +0.5] yuan / kWh. For the guiding incentive action, the system samples the incentive intensity ratio ri from the Beta distribution Beta(α, β), and then calculates the specific incentive amount using Ii = ri × 10, ensuring that Ii ∈ [0, 10] yuan. To balance exploration and utilization, the system uses a random sampling strategy to enhance exploration during the training phase, and can choose a deterministic strategy during the execution phase to directly use the expected value of the distribution as the action output. The final determined control action is ai = [ΔPi, Ii], providing a decision basis for subsequent instruction generation.

[0057] Step 404: Generate corresponding control instructions based on the control actions. The control instructions include price adjustment instructions and incentive instructions.

[0058] Specifically, the system converts the control actions ai = [ΔPi, Ii] into executable control instructions. For price adjustment instructions, the system calculates the new service price Pi_new = Pbase + ΔPi and generates a price update instruction, which is sent to the charging station's billing system. The instruction format is {"station_id": i, "action_type": "price_update", "new_price": Pi_new, "effective_time": t+1}. For incentive instructions, the system generates user guidance strategies based on the incentive amount Ii, including coupon issuance instructions and push message instructions. The coupon instruction format is {"station_id": i, "action_type": "incentive", "coupon_value": Ii, "target_users": "nearby_users", "validity_period": 30}. The system also generates APP push messages to send charging discount information to nearby users to achieve the guidance effect. All control instructions are distributed to the corresponding execution modules through a unified instruction distribution system to ensure that the control strategies can be implemented in a timely and effective manner, achieving intelligent control of charging station load and user traffic.

[0059] Step 40: Generate guidance information based on the control instructions, simulate the response of each user agent to the guidance information based on the user agent's selection preference model, and obtain the user distribution prediction results.

[0060] Specifically, the system first converts the charging station agent's control commands ai = [ΔPi, Ii] into user-oriented guidance information. For price adjustment commands, the system updates the service price of charging station i, Pi(t+1) = Pbase + ΔPi, and pushes price change information in real time via the APP. For guidance incentive commands, the system generates coupon information and sends guidance messages to users within the target area. Next, the system uses the user agent's choice preference model to simulate user responses. For user j, the system calculates the utility value for each charging station based on its utility function Uji = θjp·Pi + θjd·Dji + θjw·Wji + θjs·Sj, where Pi has been updated to the adjusted price, and the incentive information Ii is reflected in the utility calculation as a price discount. Then, the system uses the Softmax probability model Pji = exp(Uji) / ∑k exp(Ujk) to calculate the probability of user j choosing each charging station. Finally, the system summarizes the selection probabilities of all user agents and predicts the user traffic distribution of each charging station after regulation, Ni_pred(t+1) = ∑j Pji × Nj_total, providing a predictive basis for evaluating the effectiveness of the charging guidance strategy.

[0061] Step 50: Collect feedback data based on the user distribution prediction results and actual execution effects, calculate the performance evaluation value based on the feedback data, and update the network parameters of the strategy network of each charging station agent based on the performance evaluation value.

[0062] Specifically, the system collects feedback information after the execution of control commands through real-time monitoring systems of charging stations, user APP usage data, and power grid operation data. This includes key indicators such as the actual number of users arriving at each charging station, load changes, user waiting time, and charging station operating revenue. The system compares and analyzes the predicted user distribution results with the actual observed user traffic to evaluate the accuracy of the prediction and the control effect. Based on the feedback data, the system calculates a comprehensive performance evaluation value for each charging station agent, which comprehensively considers multiple dimensions such as charging station revenue performance, load balancing degree, service quality, and power grid security. The system adopts a policy optimization algorithm in deep reinforcement learning, calculates the policy gradient based on the performance evaluation value, and uses the backpropagation algorithm to update the weight parameters of the policy network. During the update process, the system uses an experience replay mechanism to store historical experience data and gradually optimizes the network parameters through a mini-batch stochastic gradient descent method, enabling the charging station agent to continuously improve its decision-making strategy based on the actual execution effect and improve the overall effect of charging guidance.

[0063] As an optional embodiment, the step of collecting feedback data based on user distribution prediction results and actual execution effects, and calculating performance evaluation values ​​based on the feedback data, may further include the following steps: Step 501: Based on the user distribution prediction results, obtain the expected state data of each charging station's intelligent agent.

[0064] Specifically, based on the prediction results of the user agent's selection probability model, the system calculates the expected user traffic for each charging station in the next time period: Ni_expected(t+1) = ∑j Pji × Nj_total, where Pji is the probability of user j choosing charging station i, and Nj_total is the total number of users. Based on the expected traffic, the system calculates the expected load rate of the charging station: Li_expected(t+1) = (Ni_expected(t+1) × Pavg) / Pmax_i, where Pavg is the average charging power, and Pmax_i is the maximum power capacity of charging station i. The system also calculates the expected queue length: Qi_expected(t+1) = max(0, Ni_expected(t+1) - Ci), where Ci is the service capacity of charging station i. Expected returns are calculated as Revi_expected(t+1) = Pi(t+1) × Ei_expected(t+1) - Ccost × Ei_expected(t+1), where Pi(t+1) is the price after adjustment, Ei_expected is the expected charging amount, and Ccost is the electricity cost. These expected state data provide a theoretical benchmark for subsequent comparisons with actual results.

[0065] Step 502: After executing the control command at each charging station, collect the actual operating data of each charging station. The actual operating data includes the actual user distribution, load status and service quality indicators.

[0066] Specifically, the system collects data on the actual number of arriving users (Ni_actual(t+1)), the actual load rate (Li_actual(t+1) = Pi_actual(t+1) / Pmax_i), and the actual queue length (Qi_actual(t+1)) through charging station monitoring equipment. Service quality indicators include the average waiting time (Wi_actual(t+1) = ∑k(tk_start - tk_arrive) / Ni_actual), where tk_arrive is the arrival time of user k and tk_start is the start time of charging. The system also collects actual revenue data (Revi_actual(t+1) = Pi(t+1)×Ei_actual(t+1)-Ccost×Ei_actual(t+1)- Ii×Ni_actual(t+1), where Ei_actual is the actual charging amount and Ii is the incentive expenditure. Through data fusion from the sensor network and the business system, the system obtains complete and accurate information on the actual operating status.

[0067] Step 503: Compare the expected state data with the actual operating data to generate feedback data.

[0068] Specifically, the system calculates the prediction errors of key indicators: user traffic prediction error is ΔNi = Ni_actual(t+1) - Ni_expected(t+1), load prediction error is ΔLi = Li_actual(t+1) - Li_expected(t+1), and revenue prediction error is ΔRevi = Revi_actual(t+1) - Revi_expected(t+1). The system calculates the relative error rates: user traffic relative error is εN_i = |ΔNi| / max(1, Ni_expected(t+1)), and load relative error is εL_i = |ΔLi| / max(0.1, Li_expected(t+1)). The feedback data vector is constructed as feedback_i = [ΔNi, ΔLi, ΔRevi, εN_i, εL_i, Wi_actual(t+1)], which contains the absolute and relative values ​​of the prediction deviations, as well as the actual performance of service quality, providing comprehensive feedback information for strategy learning.

[0069] Step 504: Based on the feedback data, calculate the instant reward value according to the reward function of each charging station agent.

[0070] The specific system substitutes the actual operating data into the reward function Ri(t+1)=ω1·Revi_actual(t+1) +ω2·(1-|Li_actual(t+1)-L*|)-ω3·Qi_actual(t+1) ² The instant reward is calculated using -ω4·Pnl(t+1), where L*=0.8 is the target load rate, and Pnl(t+1) is the grid overload penalty. The weights for the reward item are ω1=1.0, load balancing item ω2=0.5, queuing penalty item ω3=0.1, and overload penalty item ω4=2.0. When Li_actual(t+1)>0.95, the overload penalty Pnl(t+1) = 10 is triggered. The system also introduces a prediction accuracy reward item Racc_i = -0.2×(εN_i+εL_i) to encourage the agent to improve prediction accuracy. The final instant reward is ri(t+1)=Ri(t+1) + Racc_i, which comprehensively reflects the charging station's operational performance and prediction capability.

[0071] Step 505: Accumulate and calculate the instant reward values ​​to obtain the performance evaluation value of each charging station agent.

[0072] Specifically, the system uses cumulative discount rewards to calculate performance evaluation values. Where γ=0.95 is the discount factor, and t is the time window length. To eliminate scale differences between different charging stations, the system standardizes the cumulative reward: G' i =(G i -μG) / σG, where μG and σG are the mean and standard deviation of the cumulative rewards for all charging stations, respectively. The system also calculates the moving average performance evaluation value: Ḡi(t)=αḠi(t-1)+(1-α)G' i Where α = 0.9 is the smoothing coefficient. The final performance evaluation value Vi = Ḡi(t) serves as the target value for policy network updates; a positive value indicates that the policy performance is good and should be strengthened, while a negative value indicates that the optimization direction needs to be adjusted. Through this accumulation and standardization mechanism, the system can fairly evaluate the policy performance of different charging station agents.

[0073] As an optional embodiment, the step of updating the network parameters of the policy network of each charging station agent based on the performance evaluation value may further include the following steps: Step 601: Construct training samples from the state feature vectors, control actions, and performance evaluation values ​​of each charging station agent.

[0074] Specifically, the system undergoes iterative training in a simulated environment. Each agent interacts with the environment according to its current policy, storing the data (o, a, R, o') in an experience replay pool. Here, o represents the current observed state, including feature vectors such as the charging station's load state, queue length, price information, and grid state; a represents the regulatory actions performed by the agent, including policy actions such as price adjustments and incentive guidance; R represents the reward signal calculated based on performance evaluation values, reflecting the policy's effectiveness; and o' represents the next state observation after the action. The system organizes these interaction data into training samples according to time series. Each sample contains complete state-action-reward-next state transition information, providing a high-quality training data foundation for deep reinforcement learning algorithms.

[0075] Step 602: Store the training samples in the experience replay pool and sample batch training data from the experience replay pool.

[0076] Specifically, the system samples small batches of data from the experience replay pool to update the Critic network. The system randomly samples training samples with a batch size of 1024 from the stored experience data. The loss function L of the Critic network is: The system samples small batches of data from the experience replay pool to update the Critic network. In specific implementation, the system randomly samples training samples with a batch size of 1024 from the stored experience data. The loss function L of the Critic network is: ,in, , Generated by the target Actor network, representing the target Actor network in state. The action output below, where γ is the discount factor. These represent the parameters of the Critic network; This represents the current Critic network's value estimate for the state-action pair (o,a); y represents the target value. For instant rewards; For the target Critic network; For the next state, The parameters of the target Critic network are defined here. The system trains the Critic network using the mean squared error loss function and optimizes the network parameters by minimizing the difference between the predicted Q-value and the target Q-value.

[0077] Step 603: Input the batch training data into the policy network of each charging station agent, and calculate the policy gradient based on the performance evaluation value.

[0078] Specifically, the system updates the Actor network, and its policy gradient is: The system optimizes the Actor network parameters using the policy gradient method. In the policy gradient formula above, Describe the objective function The gradient relative to the policy parameters; Represents the policy parameters of the Actor network; Indicates in the parameter The lower policy network is based on observations The output action; This indicates that the policy network outputs actions relative to the network parameters. The gradient; The Q-value function of the Critic network relative to the action The gradient; This represents the expected value for all training samples. The learning rate of the Actor network is set to lr_actor=0.0001, and the parameters are updated using the Adam optimizer. The system employs a deterministic policy gradient algorithm, optimizing the policy network by maximizing the expected cumulative reward.

[0079] Step 604: Based on the policy gradient, update the network parameters of the policy network of each charging station agent using the gradient ascent algorithm.

[0080] Specifically, in practical implementation, the system is based on the calculated policy gradient. The Actor network parameters are updated using the gradient ascent method. The parameter update formula is as follows: .

[0081] in, This represents the updated policy network parameters; Indicates the current policy network parameters; This represents the learning rate, set to... ; This indicates that the policy objective function is relative to the parameters. The system uses the Adam optimizer for parameter updates, which adaptively adjusts the learning rate and provides a momentum mechanism. To prevent gradient explosion, the system sets a gradient pruning threshold of 0.5; pruning is performed when the gradient norm exceeds this threshold. Simultaneously, the system employs a soft update mechanism for the target network, with the update formula as follows: .in, For the target network parameters, These are soft update coefficients. Through this gradual update method, the system ensures the stability and convergence of the policy network training, enabling continuous optimization of the decision-making capabilities of each charging station agent.

[0082] Please see Figure 2 This document presents a MADDPG (Multi-Agent Deep Reinforcement Learning) architecture diagram, provided as an embodiment of this application. The diagram illustrates a complete framework for centralized training and distributed execution, with the entire system divided into two core parts: an environment interaction layer and an agent network layer. In the environment interaction layer, the environment simulator acts as a central coordinator, receiving individual observations from each charging station agent. and corresponding actions Environmental state transitions are performed based on the global state S, and empirical data is incorporated. Stored in the experience replay pool. At the agent network layer, each charging station agent is equipped with an independent Actor network. and Critic Network The Actor network generates joint action policies based on individual observations, while the Critic network performs value evaluation based on global information and joint actions. The central training engine samples data from the experience replay pool and simultaneously optimizes the network parameters of all agents through a parameter updater, where the policy gradient... The strategy optimization of the Actor network is guided to minimize the loss L, thereby approximating the value function of the Critic network. This architecture design realizes global information sharing during the training phase and distributed decision-making during the execution phase, effectively solving the non-stationarity problem in multi-agent environments and providing a stable and efficient learning framework for intelligent regulation of charging stations.

[0083] Step 60: Deploy the optimized network parameters and policy network to each charging station so that each charging station can independently execute charging guidance.

[0084] Specifically, the system deploys the optimized network parameters and policy network to each charging station, enabling each charging station to independently execute charging guidance. In practice, after training, the system will finalize the actor networks... Deployed to the corresponding charging stations. This represents the policy network specific to charging station i, containing network parameters optimized through deep reinforcement learning training. The system first exports the trained Actor network parameters from the cloud training environment to generate a lightweight inference model file. For each charging station i, the system extracts the corresponding policy network. The system assigns parameter weights, including the weight matrix and bias vector of the neural network. It employs model quantization and compression techniques to optimize the network parameters into a format suitable for edge device deployment, ensuring efficient operation on the local hardware of the charging station. The system uses secure network protocols to connect each charging station's dedicated Actor network. The data is pushed to the corresponding edge computing device at the charging station. Each charging station receives and verifies the integrity of the network parameters, and then... Loaded into the local inference engine. In actual operation, each station only needs to rely on local observations. Utilizing the local Actor network in, It includes real-time status information for charging station i, such as current load, queuing status, and electricity price information, as well as other locally observed data. After deployment, charging station i uses local observations... Through policy network The system calculates and outputs control actions in real time, including dynamic pricing strategies and charging guidance suggestions, enabling fully distributed autonomous decision-making capabilities and ensuring normal operation even without network connectivity or in the event of a central server failure.

[0085] Please see Figure 3 This diagram illustrates an exemplary multi-agent-based charging guidance method provided in this application. The flowchart fully demonstrates the entire process from system initialization to agent deployment. First, the system initializes, establishing an interaction framework between the environment and the agents. When a charging request arrives, the system determines whether the MADDPG training loop has been completed. If not, it enters the training phase: the charging station agent outputs a strategy based on observations, the user agent makes selections based on a utility function, the environment simulator calculates new states and rewards, the interaction data is stored in the experience replay pool, and finally, sampled data updates the Actor / Critic network parameters, forming a complete reinforcement learning training loop. After training, the system deploys the optimized Actor network to each charging station, enabling each station to independently execute distributed online guidance decisions. The entire process embodies the design concept of moving from centralized training to distributed execution, achieving global optimization through multi-agent collaborative learning, and ultimately forming an intelligent control system where each charging station makes autonomous decisions, effectively solving the complex optimization problems of charging station load balancing and user charging guidance.

[0086] Please see Figure 4This figure illustrates a system load rate comparison curve provided in this application embodiment. Through 24-hour continuous monitoring, the curve compares the load rate changes of four different charging guidance methods. The experimental environment simulates an urban charging network containing 50 charging stations and 5000 user requests. As can be seen from the figure, the method of this application (MARL-game, solid red line) maintains the most stable load rate curve throughout the entire time period, consistently remaining within the ideal range of 60%-70%, effectively avoiding peak overload and off-peak idleness. In contrast, the other three traditional methods exhibit significant peak-valley fluctuations: the single-agent DDPG method reaches a load rate of approximately 80% during peak hours, posing an overload risk; the fixed-price strategy experiences drastic load rate fluctuations during morning and evening peak hours, reaching nearly 95% at its peak; and the nearest neighbor strategy performs the worst, with a load rate exceeding 95% between 18:00 and 19:00, severely deviating from the optimal operating state. Experimental results show that the charging guidance method based on multi-agent reinforcement learning and user behavior game in this application achieves the lowest load balancing index (0.41) and load rate standard deviation (8.7%). Compared with the nearest neighbor strategy, it reduces the load rate control during peak hours by about 28 percentage points, significantly improving the overall efficiency and stability of the charging system, and verifying the superior performance of the proposed method in practical applications.

[0087] This application also provides a computer storage medium that can store multiple instructions. The instructions are adapted to be loaded and executed by a processor. The above-described multi-agent-based charging guidance method is described in detail in the above-described embodiments. The specific execution process can be found in the detailed description of the above-described embodiments, which will not be repeated here.

[0088] The following describes an electronic device based on multi-agent charging guidance provided by an embodiment of this application. Figure 5 This is a schematic diagram of an exemplary hardware structure of an electronic device provided in an embodiment of this application.

[0089] In some embodiments, the multi-agent-based charging-guided electronic device is a computer device or includes a computer device. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores data. The network interface of the computer device is used to communicate with other external terminals or servers via a network connection. In some embodiments, the network interface can be a wired network interface; in some embodiments, the network interface can also be a wireless network interface. When the computer program is executed by the processor, it implements the methods in the embodiments of this application.

[0090] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0091] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0092] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0093] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A charging guidance method based on multiple agents, characterized in that, The method includes: Obtain the operation data of the charging network, and construct a multi-agent game model based on the operation data. The multi-agent game model includes multiple charging station agents and multiple user agents. Extract the state data of each charging station intelligent agent from the operational data, and generate the state feature vector of each charging station intelligent agent based on the state data; The state feature vector is input into a preset policy network for decision calculation to obtain the control instructions of each charging station agent. Based on the control instructions, guidance information is generated, and based on the user agent's selection preference model, the response of each user agent to the guidance information is simulated to obtain the user distribution prediction results. Feedback data is collected based on the user distribution prediction results and actual execution effects, and performance evaluation values ​​are calculated based on the feedback data. The network parameters of the strategy network of each charging station agent are updated based on the performance evaluation values. The optimized network parameters are deployed to each charging station to enable each charging station to independently perform charging guidance.

2. The charging guidance method based on multiple agents according to claim 1, characterized in that, The construction of the multi-agent game model based on the operational data includes: Extract the facility configuration data and operational status data of the charging station, as well as the user's demand characteristics and behavioral data from the operational data; Based on the facility configuration data and operation status data of the charging stations, each charging station is modeled as a charging station intelligent agent, and the state space, action space and reward function of each charging station intelligent agent are determined. Based on the user's demand characteristics and behavior data, charging users are modeled as user intelligent agents, and selection preference models for each user intelligent agent are configured based on the behavior data. Based on the interaction relationship between the charging station agent and the user agent, the multi-agent game model is constructed.

3. The charging guidance method based on multiple agents according to claim 2, characterized in that, The determination of the state space, action space, and reward function of each charging station agent includes: Based on the facility configuration data and operation status data of the charging station, the state space of each charging station's intelligent agent is determined. The state space includes the number of charging piles, the current load status, the queuing status, and the price information. Based on the control capabilities of the charging station, the action space of each charging station's intelligent agent is determined, and the action space includes price adjustment actions and guidance and incentive actions. Based on the optimization objectives of charging guidance, a reward function is constructed for each charging station agent. The reward function includes a load balancing reward, a revenue reward, and a service quality penalty.

4. The charging guidance method based on multiple agents according to claim 2, characterized in that, The configuration of the selection preference model for each user agent based on the behavioral data includes: Extract users' historical selection records from the behavioral data and analyze the factors influencing users' choice of charging stations. A user utility function is constructed based on the aforementioned influencing factors, and the utility function includes distance factors, price factors, waiting time factors, and incentive factors. Based on the historical selection records, the weight parameters of each influencing factor in the utility function are determined. Based on the utility function, a probability model for the user's choice of charging station is established, which serves as the choice preference model for each user agent.

5. The charging guidance method based on multiple agents according to claim 1, characterized in that, The step of inputting the state feature vector into a preset policy network for decision calculation to obtain the control instructions for each charging station agent includes: Feature encoding is performed on the state feature vectors of each charging station's intelligent agent; The encoded state feature vector is input into a preset policy network, and forward propagation calculation is performed through the neural network layer of the policy network to obtain the action probability distribution of each charging station agent. Based on the aforementioned action probability distribution, determine the control actions of each charging station's intelligent agent; Based on the aforementioned regulatory actions, corresponding regulatory instructions are generated, including price adjustment instructions and incentive guidance instructions.

6. The charging guidance method based on multiple agents according to claim 1, characterized in that, The step of collecting feedback data based on the user distribution prediction results and actual execution results, and calculating performance evaluation values ​​based on the feedback data, includes: Based on the user distribution prediction results, obtain the expected state data of each charging station's intelligent agent; After each charging station executes the control command, the actual operating data of each charging station is collected. The actual operating data includes actual user distribution, load status and service quality indicators. By comparing the expected state data with the actual operating data, feedback data is generated. Based on the feedback data, the instantaneous reward value is calculated according to the reward function of each charging station agent; The instant reward values ​​are accumulated to obtain the performance evaluation value of each charging station agent.

7. The charging guidance method based on multiple agents according to claim 1, characterized in that, The network parameters of the strategy network for updating each charging station agent based on the performance evaluation value include: The state feature vectors, control actions, and performance evaluation values ​​of each charging station's intelligent agent are used to construct training samples. The training samples are stored in the experience replay pool, and batch training data is sampled from the experience replay pool; The batch training data is input into the policy network of each charging station agent, and the policy gradient is calculated based on the performance evaluation value. Based on the policy gradient, the network parameters of the policy network of each charging station agent are updated using the gradient ascent algorithm.

8. An electronic device based on multi-agent charging guidance, characterized in that, The electronic device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the electronic device to perform the method as described in any one of claims 1-7.

9. A computer program product containing instructions, characterized in that, When the computer program product is run on a multi-agent-based charging-guided electronic device, the electronic device performs the method as described in any one of claims 1-7.

10. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on a multi-agent-based charging-guided electronic device, the electronic device performs the method as described in any one of claims 1-7.

Images