V2X charging and discharging management method, device, system and program product

Abstract

The invention is suitable for the technical field of smart power grid and electric vehicle interaction, and provides a V2X charging and discharging management method, device and system and a program product, and the method comprises the steps: inputting obtained multi-source time sequence data into a time sequence prediction model, and outputting a prediction result and a prediction confidence coefficient corresponding to the prediction result; wherein the prediction result comprises a predicted travel probability in a preset time period and a predicted power grid electricity price in the preset time period; constructing a multi-dimensional state space; wherein the multi-dimensional state space comprises the prediction result, the prediction confidence coefficient, a vehicle real-time state and a power grid real-time state; the real-time state of the vehicle comprises at least one of battery electric quantity and battery health degree; the power grid real-time state comprises at least one of the following: real-time power grid electricity price, generated power and power grid frequency; and based on the multi-dimensional state space, adopting a reinforcement learning decision model to generate a V2X charging and discharging power instruction. Charging and discharging behaviors can be planned in advance, limitation of a traditional passive response mode is avoided, and risk self-adaption is achieved.

Description

Technical Field

[0001] This application belongs to the field of smart grid and electric vehicle interaction technology, and in particular relates to a V2X charging and discharging management method, device, system and program product. Background Technology

[0002] With the popularization of electric vehicles and the development of vehicle-to-grid (V2X) technology, vehicles are not only means of transportation, but are also gradually becoming distributed energy storage units capable of bidirectional energy interaction with the power grid. However, existing vehicle-to-grid (V2X) charging and discharging management systems have significant shortcomings in terms of intelligence and collaborative optimization.

[0003] Existing charging and discharging strategies are typically static and rigid, such as those based on fixed power and duration, or entirely dependent on manual settings by the user. Such strategies cannot be dynamically adjusted according to real-time changes in grid load, making them less than optimal in most cases and lacking intelligence and adaptability. Summary of the Invention

[0004] This application provides a V2X charging and discharging management method, device, system, and program product, which can solve the technical problem in the prior art that when faced with vague, colloquial, or incomplete user input queries, a large number of irrelevant results are often returned because the user's true intention cannot be accurately understood, resulting in low retrieval efficiency.

[0005] In a first aspect, embodiments of this application provide a V2X charge / discharge management method, including:

[0006] The acquired multi-source time-series data is input into the time-series prediction model, and the prediction results and the prediction confidence corresponding to the prediction results are output; wherein, the prediction results include the predicted travel probability within a preset time period and the predicted grid electricity price within a preset time period;

[0007] A multidimensional state space is constructed; wherein the multidimensional state space includes the prediction result, the prediction confidence, the real-time vehicle status, and the real-time power grid status; the real-time vehicle status includes at least one of the following: battery charge and battery health; the real-time power grid status includes at least one of the following: real-time power grid price, power generation capacity, and power grid frequency;

[0008] Based on the multidimensional state space, a reinforcement learning decision model is used to generate V2X charging and discharging power commands.

[0009] In one possible implementation of the first aspect, the reinforcement learning decision model is optimized based on a reward function, which is a weighted combination of positive rewards for electricity cost savings and grid service revenue, and negative penalties for battery loss estimation and unmet user demand.

[0010] Furthermore, the weight of the negative penalty for unmet user needs is dynamically adjusted based on the user's travel probability and the prediction confidence level. Specifically, when the predicted travel probability is greater than a first preset travel probability threshold and the prediction confidence level is greater than a first preset confidence level threshold, the weight of the negative penalty for unmet user needs is set to a first weight value. In other cases, the weight of the negative penalty for unmet user needs is set to a second weight value, where the first weight value is greater than the second weight value.

[0011] In one possible implementation of the first aspect, the reward function is as follows:

[0012] Rt=α×Recon(t)+β×Rgrid(t)-γ×Cdeg(t)-δ(t)×Punmet(t);

[0013] Where Rt is the total reward at time t; Recon(t) is the electricity cost saving benefit; Rgrid(t) is the benefit of participating in grid services; Cdeg(t) is the estimated cost of battery loss; Punmet(t) is the penalty for unmet user demand; α, β, γ are preset weight coefficients; δ(t) is the dynamically adjusted weight.

[0014] In one possible implementation of the first aspect, the dynamically adjusted weight δ(t) is calculated using the following formula:

[0015] δ(t)=δbase+κ×Ptravel(t)×Ctravel(t);

[0016] Where δbase is a preset base penalty weight, κ is a preset positive scaling factor, Ptravel(t) is the predicted travel probability at time t, and Ctravel(t) is the prediction confidence at time t.

[0017] In one possible implementation of the first aspect, the method further includes:

[0018] Based on each V2X unit, a local reinforcement learning decision model is trained using local data.

[0019] The model parameter updates generated during training are uploaded to the server, so that the server can aggregate the model parameter updates received from multiple V2X units to generate a global model.

[0020] Receive the global model sent by the server to update its local reinforcement learning decision model.

[0021] In one possible implementation of the first aspect, the step of uploading the updated model parameters generated during training to the server further includes:

[0022] Each V2X unit evaluates the performance of its trained reinforcement learning decision model using a pre-defined validation dataset and obtains a performance score.

[0023] Uploading the updated model parameters generated during training to the server includes:

[0024] The model parameter updates generated during training and the performance score are uploaded to the server, so that the server calculates the aggregation weight for the corresponding model parameter updates based on the performance score, and performs weighted aggregation on the received model parameter updates of multiple V2X units based on the aggregation weight to generate the global model.

[0025] Secondly, embodiments of this application provide a V2X charge / discharge management device, comprising:

[0026] The input module is used to input the acquired multi-source time series data into the time series prediction model and output the prediction results and the prediction confidence corresponding to the prediction results; wherein, the prediction results include the predicted travel probability within a preset time period and the predicted grid electricity price within a preset time period;

[0027] A construction module is used to construct a multi-dimensional state space; wherein, the multi-dimensional state space includes the prediction result, the prediction confidence, the real-time vehicle status, and the real-time power grid status; the real-time vehicle status includes at least one of the following: battery charge and battery health; the real-time power grid status includes at least one of the following: real-time power grid price, power generation capacity, and power grid frequency;

[0028] The generation module is used to generate V2X charging and discharging power commands based on the multidimensional state space and using a reinforcement learning decision model.

[0029] Thirdly, embodiments of this application provide a V2X charging and discharging management system, including: a server and multiple V2X units; the V2X unit includes an edge computing gateway;

[0030] The edge computing gateway of the V2X unit is used to acquire multi-source time-series data and generate prediction results and corresponding prediction confidence levels through a time-series prediction model; construct a multi-dimensional state space, and generate V2X charging and discharging power commands based on the multi-dimensional state space using a reinforcement learning decision model; wherein, the prediction results include the predicted travel probability and the predicted grid electricity price within a preset time period; the multi-dimensional state space includes the prediction results, the prediction confidence levels, the real-time vehicle status, and the real-time grid status; the real-time vehicle status includes at least one of the following: battery charge and battery health; the real-time grid status includes at least one of the following: real-time grid electricity price, power generation, and grid frequency;

[0031] The server is used to collaboratively train the reinforcement learning decision models of multiple V2X units; the collaborative training includes: periodically receiving model parameter updates uploaded from multiple V2X units, aggregating the received model parameter updates from multiple V2X units to generate a global model, and distributing the global model to multiple V2X units to update their local reinforcement learning decision models.

[0032] Fourthly, embodiments of this application provide a control device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the V2X charge / discharge management method described in any one of the first aspects above.

[0033] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the V2X charge / discharge management method described in any one of the first aspects.

[0034] Sixthly, embodiments of this application provide a computer program product that, when run on a control device, causes the control device to execute the V2X charge / discharge management method described in any of the first aspects above.

[0035] In this embodiment, multi-source time-series data is input into a time-series prediction model, which outputs prediction results and corresponding prediction confidence levels. The prediction results include the predicted travel probability and the predicted grid electricity price within a preset time period. A multi-dimensional state space is constructed, comprising the prediction results, the prediction confidence levels, the vehicle's real-time status, and the grid's real-time status. The vehicle's real-time status includes at least one of the following: battery charge and battery health. The grid's real-time status includes at least one of the following: real-time grid electricity price, power generation, and grid frequency. Based on the multi-dimensional state space, a reinforcement learning decision model is used to generate V2X charging and discharging power commands. By predicting user travel probability and electricity prices using the time-series prediction model, charging and discharging behavior can be planned in advance, avoiding the limitations of traditional passive response modes. Furthermore, by incorporating the prediction results and confidence levels into the decision-making process, the charging and discharging strategy can intelligently balance economic benefits and user needs based on the reliability of the prediction, achieving risk adaptation.

[0036] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of the structure of a V2X charge and discharge management system provided in an embodiment of this application;

[0039] Figure 2 This is a schematic flowchart of a V2X charge and discharge management method provided in an embodiment of this application;

[0040] Figure 3 This is a schematic flowchart of a V2X charge and discharge management method provided in an embodiment of this application;

[0041] Figure 4 This is a schematic diagram of the structure of a V2X charge and discharge management device provided in an embodiment of this application;

[0042] Figure 5 This is a schematic diagram of the structure of a control device provided in an embodiment of this application. Detailed Implementation

[0043] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0044] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0045] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0046] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0047] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0048] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0049] This embodiment provides a specific implementation of a V2X charge and discharge management method and system. Figure 1 This is a schematic diagram of a V2X charge and discharge management system provided in an embodiment of this application. Figure 2 This is a flowchart illustrating a V2X charge / discharge management method provided in an embodiment of this application.

[0050] Reference Figure 1 In this embodiment, the V2X charging and discharging management system can be divided into cloud-side, edge-side, and device-side in terms of physical and functional division. The cloud-side deploys servers, serving as a regional scheduling and model training center. The edge-side contains one or more V2X units, typically deployed on the user side, such as in home garages or office building parking lots. Each V2X unit structurally includes an edge computing gateway and a charging pile with which it communicates. The device-side refers to the vehicles connected to the charging piles for charging and discharging.

[0051] Understandably, the server and each edge computing gateway exchange data bidirectionally via network connections (such as WAN communication based on transport layer security protocols), primarily for distributing the global model and uploading local model update parameters. The edge computing gateways communicate with charging piles via local wired or wireless networks for high-speed, low-latency communication. Correspondingly, the charging piles interact with the vehicle's battery management system using standard charging communication protocols.

[0052] like Figure 2 As shown, the V2X charge / discharge management method executed by this system includes the following steps:

[0053] S201, the acquired multi-source time series data is input into the time series prediction model, and the prediction results and the prediction confidence corresponding to the prediction results are output; wherein, the prediction results include the predicted travel probability within a preset time period and the predicted grid electricity price within a preset time period.

[0054] This step is primarily performed on the edge computing gateway. Specifically, as follows: Figure 3 The schematic flowchart of the V2X charge and discharge management method shown shows that the edge computing gateway integrates a timing prediction model, which is responsible for processing multi-source timing data obtained from multiple data sources.

[0055] The multi-source time-series data includes, but is not limited to: 1) user historical behavior data, such as data that reflects user driving habits by recording vehicle charging start-stop time, daily mileage, historical driving trajectory (with user authorization), and initial battery charge during charging; 2) energy market data, such as historical time-of-use electricity prices, real-time electricity prices, electricity price forecasts for the next 24 hours, and ancillary service demand signals such as peak shaving and frequency regulation issued by the power grid, obtained from grid operators or energy service providers; 3) environmental data, such as local weather forecast data, especially information related to local photovoltaic power generation, such as light intensity and temperature; 4) vehicle static data, such as the vehicle's total battery capacity and rated charging and discharging power.

[0056] Optionally, after acquiring the above raw data, the edge computing gateway preprocesses it. This process includes data cleaning (to handle missing and outlier values), data alignment (to unify data with different sampling frequencies to the same time step), data normalization (e.g., using min-max scaling to scale all feature values ​​to the [0,1] interval to eliminate the influence of units and accelerate model convergence), and feature engineering (e.g., extracting periodic features such as day of the week and whether it is a holiday from the timestamp).

[0057] In this embodiment, the time-series prediction model can employ a model structure based on a Long Short-Term Memory (LSTM) network. This network, due to its unique gating mechanism (input gate, forget gate, output gate), is particularly adept at capturing and learning long-term dependencies in time-series data. As an optional implementation, a bidirectional LSM network can be used, which can learn information from both past and future time steps simultaneously, thus providing richer context for predictions at the current time point. After training, the time-series prediction model can receive preprocessed time-series data as input and output two key prediction results: first, the predicted travel probability within a preset future time period (e.g., the next 24 hours), denoted as Ptravel(t), which represents the likelihood that a user will need to use a vehicle for travel at any future time point t; and second, the predicted grid electricity price within the same preset future time period, denoted as Lprice(t).

[0058] It should be noted that, in addition to the prediction result itself, the time series prediction model also needs to generate a prediction confidence level corresponding to the prediction result. In this embodiment, taking the prediction of travel probability as an example, the corresponding prediction confidence level is denoted as Ctravel(t), which reflects the model's certainty about its prediction result. Optionally, a specific method to generate this confidence level is to use the Monte Carlo dropout technique. During the prediction phase, even if the model has been trained, a portion of neurons are randomly dropped with a certain probability, and multiple (e.g., 50) forward propagation calculations are performed. This will yield a distribution of the predicted travel probability, and the reciprocal of the variance or standard deviation of this distribution can be used as a measure of the prediction confidence level. That is, the smaller the variance, the more concentrated the results of multiple predictions, and the higher the confidence level; conversely, the larger the variance, the lower the confidence level.

[0059] S202, construct a multi-dimensional state space; wherein, the multi-dimensional state space includes the prediction result, the prediction confidence, the real-time vehicle status, and the real-time power grid status; the real-time vehicle status includes at least one of the following: battery charge and battery health; the real-time power grid status includes at least one of the following: real-time power grid price, power generation capacity, and power grid frequency.

[0060] This step utilizes the prediction results generated in S101, combined with the current real-time status, to make the optimal charging and discharging decision. This step is also executed on the edge computing gateway.

[0061] In this embodiment, at each decision time step (e.g., every 15 minutes), the reinforcement learning decision model first constructs a comprehensive multidimensional state space as its decision basis. This multidimensional state space St specifically includes: 1) prediction information from the time-series prediction model, such as the predicted travel probability Ptravel(t) and prediction confidence Ctravel(t) for future periods; 2) real-time vehicle status, collected by the charging pile from the vehicle's battery management system and fed back to the edge computing gateway, including at least the current battery charge and battery health, and may also include battery temperature, voltage, current, etc.; 3) real-time grid status, obtained by the edge computing gateway from an external interface, including at least the current real-time grid electricity price, and may also include the real-time power generation of local renewable energy (such as photovoltaics), grid frequency, etc.

[0062] S203, based on the multi-dimensional state space, a reinforcement learning decision model is used to generate V2X charging and discharging power commands.

[0063] In this embodiment, based on the constructed multidimensional state space St, the reinforcement learning decision model outputs an action At, namely the V2X charging and discharging power command.

[0064] The instruction is a continuous value indicating the power at which the charging station should charge or discharge the vehicle within the next time slice. For example, the action value can be set in the range [-11, 11], where a positive value represents charging and a negative value represents discharging, in kilowatts. This instruction is then sent to the charging station for precise execution.

[0065] In this embodiment, multi-source time-series data is input into a time-series prediction model, which outputs prediction results and corresponding prediction confidence levels. The prediction results include the predicted travel probability and the predicted grid electricity price within a preset time period. A multi-dimensional state space is constructed, comprising the prediction results, the prediction confidence levels, the vehicle's real-time status, and the grid's real-time status. The vehicle's real-time status includes at least one of the following: battery charge and battery health. The grid's real-time status includes at least one of the following: real-time grid electricity price, power generation, and grid frequency. Based on the multi-dimensional state space, a reinforcement learning decision model is used to generate V2X charging and discharging power commands. By predicting user travel probability and electricity prices using the time-series prediction model, charging and discharging behavior can be planned in advance, avoiding the limitations of traditional passive response modes. Furthermore, by incorporating the prediction results and confidence levels into the decision-making process, the charging and discharging strategy can intelligently balance economic benefits and user needs based on the reliability of the prediction, achieving risk adaptation.

[0066] In one optional embodiment, the reinforcement learning decision model is optimized based on a reward function, which is a weighted combination of positive rewards for electricity cost savings and grid service revenue, and negative penalties for battery loss estimation and unmet user demand.

[0067] Furthermore, the weight of the negative penalty for unmet user needs is dynamically adjusted based on the user's travel probability and the prediction confidence level. Specifically, when the predicted travel probability is greater than a first preset travel probability threshold and the prediction confidence level is greater than a first preset confidence level threshold, the weight of the negative penalty for unmet user needs is set to a first weight value. In other cases, the weight of the negative penalty for unmet user needs is set to a second weight value, where the first weight value is greater than the second weight value.

[0068] In this embodiment, the learning objective of the reinforcement learning decision model is to maximize long-term cumulative reward, and its optimization is based on a multi-objective weighted combination reward function. The specific form of this reward function can be expressed as:

[0069] Rt=α×Recon(t)+β×Rgrid(t)-γ×Cdeg(t)-δ(t)×Punmet(t);

[0070] Wherein, Rt is the total reward at time t; Recon(t) is the electricity cost saving benefit, a positive reward obtained by charging when the electricity price is low and discharging (or stopping charging) when the electricity price is high, and its value can be calculated by (power × electricity price × time slice length); Rgrid(t) is the grid service participation benefit, a positive reward, given when the system responds to grid dispatch instructions (such as frequency regulation, peak shaving and valley filling); Cdeg(t) is the battery loss estimation cost, a negative penalty, which estimates the battery life loss based on factors such as charge / discharge rate and depth and converts it into cost; Punmet(t) is the user demand unmet penalty, a negative penalty, given when the battery power is lower than the threshold required for the user's future travel; α, β, γ are preset weight coefficients used to balance the importance of different objectives; δ(t) is the dynamically adjusted weight.

[0071] In this embodiment, the weight δ(t) of the penalty for unmet user demand is not a fixed value, but is dynamically adjusted by a dynamic weight adjustment mechanism based on the output of the time-series prediction model, thereby achieving risk adaptation in the decision-making process. The specific adjustment logic is as follows: When the predicted travel probability Ptravel(tfuture) at a future time tfuture is greater than a first preset travel probability threshold (e.g., 0.8), and the corresponding prediction confidence Ctravel(tfuture) is also greater than a first preset confidence threshold (e.g., 0.9), the dynamic weight adjustment mechanism sets the penalty weight δ(t) to a higher first weight value (e.g., 10.0). That is, when the model is highly certain that the user will travel at a future time, it will prioritize charging to ensure the user's travel needs. In other cases, i.e., when the travel probability is low or the prediction uncertainty is high, δ(t) is set to a lower second weight value (e.g., 1.0), at which point the system will focus more on optimizing economic benefits or participating in grid services. This threshold-based adjustment method ensures that the first weight value is greater than the second weight value, achieving intelligent switching between guaranteeing demand and pursuing benefits.

[0072] As an alternative implementation, the dynamically adjusted weight δ(t) can also be calculated using a continuous function, as follows: δ(t)=δbase+κ×Ptravel(t)×Ctravel(t);

[0073] Here, δbase is a preset base penalty weight (e.g., 1.0), κ is a preset positive scaling factor (e.g., 10.0), and Ptravel(t) and Ctravel(t) are the predicted travel probability and prediction confidence at time t, respectively. Through this formula, the penalty weight changes continuously and proportionally to the travel probability and prediction confidence, achieving a smoother adjustment of risk preferences.

[0074] In addition, the system periodically performs swarm intelligence training based on model collaboration. This step aims to enable geographically dispersed V2X units to share learning experiences and evolve together, forming a powerful swarm intelligence, while avoiding the need to exchange their private data, thus protecting user privacy.

[0075] This step is accomplished through the collaboration of the server and all edge computing gateways, specifically using a federated learning framework. The process may include:

[0076] Step a1: Based on each V2X unit, train a local reinforcement learning decision model using local data.

[0077] In this embodiment, the server distributes an initial or previously updated global reinforcement learning decision model (i.e., model parameters ω_global) to all V2X units participating in training. Each edge computing gateway, upon receiving the global model, uses it as its local model and continuously collects empirical data generated during the decision-making process in its daily operation.<St,At,Rt,S{t+1}> The tuples are stored in the local experience replay pool. During the local training phase, the gateway uses this private local data to train the received model multiple times to generate locally optimized model update parameters Δωi.

[0078] Step a2: Upload the model parameter update data generated during training to the server so that the server can aggregate the received model parameter update data of multiple V2X units to generate a global model.

[0079] In this embodiment, after local training is completed, each edge computing gateway encrypts the generated model update parameters Δωi (instead of the original data) and uploads them to the server. After collecting a sufficient number of model update parameters uploaded by edge units (e.g., reaching a preset participation ratio), the server aggregates them. The aggregation algorithm can employ a weighted average method, for example, weighting the parameters based on the amount of data used by each edge unit during local training, to generate an updated reinforcement learning decision model ω. global_new This new model incorporates the experience of numerous edge units. The server uses this updated global model as the starting model for a new round of training and redistributes it to all V2X units.

[0080] Step a3: Receive the global model sent by the server to update its local reinforcement learning decision model.

[0081] In this embodiment, the V2X unit receives the global model from the server to update its local reinforcement learning decision model. This process is repeated continuously to achieve continuous iteration and improvement of the decision-making capability of the entire V2X network. Through the closed-loop execution of the above prediction, decision-making, and collaboration steps, an intelligent V2X charging and discharging management system with risk adaptive capability and group collaboration capability is constructed.

[0082] In one optional embodiment, before uploading the trained model parameter updates to the server, the method further includes: each V2X unit evaluating the performance of its trained reinforcement learning decision model using a preset validation dataset to obtain a performance score; the step of uploading the trained model parameter updates to the server includes: uploading the trained model parameter updates and the performance score to the server, so that the server calculates an aggregation weight for the corresponding model parameter updates based on the performance score, and performs weighted aggregation on the received model parameter updates of multiple V2X units based on the aggregation weight to generate the global model.

[0083] In this embodiment, during the swarm intelligence training process, after each V2X unit completes the training of its local reinforcement learning decision model, it first conducts a performance evaluation using a pre-set validation dataset. The validation dataset is from the same source as the training data and covers user travel behavior data (such as travel time, mileage, charging frequency, and initial charging charge) over the past n (e.g., 3) months, grid operation status data (such as real-time time-of-use electricity prices, grid frequency, and transmission power), and vehicle status data (such as battery charge, health, charging / discharging efficiency, and battery temperature). After standardized preprocessing, the model is comprehensively scored from dimensions such as economy, reliability, and grid adaptability, with a score range of 0-100, directly reflecting the model's actual operating performance.

[0084] After evaluation, the V2X units encrypt and upload the model parameter updates and corresponding performance scores to the server via the 5G-V2X secure communication protocol. The upload timing is synchronized with the local training cycle to ensure data transmission security and real-time performance. The server first verifies the validity of the uploaded data, removes abnormal data, and then calculates the aggregate weight based on the performance scores of each unit. The higher the score of the V2X unit, the larger its weight ratio, and the more significant its parameter updates are to the global model.

[0085] The server then employs a federated averaging strategy to weight and aggregate effective parameter updates, generating a global model that incorporates optimization experience from multiple units, and distributes it to all V2X units. Each V2X unit receives the model and updates its local model, forming a closed-loop collaborative mechanism of local training, performance evaluation, weighted aggregation, and global update, continuously improving the intelligence and adaptability of the entire V2X network's charging and discharging scheduling.

[0086] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0087] Corresponding to the V2X charge and discharge management method described in the above embodiments, Figure 4A structural block diagram of the V2X charge and discharge management device provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0088] Reference Figure 4 The V2X charge / discharge management device includes:

[0089] The input module is used to input the acquired multi-source time series data into the time series prediction model and output the prediction results and the prediction confidence corresponding to the prediction results; wherein, the prediction results include the predicted travel probability within a preset time period and the predicted grid electricity price within a preset time period;

[0090] A construction module is used to construct a multi-dimensional state space; wherein, the multi-dimensional state space includes the prediction result, the prediction confidence, the real-time vehicle status, and the real-time power grid status; the real-time vehicle status includes at least one of the following: battery charge and battery health; the real-time power grid status includes at least one of the following: real-time power grid price, power generation capacity, and power grid frequency;

[0091] The generation module is used to generate V2X charging and discharging power commands based on the multidimensional state space and using a reinforcement learning decision model.

[0092] In one possible implementation, the reinforcement learning decision model is optimized based on a reward function, which is a weighted combination of positive rewards for electricity cost savings and grid service revenue, and negative penalties for battery loss estimation and unmet user demand.

[0093] Furthermore, the weight of the negative penalty for unmet user needs is dynamically adjusted based on the user's travel probability and the prediction confidence level. Specifically, when the predicted travel probability is greater than a first preset travel probability threshold and the prediction confidence level is greater than a first preset confidence level threshold, the weight of the negative penalty for unmet user needs is set to a first weight value. In other cases, the weight of the negative penalty for unmet user needs is set to a second weight value, where the first weight value is greater than the second weight value.

[0094] In one possible implementation, the reward function is as follows:

[0095] Rt=α×Recon(t)+β×Rgrid(t)-γ×Cdeg(t)-δ(t)×Punmet(t);

[0096] Where Rt is the total reward at time t; Recon(t) is the electricity cost saving benefit; Rgrid(t) is the benefit of participating in grid services; Cdeg(t) is the estimated cost of battery loss; Punmet(t) is the penalty for unmet user demand; α, β, γ are preset weight coefficients; δ(t) is the dynamically adjusted weight.

[0097] In one possible implementation, the dynamically adjusted weight δ(t) is calculated using the following formula:

[0098] δ(t)=δbase+κ×Ptravel(t)×Ctravel(t);

[0099] Where δbase is a preset base penalty weight, κ is a preset positive scaling factor, Ptravel(t) is the predicted travel probability at time t, and Ctravel(t) is the prediction confidence at time t.

[0100] In one possible implementation, the V2X charge / discharge management device further includes a training module for:

[0101] Based on each V2X unit, a local reinforcement learning decision model is trained using local data.

[0102] The model parameter updates generated during training are uploaded to the server, so that the server can aggregate the model parameter updates received from multiple V2X units to generate a global model.

[0103] The device receives the global model from the server to update its local reinforcement learning decision model. In one possible implementation, the V2X charge / discharge management device further includes an evaluation module for:

[0104] Each V2X unit evaluates the performance of its trained reinforcement learning decision model using a pre-defined validation dataset and obtains a performance score.

[0105] The training module is also used to upload the model parameter update amount generated during training and the performance score to the server, so that the server calculates the aggregation weight for the corresponding model parameter update amount based on the performance score, and performs weighted aggregation on the model parameter update amounts of the received multiple V2X units based on the aggregation weight to generate the global model.

[0106] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0107] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0108] Corresponding to the V2X charge / discharge management method described in the above embodiments, this application also provides a description of a V2X charge / discharge management system, which can be found in [reference needed]. Figure 1 The accompanying drawings are shown. For ease of explanation, only the parts relevant to the embodiments of this application are shown.

[0109] A V2X charging and discharging management system includes: a server and multiple V2X units; the V2X units include edge computing gateways;

[0110] The edge computing gateway of the V2X unit is used to acquire multi-source time-series data and generate prediction results and corresponding prediction confidence levels through a time-series prediction model; construct a multi-dimensional state space, and generate V2X charging and discharging power commands based on the multi-dimensional state space using a reinforcement learning decision model; wherein, the prediction results include the predicted travel probability and the predicted grid electricity price within a preset time period; the multi-dimensional state space includes the prediction results, the prediction confidence levels, the real-time vehicle status, and the real-time grid status; the real-time vehicle status includes at least one of the following: battery charge and battery health; the real-time grid status includes at least one of the following: real-time grid electricity price, power generation, and grid frequency;

[0111] The server is used to collaboratively train the reinforcement learning decision models of multiple V2X units; the collaborative training includes: periodically receiving model parameter updates uploaded from multiple V2X units, aggregating the received model parameter updates from multiple V2X units to generate a global model, and distributing the global model to multiple V2X units to update their local reinforcement learning decision models.

[0112] In one possible implementation, the V2X unit further includes a charging pile that is communicatively connected to the edge computing gateway;

[0113] The edge computing gateway is used to send the generated V2X charging and discharging power command to the charging pile;

[0114] The charging pile is used to execute the V2X charging and discharging power command, and to collect the real-time status of the vehicle and feed the real-time status of the vehicle back to the edge computing gateway as input to the reinforcement learning decision model.

[0115] Corresponding to the V2X charging and discharging management method described above, the V2X charging and discharging management system provided in this application adopts a three-level architecture design of cloud-edge-device, and realizes dynamic optimization management of V2X charging and discharging through distributed collaboration and intelligent decision-making.

[0116] The system consists of a server and multiple V2X units. Each V2X unit serves as the core execution unit on the edge side and is deployed on the user side (such as home garages, office park parking lots, etc.). Each V2X unit contains at least one edge computing gateway. In some implementation scenarios, charging piles that communicate with the edge computing gateway can also be configured to form a local interactive closed loop of gateway-charging pile-vehicle.

[0117] In this embodiment, the edge computing gateway is equipped with a reinforcement learning decision model. Using a multi-dimensional state space as input, and through the model's reward function optimization mechanism, it generates precise V2X charging and discharging power commands, clearly defining the charging and discharging power and execution time of the charging pile. If the V2X unit is equipped with a charging pile, the edge computing gateway will send the command to the charging pile in real time, allowing the charging pile to execute the specific charging and discharging operations. Simultaneously, the charging pile will continuously collect the vehicle's real-time status and feed it back to the edge computing gateway as dynamic input to the reinforcement learning decision model, forming a local decision-making closed loop. This system integrates multi-scenario experience through cloud-side collaborative training, optimizes the model's generalization ability, and, relying on the multi-dimensional state space and reinforcement learning decision model, achieves advance planning and risk adaptation of charging and discharging strategies, effectively solving the problems of rigid static strategies and lack of collaborative optimization in traditional V2X charging and discharging management.

[0118] This application also provides a control device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0119] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0120] This application provides a computer program product that, when run on a control device, enables the control device to perform the steps described in the above-described method embodiments.

[0121] Figure 5 This is a schematic diagram of the structure of a control device provided in one embodiment of this application. Figure 5 As shown, the control device of this embodiment includes: at least one processor 50 ( Figure 5 (Only one is shown in the diagram), memory 51, and computer program 52 stored in said memory 51 and executable on said at least one processor 50, wherein said processor 50 executes said computer program 52 to implement the steps in any of the above-described V2X charge and discharge management method embodiments.

[0122] The control device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will understand that... Figure 5 This is merely an example of a control device and does not constitute a limitation on the control device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0123] The processor 50 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0124] In some embodiments, the memory 51 may be an internal storage unit of the control device, such as a hard disk or memory. In other embodiments, the memory 51 may be an external storage device of the control device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., provided on the control device. Furthermore, the memory 51 may include both internal and external storage units of the control device. The memory 51 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0125] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / control device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. Computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0126] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0127] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0128] In the embodiments provided in this application, it should be understood that the disclosed apparatus / control devices and methods can be implemented in other ways. For example, the apparatus / control device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0129] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0130] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A V2X charge-discharge management method, characterized in that, The V2X charging and discharging management method comprises: inputting the obtained multi-source time series data into a time series prediction model to output a prediction result and a prediction confidence corresponding to the prediction result; wherein the prediction result comprises a predicted travel probability within a preset time period and a predicted power grid price within a preset time period; constructing a multi-dimensional state space; wherein the multi-dimensional state space comprises the prediction result, the prediction confidence, a vehicle real-time state, and a power grid real-time state; the vehicle real-time state comprises at least one of a battery capacity and a battery health degree; the power grid real-time state comprises at least one of a real-time power grid price, a power generation capacity, and a power grid frequency; based on the multi-dimensional state space, a reinforcement learning decision model is used to generate a V2X charging and discharging power instruction.

2. The V2X charge and discharge management method of claim 1, wherein, The reinforcement learning decision model is optimized according to a reward function, and the reward function is a weighted combination of positive rewards of electricity saving benefits and power grid service benefits, and negative punishments of battery consumption estimation and user demand unmet; and the weight of the negative punishment of the user demand unmet is dynamically adjusted according to the user travel probability and the prediction confidence; wherein when the predicted travel probability is greater than a first preset travel probability threshold and the prediction confidence is greater than a first preset confidence threshold, the weight of the negative punishment of the user demand unmet is set to a first weight value, and in other cases, the weight of the negative punishment of the user demand unmet is set to a second weight value, the first weight value is greater than the second weight value. 3.The V2X charge and discharge management method of claim 2, wherein, The reward function is as follows: Rt=α×Recon(t)+β×Rgrid(t)-γ×Cdeg(t)-δ(t)×Punmet(t); wherein Rt is the total reward at time t; Recon(t) is the electricity saving benefit item; Rgrid(t) is the power grid service benefit item; Cdeg(t) is the battery consumption estimation cost item; Punmet(t) is the user demand unmet punishment item; α, β, γ are preset weight coefficients; δ(t) is a dynamically adjusted weight.

4. The V2X charge and discharge management method of claim 3, wherein, The dynamically adjusted weight δ(t) is calculated by the following formula: δ(t)=δbase+κ×Ptravel(t)×Ctravel(t); wherein δbase is a preset basic punishment weight, κ is a preset positive scaling coefficient, Ptravel(t) is the predicted travel probability at time t, and Ctravel(t) is the prediction confidence at time t. 5.The V2X charge-discharge management method of claim 1, wherein, The method further comprises: based on each V2X unit, training a local reinforcement learning decision model using local data; uploading the model parameter update generated by the training to a server to enable the server to aggregate the model parameter updates of the plurality of V2X units received to generate a global model; receiving the global model issued by the server to update the local reinforcement learning decision model thereof.

6. The V2X charge and discharge management method of claim 5, wherein, The uploading of the model parameter update generated by the training to the server further comprises: Each V2X unit evaluates the performance of the trained reinforcement learning decision model through a preset verification data set to obtain a performance score; The model parameter update generated by the training is uploaded to a server, including: The model parameter update generated by the training and the performance score are uploaded to the server, so that the server calculates an aggregation weight according to the performance score for the corresponding model parameter update, and performs weighted aggregation on the model parameter updates of the multiple V2X units received according to the aggregation weight, to generate the global model. 7.A V2X charge-discharge management apparatus, characterized by, Including: An input module is configured to input the acquired multi-source time series data into a time series prediction model, and output a prediction result and a prediction confidence corresponding to the prediction result; wherein the prediction result includes a predicted travel probability within a preset time period and a predicted power grid price within a preset time period; A construction module is configured to construct a multi-dimensional state space; wherein the multi-dimensional state space includes the prediction result, the prediction confidence, a vehicle real-time state, and a power grid real-time state; the vehicle real-time state includes at least one of a battery capacity and a battery health degree; and the power grid real-time state includes at least one of a real-time power grid price, a power generation capacity, and a power grid frequency; A generation module is configured to generate a V2X charging and discharging power instruction based on the multi-dimensional state space and using a reinforcement learning decision model. 8.A V2X charge-discharge management system, characterized in that, Including: A server and multiple V2X units; The V2X unit includes an edge computing gateway; The edge computing gateway of the V2X unit is configured to acquire multi-source time series data, and generate a prediction result and a prediction confidence corresponding to the prediction result through a time series prediction model; A multi-dimensional state space is constructed, and a V2X charging and discharging power instruction is generated based on the multi-dimensional state space and using a reinforcement learning decision model; wherein the prediction result includes a predicted travel probability within a preset time period and a predicted power grid price within a preset time period; the multi-dimensional state space includes the prediction result, the prediction confidence, a vehicle real-time state, and a power grid real-time state; the vehicle real-time state includes at least one of a battery capacity and a battery health degree; and the power grid real-time state includes at least one of a real-time power grid price, a power generation capacity, and a power grid frequency; The server is configured to cooperatively train the reinforcement learning decision models of the multiple V2X units; the cooperative training includes periodically receiving model parameter updates uploaded from the multiple V2X units, aggregating the received model parameter updates of the multiple V2X units to generate a global model, and distributing the global model to the multiple V2X units to update their local reinforcement learning decision models. 9.The V2X charge and discharge management system of claim 8, wherein, The V2X unit further includes a charging pile in communication connection with the edge computing gateway; The edge computing gateway is configured to send the generated V2X charging and discharging power instruction to the charging pile; The charging pile is configured to execute the V2X charging and discharging power instruction, and collect a vehicle real-time state and feed back the vehicle real-time state to the edge computing gateway as an input of the reinforcement learning decision model.

10. A computer program product, characterised in that, The computer program comprises a computer program which, when executed, implements the V2X charging and discharging management method according to any one of claims 1 to 6.

Images