A formal verification method and system for deep reinforcement learning charging strategy
By constructing a hybrid integer abstract domain and a planning method, the problem that existing verification methods cannot cover all states in electric vehicle charging scenarios is solved. This enables accurate calculation of the lower bound of deep reinforcement learning strategies, thereby improving the safety and reliability of intelligent charging systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTHEAST UNIV
- Filing Date
- 2026-01-09
- Publication Date
- 2026-05-15
AI Technical Summary
Existing sampling-based testing and verification methods cannot cover all possible states and random perturbations in electric vehicle charging scenarios, making it impossible to theoretically obtain an accurate lower bound on the performance of deep reinforcement learning strategies, thus limiting the safe deployment and reliability of smart charging strategies.
A mixed-integer abstract domain is constructed to map the continuous state space of electric vehicle charging stations to abstract elements. The mixed-integer programming method is used to verify the deep reinforcement learning network and stochastic environment dynamics, forming an accurate lower bound for policy performance.
By employing a mixed-integer programming approach, all possible system states and random disturbances can be covered, providing a mathematically provable lower bound on the policy performance, thereby improving the economy and safety of smart charging strategies.
Smart Images
Figure CN121504223B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of smart grid scheduling, electric vehicle charging and discharging optimization, and artificial intelligence safety verification technology, and in particular to a formal verification method and system for deep reinforcement learning charging strategies based on a hybrid integer abstraction domain. Background Technology
[0002] With the acceleration of global energy transition and transportation electrification, electric vehicles (EVs) have become an indispensable flexibility resource in smart grids. In the context of large-scale EV integration, orderly charging and discharging scheduling through vehicle-to-grid (V2G) technology is of great significance for mitigating grid load fluctuations, absorbing renewable energy, and reducing user charging costs.
[0003] In recent years, Deep Reinforcement Learning (DRL) has been widely applied to solve high-dimensional continuous control problems such as electric vehicle charging scheduling and electricity market bidding due to its powerful nonlinear fitting capabilities and decision-making advantages in complex dynamic environments. Compared with traditional model-based optimization methods, DRL can autonomously learn strategies through interaction with the environment, without requiring precise prior physical models, and shows significant advantages in handling multiple uncertainties. However, a DRL strategy is essentially a black-box model parameterized by a deep neural network, and its decision-making logic lacks transparency and interpretability. Neural networks are often extremely sensitive to small perturbations in the input state, and when faced with extreme scenarios outside the training data distribution, they may output actions that violate safety constraints or lead to serious economic losses. Therefore, before applying DRL strategies to actual charging station control, rigorous verification of safety and performance lower bounds is necessary.
[0004] Currently, validation methods for DRL strategies mainly fall into two categories, but both have significant limitations:
[0005] Sampling-based testing methods, such as Monte Carlo simulation and Latin hypercube sampling, are currently the most commonly used methods in industry, which simulate and evaluate agents on a finite test set. However, the state space of an electric vehicle charging scenario is continuous and high-dimensional, containing continuous battery states of charge, random photovoltaic and electricity price sequences, and the environmental dynamics are highly stochastic. A limited simulation trajectory can only cover an extremely sparse subset of the state space, unable to traverse all possible environmental evolution paths. Sampling results only represent performance statistics under a specific probability distribution, exhibiting severe path dependence, failing to theoretically rule out catastrophic failures, and unable to provide a precise lower bound on the strategy's performance in the worst-case scenario.
[0006] Existing formal verification methods, such as those based on reachability analysis, mathematically prove that a system's behavior satisfies specific properties under all possible inputs. However, existing formal verification tools are primarily designed for deterministic systems or discrete state-space systems. When dealing with complex charging systems involving continuous variables and stochastic dynamics, existing methods often approximate the set of states using simple geometries. This coarse approximation leads to a significant accumulation of over-approximation errors during multi-step sequential decision-making, resulting in overly loose performance boundaries that lack practical guidance. Furthermore, existing abstract domains struggle to capture the logical coupling relationships between variables, such as the mutual exclusion constraints of battery charging and discharging states and physical capacity limitations, leading to inaccurate verification results.
[0007] Therefore, there is a need for a formal verification method that can accurately capture the physical and logical constraints of a system in continuous state space and stochastic environmental dynamics, and can calculate the exact lower bound of the performance of the DRL strategy, so as to ensure the safety and reliability of smart charging systems in practical applications. Summary of the Invention
[0008] The technical problem this invention aims to solve is that in the context of smart grid vehicle-to-grid (V2G) interaction, the use of deep reinforcement learning to control electric vehicle charging requires verification of safety and performance lower bounds. Existing sampling-based testing and verification methods can only cover a limited number of test cases, making it difficult to traverse all possible system states and random disturbances. This results in the inability to theoretically obtain a precise lower bound on the strategy's performance, i.e., the worst-case economic or safety indicators, thus limiting the safe deployment and reliability assessment of smart charging strategies in practical engineering.
[0009] To address the aforementioned technical problems, this invention provides a formal verification method for deep reinforcement learning charging strategies, comprising the following steps:
[0010] Step S1: Construct a hybrid integer abstract domain, and map the continuous state space in the electric vehicle charging station operation scenario into abstract elements in the hybrid integer abstract domain. The continuous state space includes the electric vehicle battery state of charge (SoC), light intensity, and electricity price as continuous variables.
[0011] Step S2: Model the deep reinforcement learning network and the stochastic environment dynamics as abstract transformers on the hybrid integer abstract domain, respectively;
[0012] Step S3: Within a preset time step, starting from the initial abstract element, alternately apply the abstract transformer of the deep reinforcement learning network and the abstract transformer of the stochastic environment dynamics to obtain the final abstract element;
[0013] Step S4: Using the lower bound of the cumulative reward of the strategy as the objective function, transform the symbolic constraints in the final abstract element into a specific mixed integer programming problem, and solve it using the MIP solver.
[0014] In the aforementioned formal verification method for deep reinforcement learning charging strategies, step S1 defines the abstract element as an ordered pair. ,in For a set of variables, For constraint sets;
[0015] The set of variables It includes continuous variables; it also includes integer variables to represent charging / discharging state switching and logical judgment results; the constraint set Includes mixed-integer linear constraints that describe the coupling relationships between variables.
[0016] In the aforementioned formal verification method for deep reinforcement learning charging strategies, in step S2, the abstract converter... Defined as an operation used to receive the current abstract element. And the input and output definitions of the electric vehicle charging station system to be verified, generating a new abstract element. The new abstract element It includes an updated set of variables and constraints, thus tracking the evolution of the system at the symbolic level.
[0017] In the aforementioned formal verification method for deep reinforcement learning charging strategies, step S2, the modeling of the deep reinforcement learning network includes: the deep reinforcement learning network is transformed into mixed integer constraints through layer-by-layer modeling, including:
[0018] For linear layers, the affine transformation formula is used to directly model them as linear constraints;
[0019] For nonlinear activation layers, including the ReLU function, the Big-M method is used to transform the nonlinear activation layer into a mixed-integer linear constraint. Let the input be... The output is Then the ReLU function The model is defined by the following constraints:
[0020] (1)
[0021] in, For the introduced binary auxiliary variables, This is used to indicate whether a neuron is activated; and Inputs The preset lower and upper bound vectors; ⊙ represents element-wise multiplication.
[0022] In the aforementioned formal verification method for deep reinforcement learning charging strategies, step S2 involves modeling the stochastic environment dynamics, which includes modeling the stochastic environment dynamics based on the Chargym simulation platform and encompassing physical constraints and stochastic environment variables.
[0023] For random environmental variables in a continuous state space, a bounded perturbation model is adopted, assuming the predicted value is... , the actual value The constraint is within a bounded interval centered on the predicted value:
[0024] (2)
[0025] in The maximum permissible disturbance amplitude;
[0026] In the physical constraints, standardized modeling is performed using the Big-M method, including:
[0027] Current state of charge of electric vehicle batteries Recorded as ,action Recorded as The state of charge at the next moment Recorded as ;
[0028] Define auxiliary variable: available discharge capacity ( Available charging capacity ( Actual execution capacity ( Define binary state variables: charging and discharging states. Insufficient discharge capacity indicator Low charging capacity indicator ;
[0029] The physical constraints are constructed as follows:
[0030] Charge / discharge state constraints: based on action The positive or negative sign determines whether the electric vehicle battery is in a charging or discharging state. 1 indicates charging, 0 indicates discharging:
[0031] (3)
[0032] Available capacity limitation: Available discharge capacity Limited by the rated power of the charging pile and the energy currently contained in the battery The minimum value, The total battery capacity is modeled using the Big-M method as follows:
[0033] (4)
[0034] Among them, insufficient discharge capacity indication A value of 1 indicates that the power is limited by the battery energy, and 0 indicates that the power is limited by the charging pile.
[0035] Similarly, available charging capacity Limited by the rated power of the charging pile and remaining battery space The minimum value indicates insufficient charging capacity. The model is as follows:
[0036] (5)
[0037] Actual execution and state update constraints: Actual execution capacity According to the charge and discharge state Select available discharge capacity Or available charging capacity :
[0038] (6)
[0039] The next moment's SoC (System-on-a-Chip) The updated formula is:
[0040] (7).
[0041] In the aforementioned formal verification method for deep reinforcement learning charging strategies, in step S3, the verification time step is set to be... From the initial abstract element First, a deep reinforcement learning network abstract transformer is applied. Obtain the abstract element containing policy action constraints. Then, an abstract transformer based on stochastic environment dynamics is applied. Obtain the abstract element of the state at the next moment. ;
[0042] The iterative process is represented as:
[0043] (8)
[0044] The final abstract element Contains a complete set of mixed integer constraints .
[0045] In the aforementioned formal verification method for deep reinforcement learning charging strategies, in step S4, the objective function is defined as minimizing the lower bound of the cumulative reward, i.e. maximizing the cumulative cost.
[0046] Cost function Including electricity purchase costs from the grid Penalty for insufficient battery power ,
[0047] (9)
[0048] The cost of electricity purchased by the power grid is expressed as follows:
[0049] (10)
[0050] in, This represents the actual power absorbed from or fed back to the grid; Indicates that the photovoltaic system is based on light intensity The power generation capacity; For real-time electricity prices, This represents the action vector output by the deep reinforcement learning network. This represents the rated power vector of the charging station. Indicates the photovoltaic power conversion factor;
[0051] The penalty for insufficient battery charge is only applied when the vehicle leaves the parking area.
[0052] (11)
[0053] in, The penalty coefficient is... This indicates the degree to which the battery is not fully charged. This represents the departure indicator variable, when the vehicle... At any moment Present and at the moment The value is 1 when leaving the field, and 0 otherwise. Indicates diagonal matrix operations, superscript This represents the mathematical transpose operation;
[0054] The problem of calculating the lower bound of the final policy performance is transformed into the following mixed integer programming problem:
[0055] (12)
[0056] in This refers to the vector of all specific variables involved.
[0057] A computer system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described above.
[0058] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.
[0059] The beneficial effects achieved by this invention are as follows: The verification framework based on a mixed-integer abstract domain proposed in this invention, through abstract interpretation theory and mixed-integer programming, can cover all possibilities in continuous state space and stochastic environment dynamics, overcoming the bottlenecks of traditional sampling testing methods, such as insufficient coverage in continuous space and inability to capture extreme scenarios. By constructing an accurate mixed-integer abstract transformer, the precise lower bound of the cumulative reward of the deep reinforcement learning policy within a preset time step can be calculated. Compared with methods that can only provide statistically meaningful evaluation results, this lower bound is mathematically provable, improving the economy and security of the charging strategy.
[0060] Meanwhile, this invention utilizes the Big-M method to integrate the complex vehicle-to-grid (V2G) environment, which includes logical judgments, nonlinear activation functions, and random perturbations, into a mixed-integer programming model, significantly improving the verification accuracy and reliability of intelligent charging strategies in complex real-world scenarios. Through iterative application of an abstract transformer, this invention achieves reachability analysis of multi-step sequential decision-making processes, capturing the cumulative error and long-term dependencies of the strategy over time, making it suitable for verifying charging scheduling problems with temporal characteristics. Attached Figure Description
[0061] Figure 1 This is a flowchart of a formal verification method framework for a deep reinforcement learning charging strategy according to Embodiment 1 of the present invention.
[0062] Figure 2 This is a schematic diagram comparing the formal optimal lower bound with the search results of the LHS sampling method provided in Embodiment 1 of the present invention;
[0063] Figure 3 This is a graph showing the computation time or computational efficiency of the verification method under different verification step sizes provided in Embodiment 1 of the present invention. Detailed Implementation
[0064] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings.
[0065] Example 1
[0066] like Figure 1As shown, this embodiment provides a formal verification method for deep reinforcement learning charging strategies, addressing the problem that existing sampling-based testing methods cannot provide a precise lower bound on performance under continuous state space and stochastic environmental dynamics. The method includes the following steps:
[0067] Step S1: Construct a Mixed-Integer Abstract Domain, in which the continuous state space of the electric vehicle charging station operation scenario is mapped to abstract elements. The continuous state space includes continuous variables such as the electric vehicle battery state of charge (SoC), light intensity, and electricity price.
[0068] In this step, the abstract element is defined as an ordered pair. ,in For a set of variables, For the constraint set.
[0069] Unlike existing interval domains, in the Mixed-Integer (MI) abstract domain, the set of variables... It includes not only continuous variables, representing battery state of charge (SoC), light intensity, and electricity price, but also integer variables, representing charging / discharging state switching, logical judgment results, etc. The constraint set... Includes mixed-integer linear constraints that describe the coupling relationships between variables.
[0070] By constructing boundary constraints for the initial state, the continuous initial state space is mapped to initial abstract elements. If the initial battery state of charge (SoC) is between [0.2, 0.8], it is mapped as follows: .
[0071] Step S2: Model the deep reinforcement learning network and the stochastic environment dynamics as abstract transformers on the hybrid integer abstract domain, respectively.
[0072] The Abstract Transformer Defined as an operation used to receive the current abstract element. And the input and output definitions of the electric vehicle charging station system to be verified, generating a new abstract element. The new abstract element It includes an updated set of variables and constraints, thus tracking the evolution of the system at the symbolic level.
[0073] Modeling of the deep reinforcement learning network:
[0074] In this embodiment, the deep reinforcement learning network is transformed into mixed integer constraints through layer-by-layer modeling, including:
[0075] For linear layers, the affine transformation formula is used to directly model them as linear constraints.
[0076] For nonlinear activation layers, including the ReLU function, the Big-M method is used to transform the nonlinear activation layer into a mixed-integer linear constraint. Let the input be... The output is ReLU function The model is defined by the following constraints:
[0077] (1)
[0078] in, For the introduced binary auxiliary variables, This is used to indicate whether a neuron is activated; and Inputs The preset lower and upper bound vectors are obtained in advance through interval arithmetic; ⊙ represents element-wise multiplication. The above constraints accurately capture the piecewise linearity of the ReLU function without any approximation error.
[0079] The modeling of the stochastic environment dynamics includes:
[0080] Stochastic environment dynamics are modeled using the Chargym simulation platform, including physical constraints and stochastic environment variables:
[0081] For stochastic environmental variables in continuous state space, such as light intensity and electricity price, a bounded perturbation model is adopted. Let the predicted value be... , the actual value The constraint is within a bounded interval centered on the predicted value:
[0082] (2)
[0083] in This represents the maximum permissible disturbance amplitude.
[0084] In the physical constraints, the core of the environment is the charging dynamics of the electric vehicle battery. In order to handle the nonlinear logic of battery charging / discharging state switching and capacity constraints in mixed integer programming, this embodiment uses the Big-M method for standardized modeling.
[0085] Based on the definition of the above mixed-integer abstract field, the following variable is defined: the current state of charge of the electric vehicle battery. Recorded as ,action Recorded as The state of charge at the next moment Recorded as ;
[0086] The auxiliary variables defined include: available discharge capacity. ( Available charging capacity ( Actual execution capacity ( Define binary state variables: charging and discharging states. Insufficient discharge capacity indicator Low charging capacity indicator .
[0087] The physical constraints are constructed as follows:
[0088] 1) Charge / discharge state constraints: based on the action The positive or negative sign determines whether the electric vehicle battery is in a charging or discharging state. 1 indicates charging, 0 indicates discharging:
[0089] (3)
[0090] 2) Available capacity limitation: Available discharge capacity Limited by the rated power of the charging pile and the energy currently contained in the battery The minimum value, The total battery capacity is modeled using the Big-M method as follows:
[0091] (4)
[0092] Among them, insufficient discharge capacity indication A value of 1 indicates that the power is limited by the battery energy, and 0 indicates that the power is limited by the charging pile.
[0093] Similarly, available charging capacity Limited by the rated power of the charging pile and remaining battery space The minimum value, using variable charging capacity insufficient indication The model is as follows:
[0094] (5)
[0095] 3) Actual execution and state update constraints: Actual execution capacity According to the charge and discharge state Select available discharge capacity Or available charging capacity :
[0096] (6)
[0097] Ultimately, the SoC in the next moment ( The updated formula is:
[0098] (7).
[0099] Step S3: Within a preset time step, starting from the initial abstract element, alternately apply the abstract transformer of the deep reinforcement learning network and the abstract transformer of the stochastic environment dynamics to obtain the final abstract element.
[0100] Let the time step of the verification be... From the initial abstract element First, a deep reinforcement learning network abstract transformer is applied. Obtain the abstract element containing policy action constraints. Then, an abstract transformer based on stochastic environment dynamics is applied. Obtain the abstract element of the state at the next moment. .
[0101] The iterative process is represented as:
[0102] (8)
[0103] The final abstract element Contains a complete set of mixed integer constraints The mixed integer constraint set fully describes the electric vehicle charging system starting from the initial state, under all possible random perturbations and deep reinforcement learning network responses. All feasible trajectories within a step.
[0104] Step S4: Using the lower bound of the cumulative reward of the strategy as the objective function, transform the symbolic constraints in the final abstract element into a specific mixed integer programming (MIP) problem, and solve it using a general MIP solver.
[0105] To verify the economy and security of the strategy, this embodiment defines the objective function as minimizing the lower bound of the cumulative reward, i.e. maximizing the cumulative cost.
[0106] Cost function Including electricity purchase costs from the grid Penalty for insufficient battery power ,
[0107] (9)
[0108] The cost of electricity purchased by the power grid is expressed as follows:
[0109] (10)
[0110] in, This represents the actual power absorbed from or fed back to the grid; Indicates that the photovoltaic system is based on light intensity Power generation capacity; For real-time electricity prices, This represents the action vector output by the deep reinforcement learning network. This represents the rated power vector of the charging station. This represents the photovoltaic power generation conversion factor.
[0111] The penalty for insufficient battery charge is only applied when the vehicle leaves the parking area.
[0112] (11)
[0113] in, The penalty coefficient is... This indicates the degree to which the battery is not fully charged. This represents the departure indicator variable, when the vehicle... At any moment Present and at the moment The value is 1 when leaving the game, and 0 otherwise. Indicates diagonal matrix operations, superscript This represents the transpose operation in mathematics.
[0114] The problem of calculating the lower bound of the final policy performance is transformed into the following mixed integer programming (MIP) problem:
[0115] (12)
[0116] in For all the specific variable vectors involved, solving the MIP problem yields an accurate lower bound on the strategy performance considering all environmental uncertainties and the physical constraints of the charging station and battery system, including the worst combination of photovoltaic and electricity prices.
[0117] In addition to the continuous state space, the electric vehicle charging station operation scenario also includes the discrete variable of the remaining charging time vector. Treating the remaining charging time directly as an integer variable in the MIP would significantly increase the solution complexity. This embodiment employs a parametric decomposition method to decompose the remaining charging time vector... Treating it as a parameter rather than a variable, due to the remaining charging time vector The value of is finite and definite, decreasing with time; we can enumerate the vector of remaining charging time. All possible combinations of initial values, for each fixed remaining charging time vector Value combinations greatly simplify the related logic in the MIP problem mentioned above. The final global lower bound can be solved in parallel for all... The subproblems corresponding to the value combinations are identified, and the minimum value is obtained. This not only reduces the size of a single MIP but also supports parallel computation, significantly improving verification efficiency.
[0118] To verify the effectiveness and superiority of the method proposed in this invention, tests were conducted in a simulation scenario with 10 charging piles and integrated photovoltaic and V2G functions. The MIP solution was performed on a server using the Gurobi solver.
[0119] The formal verification method proposed in this invention was compared with the Latin hypercube sampling (LHS) test method commonly used in industry.
[0120] like Figure 2 As shown, although the LHS method (blue) can explore some high-cost regions through a large number of samples, the "worst-case sampling" it finds is always lower than the formal optimal lower bound (green) calculated by this invention.
[0121] Experimental results show that even with increasing sampling frequency, the discovery capability of sampling methods exhibits diminishing marginal returns and tends towards saturation, failing to theoretically guarantee finding the true worst-case scenario. In contrast, the method based on a mixed-integer abstract field in this invention can cover all possibilities in continuous space and stochastic dynamics, providing a mathematically provable lower bound on the exact performance. Furthermore, the small difference between this lower bound and the sampling limit indicates that the abstract field constructed in this invention has high compactness and does not introduce excessive approximation errors.
[0122] Further analysis of the computational efficiency of this invention under different verification step sizes. For example... Figure 3 As shown, the solution time for the MIP problem increases with the increase of the lookahead steps. Experimental data shows that for a verification task with 3 time steps (corresponding to a 3-hour operational window in reality), the method of this invention can complete the solution within 1 hour. This indicates that the present invention can complete the rigorous security verification of short-term critical operational windows within a reasonable time cost, and is suitable for gating detection before the deployment of vehicle-to-everything (V2X) interaction control strategies with extremely high security requirements.
[0123] Example 2
[0124] A computer system includes a memory, a processor, and a computer program stored in the memory, the processor executing the computer program to implement the steps of the method as described in Embodiment 1.
[0125] Example 3
[0126] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in Example 1.
[0127] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A formal verification method for deep reinforcement learning charging strategies, characterized in that, include: Step S1: Construct a hybrid integer abstract domain, and map the continuous state space in the electric vehicle charging station operation scenario into abstract elements in the hybrid integer abstract domain. The continuous state space includes the electric vehicle battery state of charge (SoC), light intensity, and electricity price as continuous variables. Step S2: Model the deep reinforcement learning network and the stochastic environment dynamics as abstract transformers on the hybrid integer abstract domain, respectively; Step S3: Within a preset time step, starting from the initial abstract element, alternately apply the abstract transformer of the deep reinforcement learning network and the abstract transformer of the stochastic environment dynamics to obtain the final abstract element; Step S4: Using the lower bound of the cumulative reward of the strategy as the objective function, transform the symbolic constraints in the final abstract element into a specific mixed integer programming problem, and solve it using the MIP solver; In step S1, the abstract element is defined as an ordered pair. ,in For a set of variables, For constraint sets; The set of variables It includes continuous variables; it also includes integer variables to represent charging / discharging state switching and logical judgment results; the constraint set Includes mixed-integer linear constraints that describe the coupling relationships between variables; In step S2, the abstract converter Defined as an operation used to receive the current abstract element. And the input and output definitions of the electric vehicle charging station system to be verified, generating a new abstract element. The new abstract element It includes the updated set of variables and constraints, thus tracking the evolution of the system at the symbolic level; In step S2, the modeling of the deep reinforcement learning network includes: the deep reinforcement learning network is transformed into mixed integer constraints through layer-by-layer modeling, including: For linear layers, the affine transformation formula is used to directly model them as linear constraints; For nonlinear activation layers, including the ReLU function, the Big-M method is used to transform the nonlinear activation layer into a mixed-integer linear constraint. Let the input be... The output is Then the ReLU function The model is defined by the following constraints: （1） in, For the introduced binary auxiliary variables, This is used to indicate whether a neuron is activated; and Inputs The preset lower and upper bound vectors; ⊙ represents element-wise multiplication; In step S2, the modeling of the stochastic environment dynamics includes: modeling the stochastic environment dynamics based on the Chargym simulation platform, including physical constraints and stochastic environment variables: For random environmental variables in a continuous state space, a bounded perturbation model is adopted, assuming the predicted value is... , the actual value The constraint is within a bounded interval centered on the predicted value: （2） in The maximum permissible disturbance amplitude; In the physical constraints, standardized modeling is performed using the Big-M method, including: Current state of charge of electric vehicle batteries Recorded as ,action Recorded as The state of charge at the next moment Recorded as ; Define auxiliary variable: available discharge capacity ( Available charging capacity ( Actual execution capacity ( Define binary state variables: charging and discharging states. Insufficient discharge capacity indicator Low charging capacity indicator ; The physical constraints are constructed as follows: Charge / discharge state constraints: based on action The positive or negative sign determines whether the electric vehicle battery is in a charging or discharging state. 1 indicates charging, 0 indicates discharging: （3） Available capacity limitation: Available discharge capacity Limited by the rated power of the charging pile and the energy currently contained in the battery The minimum value, The total battery capacity is modeled using the Big-M method as follows: （4） Among them, insufficient discharge capacity indication A value of 1 indicates that the power is limited by the battery energy, and 0 indicates that the power is limited by the charging pile. Similarly, available charging capacity Limited by the rated power of the charging pile and remaining battery space The minimum value indicates insufficient charging capacity. The model is as follows: （5） Actual execution and state update constraints: Actual execution capacity According to the charge and discharge state Select available discharge capacity Or available charging capacity : （6） The next moment's SoC (System-on-a-Chip) The updated formula is: （7）。 2. The formal verification method for deep reinforcement learning charging strategies according to claim 1, characterized in that, In step S3, let the verification time step be... From the initial abstract element First, a deep reinforcement learning network abstract transformer is applied. Obtain the abstract element containing policy action constraints. Then, an abstract transformer based on stochastic environment dynamics is applied. Obtain the abstract element of the state at the next moment. ; The iterative process is represented as: （8） The final abstract element Contains a complete set of mixed integer constraints .
3. The formal verification method for deep reinforcement learning charging strategies according to claim 1, characterized in that, In step S4, the objective function is defined as minimizing the lower bound of the cumulative reward, i.e. maximizing the cumulative cost; Cost function Including electricity purchase costs from the grid Penalty for insufficient battery power , （9） The cost of electricity purchased by the power grid is expressed as follows: （10） in, This represents the actual power absorbed from or fed back to the grid; Indicates that the photovoltaic system is based on light intensity The power generation capacity; For real-time electricity prices, This represents the action vector output by the deep reinforcement learning network. This represents the rated power vector of the charging station. Indicates the photovoltaic power conversion factor; The penalty for insufficient battery charge is only applied when the vehicle leaves the parking area. （11） in, The penalty coefficient is... This indicates the degree to which the battery is not fully charged. This represents the departure indicator variable, when the vehicle... At any moment Present and at the moment The value is 1 when leaving the field, and 0 otherwise. Indicates diagonal matrix operations, superscript This represents the mathematical transpose operation; The problem of calculating the lower bound of the final policy performance is transformed into the following mixed integer programming problem: （12） in This refers to the vector of all specific variables involved.
4. A computer system comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1 to 3.
5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 3.