Model evaluation method and apparatus, and computer device, storage medium and program product
By generating adversarial examples and conducting adversarial attacks, the robustness of the load forecasting model is evaluated, which solves the problem of robustness evaluation of deep learning models in load forecasting and improves the model's forecasting accuracy and the balance of power supply.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- ELECTRIC POWER RES INST CHINA SOUTHERN POWER GRID CO LTD
- Filing Date
- 2025-08-22
- Publication Date
- 2026-06-11
AI Technical Summary
How to robustly evaluate deep learning models used for load forecasting to ensure a balance between power supply and demand.
By generating adversarial examples and conducting adversarial attacks, the robustness metrics of the load forecasting model are evaluated, including the success rate of adversarial attacks, interference sensitivity, and the magnitude of adversarial example perturbations, in order to optimize the model's robustness.
This improves the accuracy of load forecasting models in load forecasting, ensuring a balance between power supply and demand.
Smart Images
Figure CN2025116353_11062026_PF_FP_ABST
Abstract
Description
Model evaluation methods, apparatus, computer equipment, storage media and program products
[0001] Related applications
[0002] This application claims priority to Chinese patent application filed on December 3, 2024, application number 2024117578756, entitled "Model Evaluation Method, Apparatus, Computer Equipment, Storage Medium and Program Product", the entire contents of which are incorporated herein by reference. Technical Field
[0003] This application relates to the field of robustness assessment technology, and in particular to a model assessment method, apparatus, computer equipment, storage medium, and program product. Background Technology
[0004] Load forecasting plays a crucial role in power system planning and operation. Essentially, load forecasting is the prediction of electricity market demand. By analyzing and studying historical data, the load demand for electricity is predicted, and the required electricity supply is determined based on the predicted load.
[0005] Currently, load forecasting is typically based on pre-trained deep learning models to determine the required electricity supply based on the predicted load, thereby achieving a balance between power supply and demand. Therefore, the robustness of deep learning models is crucial. Consequently, how to evaluate the robustness of deep learning models used for load forecasting has become a pressing technical problem in this field. Summary of the Invention
[0006] Based on this, this application provides a model evaluation method, apparatus, computer device, storage medium, and program product capable of robustly evaluating deep learning models used for load forecasting.
[0007] Firstly, this application provides a model evaluation method. The method includes:
[0008] Based on sample generation constraints and historical load monitoring data of the power system, adversarial examples are generated.
[0009] The adversarial sample is input into the load forecasting model to perform an adversarial attack, so as to obtain the load forecast sample corresponding to the adversarial sample;
[0010] Based on the load forecast sample, evaluate the robustness index of the load forecast model; the robustness index is used to evaluate the robustness of the load forecast model.
[0011] In one embodiment, evaluating the robustness metrics of the load forecasting model based on the load forecasting sample includes:
[0012] Based on the first number of adversarial samples, the load prediction samples, and the initial load prediction data corresponding to the historical load monitoring data, the interference sensitivity is determined; the robustness index includes the interference sensitivity.
[0013] In one embodiment, evaluating the robustness metrics of the load forecasting model based on the load forecasting sample includes:
[0014] Based on the first number of adversarial samples, the historical load monitoring data, and the adversarial samples, the perturbation amplitude of the adversarial samples is determined; the robustness index includes the perturbation amplitude of the adversarial samples.
[0015] In one embodiment, the generation of adversarial examples based on sample generation constraints and historical load monitoring data of the power system includes:
[0016] Using at least one adversarial example generation algorithm, the adversarial example is generated based on the historical load monitoring data under the constraints of the example generation.
[0017] In one embodiment, the sample generation constraints include target domain constraints, and the method further includes:
[0018] Identify at least one initial domain constraint related to the historical load monitoring data;
[0019] For each of the initial domain constraints, determine whether the historical load monitoring data meets the initial domain constraints;
[0020] If the historical load monitoring data satisfies the initial domain constraint, then the initial domain constraint is determined as the target domain constraint.
[0021] In one embodiment, evaluating the robustness metrics of the load forecasting model based on the load forecasting sample includes:
[0022] When a new adversarial sample generation algorithm is detected, the algorithm is used to generate new adversarial samples based on the historical load monitoring data, under the constraints of sample generation.
[0023] The new adversarial sample is input into the load forecasting model to perform an adversarial attack, so as to obtain a new load forecasting sample corresponding to the new adversarial sample;
[0024] Based on the existing load forecast sample and the new load forecast sample, evaluate the robustness index of the load forecasting model.
[0025] Secondly, this application also provides a model evaluation apparatus. The apparatus includes:
[0026] The generation module is used to generate adversarial examples based on sample generation constraints and historical load monitoring data of the power system.
[0027] The adversarial attack module is used to input the adversarial sample into the load forecasting model to perform an adversarial attack in order to obtain the load forecasting sample corresponding to the adversarial sample.
[0028] An evaluation module is used to evaluate the robustness index of the load forecasting model based on the load forecasting sample; the robustness index is used to evaluate the robustness of the load forecasting model.
[0029] Thirdly, this application also provides a computer device, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of any of the above methods.
[0030] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above methods.
[0031] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above methods.
[0032] The aforementioned model evaluation method, apparatus, computer equipment, storage medium, and program product generate adversarial examples based on sample generation constraints and historical load monitoring data of the power system. These adversarial examples are then input into the load forecasting model to perform adversarial attacks, resulting in load forecasting samples corresponding to the adversarial examples. Based on these load forecasting samples, the robustness index of the load forecasting model is evaluated, thereby assessing the robustness of the load forecasting model. Furthermore, when the robustness index indicates poor robustness of the load forecasting model, the model can be optimized to improve its accuracy in load forecasting.
[0033] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the disclosed drawings without creative effort.
[0035] Figure 1 is an internal structural diagram of a computer device provided in an embodiment of this application;
[0036] Figure 2 is a flowchart illustrating a model evaluation method provided in an embodiment of this application;
[0037] Figure 3 is a flowchart illustrating a method for determining target domain constraints according to an embodiment of this application;
[0038] Figure 4 is a flowchart illustrating a robustness index evaluation method provided in an embodiment of this application;
[0039] Figure 5 is a flowchart illustrating a robustness evaluation method for a load forecasting model provided in an embodiment of this application.
[0040] Figure 6 is a structural block diagram of a model evaluation device provided in an embodiment of this application. Detailed Implementation
[0041] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0042] Load forecasting plays a crucial role in power system planning and operation. Essentially, load forecasting is the prediction of electricity market demand. By analyzing and studying historical data, the load demand for electricity is predicted, and the required electricity supply is determined based on the predicted load.
[0043] Currently, load forecasting is typically based on pre-trained deep learning models to determine the required electricity supply based on the predicted load, thereby achieving a balance between power supply and demand. Therefore, the robustness of deep learning models is crucial. Consequently, how to evaluate the robustness of deep learning models used for load forecasting has become a pressing technical problem in this field.
[0044] The model evaluation method provided in this application embodiment can be applied to the application environment shown in Figure 1. Figure 1 is an internal structure diagram of a computer device provided in this application embodiment. The computer device can be a server, and its internal structure diagram is as shown in Figure 1. The computer device includes a processor, memory, and network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a model evaluation method.
[0045] Those skilled in the art will understand that the structure shown in Figure 1 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or may combine certain components, or may have different component arrangements.
[0046] In one embodiment, as shown in FIG2, FIG2 is a schematic flowchart of a model evaluation method provided by an embodiment of the present application. The method can be applied to the computer device in FIG1, and the method includes the following steps:
[0047] S201 generates adversarial examples based on sample generation constraints and historical load monitoring data of the power system.
[0048] For example, historical load monitoring data can be real load monitoring data obtained through sensors, monitoring systems, historical databases, etc., or it can be historical load data obtained by correcting historical load prediction samples output by load prediction models.
[0049] Optionally, the acquired historical load monitoring data can be preprocessed, including data cleaning, removal of missing and outlier values, standardization and normalization of data, and then adversarial examples can be generated based on the processed historical load monitoring data.
[0050] In this embodiment, the sample generation constraints may include, for example, target domain constraints, preset power system constraints, and adversarial constraints. The target domain constraints are used to constrain the domain to which the generated adversarial sample belongs, so that the generated adversarial sample meets the requirements of a specific domain.
[0051] In addition, artificial rules related to power load forecasting formulated by artificial intelligence experts in the power system can be obtained, and then the artificial rules can be transformed into corresponding Boolean constraint forms to obtain the aforementioned preset power system constraints.
[0052] In one embodiment, at least one adversarial example generation algorithm can be used to apply small perturbation samples to historical load monitoring data under sample generation constraints in order to obtain adversarial examples.
[0053] Alternatively, adversarial example generation algorithms may include, for example, Fast Signed Gradient (FGSM), Projected Gradient Descent (PGD), Carlini and Wagner attacks (C&W), etc.
[0054] S202, the adversarial sample is input into the load prediction model to perform an adversarial attack in order to obtain the load prediction sample corresponding to the adversarial sample.
[0055] In one embodiment, adversarial samples can be input into the load forecasting model to perform adversarial attacks on the load forecasting model using small perturbation samples, and then the load forecasting samples corresponding to the adversarial samples output by the load forecasting model can be obtained.
[0056] S203, Evaluate the robustness index of the load forecasting model based on the load forecasting sample.
[0057] Among them, the robustness index is used to evaluate the robustness of the load forecasting model.
[0058] Optionally, robustness metrics may include at least one of the following: Attack Success Rate (ASR), Perturbation Sensitivity (PS), and Perturbation Magnitude (PM).
[0059] The adversarial attack success rate is used to characterize the proportion of adversarial samples that successfully defeat the adversarial attack out of all adversarial samples. Successful adversarial attack means that the adversarial sample causes the output error of the load prediction model to exceed a preset threshold.
[0060] In one embodiment, the Adversarial Sweep (ASR) success rate can be calculated using the following formula (1):
[0061] In formula (1), n is the number of adversarial samples that were successfully countered, and sum is the total number of adversarial samples.
[0062] In one possible implementation, the robustness of the load forecasting model can be evaluated based on a robustness index. If the robustness index indicates that the load forecasting model is not robust, the load forecasting model can be optimized to improve its robustness and thus improve the accuracy of load forecasting.
[0063] For example, the success rate of adversarial attacks can be calculated, and then used as a robustness metric to evaluate the robustness of the load forecasting model. Alternatively, the perturbation sensitivity can be calculated, and then used as a robustness metric to evaluate the robustness of the load forecasting model. Alternatively, the magnitude of adversarial sample perturbations can be calculated, and then used as a robustness metric to evaluate the robustness of the load forecasting model.
[0064] In addition, the success rate of adversarial attacks, interference sensitivity, and adversarial sample perturbation amplitude can be calculated. Then, the success rate of adversarial attacks, interference sensitivity, and adversarial sample perturbation amplitude can be used together as robustness indicators to evaluate the robustness of the load forecasting model.
[0065] In this embodiment, adversarial samples are generated based on sample generation constraints and historical load monitoring data of the power system. These adversarial samples are then input into the load forecasting model for adversarial attacks to obtain load forecasting samples corresponding to the adversarial samples. Based on the load forecasting samples, the robustness index of the load forecasting model is evaluated, thereby assessing the robustness of the load forecasting model. Furthermore, when the robustness index shows that the load forecasting model has poor robustness, the load forecasting model can be optimized to improve the accuracy of the load forecasting model in load forecasting.
[0066] Based on the above embodiments, S203 may further include the following steps:
[0067] The interference sensitivity is determined based on the first number of adversarial samples, the load prediction samples, and the initial load prediction data corresponding to the historical load monitoring data.
[0068] Among them, robustness indicators include interference sensitivity.
[0069] In this embodiment, the interference sensitivity is used to characterize the sensitivity of the load forecasting model to input disturbances, that is, the change in the output of the load forecasting model in response to adversarial examples relative to the output of historical load monitoring data.
[0070] For example, the interference sensitivity PS can be determined using the following formula (2) based on the first number of adversarial examples, the load prediction samples, and the initial load prediction data corresponding to the historical load monitoring data:
[0071] In formula (2), N is the total number of adversarial samples, i.e., the first quantity, f(x i (x) represents historical load monitoring data. i The corresponding initial load forecast data, For adversarial examples The corresponding load forecast sample.
[0072] In this embodiment, based on the first number of adversarial samples, load prediction samples, and the initial load prediction data corresponding to historical load monitoring data, the interference sensitivity is determined. This allows for the evaluation of the robustness of the load prediction model based on the interference sensitivity. Consequently, when the interference sensitivity indicates poor robustness of the load prediction model, the load prediction model can be optimized to improve its accuracy in load prediction.
[0073] Based on the above embodiments, S203 may further include the following steps:
[0074] Based on the initial number of adversarial examples, historical load monitoring data, and adversarial examples, the perturbation amplitude of the adversarial examples is determined.
[0075] Among them, robustness metrics include the magnitude of adversarial sample perturbations.
[0076] In this embodiment, the perturbation amplitude of the adversarial sample is used to characterize the degree of change of the adversarial sample relative to the original sample.
[0077] For example, the adversarial sample perturbation magnitude PM can be calculated using the L2 norm, i.e., the following formula (3). L2 :
[0078] Alternatively, the adversarial sample perturbation magnitude PM can be calculated using the L∞ norm, i.e., the following formula (4). L∞ :
[0079] In formulas (3) and (4), N is the total number of adversarial examples, and x i This is historical load monitoring data. For adversarial examples.
[0080] In this embodiment, the adversarial sample disturbance amplitude is determined based on the first number of adversarial samples, historical load monitoring data, and adversarial samples. This allows for the evaluation of the robustness of the load forecasting model based on the adversarial sample disturbance amplitude. Consequently, when the adversarial sample disturbance amplitude indicates poor robustness of the load forecasting model, the load forecasting model can be optimized to improve its accuracy in load forecasting.
[0081] Based on the above embodiments, S201 may further include the following steps:
[0082] Using at least one adversarial example generation algorithm, adversarial examples are generated based on historical load monitoring data under sample generation constraints.
[0083] For example, adversarial example generation algorithms may include Fast Signed Gradient (FGSM), Projected Gradient Descent (PGD), Carlini and Wagner attacks (C&W), etc.
[0084] Preferably, based on multiple adversarial example generation algorithms, under adversarial constraints, small perturbation samples are applied to historical load monitoring data to obtain intermediate adversarial samples. Then, the intermediate adversarial samples that do not conform to the target domain constraints are projected into the constraint space through the DPLL (Davis-Putnam-Logemann-Loveland) algorithm to obtain adversarial samples.
[0085] In one possible implementation, if adversarial samples are generated based on multiple adversarial sample generation algorithms, the various adversarial sample generation algorithms can be uniformly managed and standardized to improve the consistency of adversarial samples.
[0086] In this embodiment, at least one adversarial example generation algorithm is used to generate adversarial examples based on historical load monitoring data under sample generation constraints, thereby improving the diversity of adversarial examples and thus improving the comprehensiveness and accuracy of robustness assessment.
[0087] Referring to Figure 3, which is a flowchart illustrating a method for determining target domain constraints according to an embodiment of this application, the method further includes the following steps, based on the above embodiment, where sample generation constraints include target domain constraints:
[0088] S301, determine at least one initial domain constraint related to historical load monitoring data.
[0089] In one embodiment, initial domain constraints related to historical load monitoring data can be enumerated. Assuming there are n features related to historical load monitoring data, then theoretically, the order of magnitude of the combination of initial domain constraints related to historical load monitoring data is O(2n).
[0090] For example, if features A and B are associated with historical load monitoring data, then the initial domain constraints associated with the historical load monitoring data may include A and B, A or B, not A and B, and not A or not B. These possible combinations of initial domain constraints constitute the initial constraint space T.
[0091] S302, for each initial domain constraint, determine whether the historical load monitoring data meets the initial domain constraint.
[0092] For example, the values of feature A and feature B corresponding to each historical load monitoring data can be substituted into the aforementioned initial domain constraints for checking. If the result is False, the historical load monitoring data does not meet the initial domain constraints; if the result is True, the historical load monitoring data meets the initial domain constraints.
[0093] S303, if the historical load monitoring data meets the initial domain constraint, then the initial domain constraint is determined as the target domain constraint.
[0094] For example, the initial domain constraints that are not satisfied by historical load monitoring data can be deleted, and then the initial domain constraints that are satisfied by historical load monitoring data can be identified as target domain constraints.
[0095] It should be noted that the target domain constraint should be the set of all initial domain constraints satisfied by the historical load monitoring data.
[0096] In this embodiment, at least one initial domain constraint related to historical load monitoring data is determined. For each initial domain constraint, it is determined whether the historical load monitoring data satisfies the initial domain constraint. If the historical load monitoring data satisfies the initial domain constraint, the initial domain constraint is determined as the target domain constraint. This enables the generation of adversarial samples that meet specific domain requirements based on the target domain constraint, improving the targeting of adversarial attacks and further enhancing the accuracy of robustness assessment.
[0097] Referring to Figure 4, which is a flowchart illustrating a robustness index evaluation method provided in an embodiment of this application, this embodiment relates to a possible implementation of how to evaluate the robustness index of a load forecasting model based on load forecasting samples. Based on the above embodiment, S203 includes the following steps:
[0098] S401, when a new adversarial sample generation algorithm is detected, the new adversarial sample generation algorithm is used to generate new adversarial samples based on historical load monitoring data under the condition of sample generation constraints.
[0099] S402, input the new adversarial sample into the load prediction model to perform an adversarial attack, so as to obtain the new load prediction sample corresponding to the new adversarial sample.
[0100] In one embodiment, a monitoring module can be set up to acquire updates to the adversarial example generation algorithm at preset intervals. When a new adversarial example generation algorithm is detected, it is used to generate new adversarial examples based on historical load monitoring data, under sample generation constraints. The new adversarial examples are then input into the load prediction model for adversarial attack to obtain new load prediction samples corresponding to the new adversarial examples.
[0101] S403, Evaluate the robustness index of the load forecasting model based on the load forecast sample and the new load forecast sample.
[0102] In one embodiment, when a new adversarial example generation algorithm is detected, a robustness index of the load forecasting model can be calculated based on both the load forecasting sample and the new load forecasting sample, so as to evaluate the robustness of the load forecasting model through the robustness index.
[0103] In another implementation, when a new adversarial example generation algorithm is detected, it can be used to generate new adversarial examples based on historical load monitoring data, under sample generation constraints. These new adversarial examples, along with existing adversarial examples, are then input into the load forecasting model for an adversarial attack to obtain new load forecasting samples. The robustness of the load forecasting model is then evaluated based on these new load forecasting samples.
[0104] In this embodiment, when a new adversarial sample generation algorithm is detected, the algorithm is used to generate new adversarial samples based on historical load monitoring data under sample generation constraints. The new adversarial samples are then input into the load prediction model for adversarial attacks to obtain new load prediction samples corresponding to the new adversarial samples. Based on the load prediction samples and the new load prediction samples, the robustness index of the load prediction model is evaluated. This enables dynamic updating of the adversarial sample generation algorithm and the use of the latest adversarial sample generation algorithm to generate new adversarial samples. This ensures that the latest adversarial sample generation technology is always integrated during the robustness evaluation process, allowing the model to keep pace with the latest attack methods and overcoming the limitations of existing technologies in the static evaluation process.
[0105] Referring to Figure 5, which is a flowchart illustrating a robustness evaluation method for a load forecasting model provided in an embodiment of this application, the method includes the following steps:
[0106] S501, determine at least one initial domain constraint related to historical load monitoring data.
[0107] S502, for each initial domain constraint, determine whether the historical load monitoring data meets the initial domain constraint.
[0108] S503, if the historical load monitoring data meets the initial domain constraint, then the initial domain constraint is determined as the target domain constraint.
[0109] S504, using at least one adversarial example generation algorithm, adversarial examples are generated based on historical load monitoring data under sample generation constraints. These constraints include target domain constraints.
[0110] S505 inputs the adversarial sample into the load forecasting model to perform an adversarial attack in order to obtain the load forecasting sample corresponding to the adversarial sample.
[0111] S506, Evaluate the robustness index of the load forecasting model based on load forecasting samples.
[0112] S507: When a new adversarial sample generation algorithm is detected, the new adversarial sample generation algorithm is used to generate new adversarial samples based on historical load monitoring data under the condition of sample generation constraints.
[0113] S508, input the new adversarial sample into the load prediction model to perform an adversarial attack, so as to obtain the new load prediction sample corresponding to the new adversarial sample.
[0114] S509, Evaluate the robustness index of the load forecasting model based on the load forecast sample and the new load forecast sample.
[0115] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0116] Based on the same inventive concept, this application also provides a model evaluation apparatus for implementing the model evaluation method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more model evaluation apparatus embodiments provided below can be found in the limitations of the model evaluation method described above, and will not be repeated here.
[0117] In one embodiment, as shown in FIG6, FIG6 is a structural block diagram of a model evaluation device provided in an embodiment of the present application. The device 600 includes:
[0118] The generation module 601 is used to generate adversarial examples based on sample generation constraints and historical load monitoring data of the power system.
[0119] The adversarial attack module 602 is used to input adversarial samples into the load prediction model to perform adversarial attacks in order to obtain load prediction samples corresponding to the adversarial samples.
[0120] Evaluation module 603 is used to evaluate the robustness index of the load forecasting model based on the load forecasting sample; the robustness index is used to evaluate the robustness of the load forecasting model.
[0121] In one embodiment, the evaluation module 603 includes:
[0122] The first determining unit is used to determine the interference sensitivity based on the first number of adversarial examples, the load prediction samples, and the initial load prediction data corresponding to the historical load monitoring data; the robustness index includes interference sensitivity.
[0123] In one embodiment, the evaluation module 603 includes:
[0124] The second determining unit is used to determine the perturbation amplitude of adversarial samples based on the first number of adversarial samples, historical load monitoring data, and adversarial samples; the robustness index includes the perturbation amplitude of adversarial samples.
[0125] In one embodiment, the generation module 601 includes:
[0126] The first generation unit is used to generate adversarial examples based on historical load monitoring data, under the condition of sample generation constraints, using at least one adversarial example generation algorithm.
[0127] In one embodiment, the sample generation constraints include target neighborhood constraints, and the device 600 further includes:
[0128] The first determining module is used to determine at least one initial domain constraint related to historical load monitoring data.
[0129] The judgment module is used to determine whether the historical load monitoring data meets the initial domain constraints for each initial domain constraint.
[0130] The second determining module is used to determine the initial domain constraint as the target domain constraint if the historical load monitoring data meets the initial domain constraint.
[0131] In one embodiment, the evaluation module 603 includes:
[0132] The second generation unit is used to generate new adversarial samples based on historical load monitoring data when a new adversarial sample generation algorithm is detected, under the condition of sample generation constraints.
[0133] The adversarial attack unit is used to input new adversarial samples into the load forecasting model to perform adversarial attacks in order to obtain new load forecasting samples corresponding to the new adversarial samples.
[0134] The evaluation unit is used to evaluate the robustness indicators of the load forecasting model based on the load forecast sample and the new load forecast sample.
[0135] Each module in the aforementioned model evaluation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.
[0136] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0137] Based on sample generation constraints and historical load monitoring data of the power system, adversarial examples are generated.
[0138] The adversarial sample is input into the load forecasting model to perform an adversarial attack, so as to obtain the load forecast sample corresponding to the adversarial sample;
[0139] Based on the load forecasting samples, the robustness index of the load forecasting model is evaluated; the robustness index is used to evaluate the robustness of the load forecasting model.
[0140] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0141] Based on the first number of adversarial examples, load forecast samples, and initial load forecast data corresponding to historical load monitoring data, the interference sensitivity is determined; robustness indicators include interference sensitivity.
[0142] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0143] Based on the initial number of adversarial examples, historical load monitoring data, and adversarial examples, the perturbation amplitude of adversarial examples is determined; robustness indicators include the perturbation amplitude of adversarial examples.
[0144] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0145] Using at least one adversarial example generation algorithm, adversarial examples are generated based on historical load monitoring data under sample generation constraints.
[0146] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0147] Identify at least one initial domain constraint related to historical load monitoring data;
[0148] For each initial domain constraint, determine whether the historical load monitoring data meets the initial domain constraint;
[0149] If the historical load monitoring data meets the initial domain constraints, then the initial domain constraints will be determined as the target domain constraints.
[0150] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0151] When a new adversarial sample generation algorithm is detected, the algorithm is used to generate new adversarial samples based on historical load monitoring data, under the constraints of sample generation.
[0152] The new adversarial sample is input into the load forecasting model to perform an adversarial attack, so as to obtain a new load forecast sample corresponding to the new adversarial sample;
[0153] The robustness index of the load forecasting model is evaluated based on the load forecast sample and the new load forecast sample.
[0154] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0155] Based on sample generation constraints and historical load monitoring data of the power system, adversarial examples are generated.
[0156] The adversarial sample is input into the load forecasting model to perform an adversarial attack, so as to obtain the load forecast sample corresponding to the adversarial sample;
[0157] Based on the load forecasting samples, the robustness index of the load forecasting model is evaluated; the robustness index is used to evaluate the robustness of the load forecasting model.
[0158] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0159] Based on the first number of adversarial examples, load forecast samples, and initial load forecast data corresponding to historical load monitoring data, the interference sensitivity is determined; robustness indicators include interference sensitivity.
[0160] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0161] Based on the initial number of adversarial examples, historical load monitoring data, and adversarial examples, the perturbation amplitude of adversarial examples is determined; robustness indicators include the perturbation amplitude of adversarial examples.
[0162] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0163] Using at least one adversarial example generation algorithm, adversarial examples are generated based on historical load monitoring data under sample generation constraints.
[0164] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0165] Identify at least one initial domain constraint related to historical load monitoring data;
[0166] For each initial domain constraint, determine whether the historical load monitoring data meets the initial domain constraint;
[0167] If the historical load monitoring data meets the initial domain constraints, then the initial domain constraints will be determined as the target domain constraints.
[0168] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0169] When a new adversarial sample generation algorithm is detected, the algorithm is used to generate new adversarial samples based on historical load monitoring data, under the constraints of sample generation.
[0170] The new adversarial sample is input into the load forecasting model to perform an adversarial attack, so as to obtain a new load forecast sample corresponding to the new adversarial sample;
[0171] The robustness index of the load forecasting model is evaluated based on the load forecast sample and the new load forecast sample.
[0172] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:
[0173] Based on sample generation constraints and historical load monitoring data of the power system, adversarial examples are generated.
[0174] The adversarial sample is input into the load forecasting model to perform an adversarial attack, so as to obtain the load forecast sample corresponding to the adversarial sample;
[0175] Based on the load forecasting samples, the robustness index of the load forecasting model is evaluated; the robustness index is used to evaluate the robustness of the load forecasting model.
[0176] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0177] Based on the first number of adversarial examples, load forecast samples, and initial load forecast data corresponding to historical load monitoring data, the interference sensitivity is determined; robustness indicators include interference sensitivity.
[0178] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0179] Based on the initial number of adversarial examples, historical load monitoring data, and adversarial examples, the perturbation amplitude of adversarial examples is determined; robustness indicators include the perturbation amplitude of adversarial examples.
[0180] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0181] Using at least one adversarial example generation algorithm, adversarial examples are generated based on historical load monitoring data under sample generation constraints.
[0182] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0183] Identify at least one initial domain constraint related to historical load monitoring data;
[0184] For each initial domain constraint, determine whether the historical load monitoring data meets the initial domain constraint;
[0185] If the historical load monitoring data meets the initial domain constraints, then the initial domain constraints will be determined as the target domain constraints.
[0186] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0187] When a new adversarial sample generation algorithm is detected, the algorithm is used to generate new adversarial samples based on historical load monitoring data, under the constraints of sample generation.
[0188] The new adversarial sample is input into the load forecasting model to perform an adversarial attack, so as to obtain a new load forecast sample corresponding to the new adversarial sample;
[0189] The robustness index of the load forecasting model is evaluated based on the load forecast sample and the new load forecast sample.
[0190] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0191] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0192] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A model evaluation method, wherein, The method includes: Based on sample generation constraints and historical load monitoring data of the power system, adversarial examples are generated. The adversarial sample is input into the load prediction model to perform an adversarial attack, so as to obtain the load prediction sample corresponding to the adversarial sample; Based on the load forecasting samples, the robustness index of the load forecasting model is evaluated; the robustness index is used to evaluate the robustness of the load forecasting model.
2. The method according to claim 1, wherein, The step of evaluating the robustness index of the load forecasting model based on the load forecasting sample includes: Based on the first number of adversarial samples, the load prediction samples, and the initial load prediction data corresponding to the historical load monitoring data, the interference sensitivity is determined; the robustness index includes the interference sensitivity.
3. The method according to claim 1 or 2, wherein, The step of evaluating the robustness index of the load forecasting model based on the load forecasting sample includes: Based on the first number of adversarial samples, the historical load monitoring data, and the adversarial samples, the perturbation amplitude of the adversarial samples is determined; the robustness index includes the perturbation amplitude of the adversarial samples.
4. The method according to claim 1 or 2, wherein, The adversarial examples generated based on sample generation constraints and historical load monitoring data of the power system include: Using at least one adversarial example generation algorithm, the adversarial examples are generated based on the historical load monitoring data under the constraints of the example generation.
5. The method according to claim 1 or 2, wherein, The sample generation constraints include target domain constraints, and the method further includes: Determine at least one initial domain constraint related to the historical load monitoring data; For each of the initial domain constraints, determine whether the historical load monitoring data satisfies the initial domain constraints; If the historical load monitoring data satisfies the initial domain constraint, then the initial domain constraint is determined as the target domain constraint.
6. The method according to claim 4, wherein, The step of evaluating the robustness index of the load forecasting model based on the load forecasting sample includes: When a new adversarial sample generation algorithm is detected, the new adversarial sample is generated based on the historical load monitoring data under the constraints of the sample generation, using the new adversarial sample generation algorithm. The new adversarial sample is input into the load prediction model to perform an adversarial attack, so as to obtain a new load prediction sample corresponding to the new adversarial sample; The robustness index of the load forecasting model is evaluated based on the load forecasting sample and the new load forecasting sample.
7. A model evaluation device, wherein, The device includes: The generation module is used to generate adversarial examples based on sample generation constraints and historical load monitoring data of the power system. The adversarial attack module is used to input the adversarial sample into the load prediction model to perform an adversarial attack, so as to obtain the load prediction sample corresponding to the adversarial sample; An evaluation module is used to evaluate the robustness index of the load forecasting model based on the load forecasting sample; the robustness index is used to evaluate the robustness of the load forecasting model.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein... When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.