Model evaluation method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a model evaluation method and device, computer equipment, a storage medium and a program product. The method comprises the steps of generating an adversarial sample through historical load monitoring data based on a sample generation constraint and a power system, inputting the adversarial sample into a load prediction model for adversarial attack to obtain a load prediction sample corresponding to the adversarial sample, and evaluating a robustness index of the load prediction model according to the load prediction sample to obtain a load prediction result. Therefore, the robustness of the load prediction model is evaluated, and the load prediction model can be optimized when the robustness index shows that the robustness of the load prediction model is poor, so that the accuracy of the load prediction model during load prediction is improved.

Description

Technical Field

[0001] The present application relates to the technical field of robustness evaluation, and in particular to a model evaluation method, apparatus, computer equipment, storage medium and program product. Background Art

[0002] Load forecasting plays a vital role in power system planning and operation. Load forecasting is essentially a forecast of electricity market demand. Through the analysis and research of historical data, the load of electricity demand is predicted, and the required amount of electricity supply is determined based on the predicted load.

[0003] At present, load forecasting is usually performed based on pre-trained deep learning models, so that the required amount of electricity is determined based on the predicted load to achieve a balance between power supply and demand. Therefore, the robustness of the deep learning model is crucial. Therefore, how to evaluate the robustness of the deep learning model used for load forecasting has become a technical problem that needs to be solved urgently in this field. Summary of the invention

[0004] Based on this, it is necessary to provide a model evaluation method, device, computer equipment, storage medium and program product that can perform robustness evaluation on deep learning models used for load forecasting in response to the above technical problems.

[0005] In a first aspect, the present application provides a model evaluation method. The method comprises:

[0006] Generate adversarial samples based on sample generation constraints and historical load monitoring data of the power system;

[0007] Input the adversarial sample into the load forecasting model to perform adversarial attack to obtain a load forecasting sample corresponding to the adversarial sample;

[0008] According to the load forecasting sample, a robustness index of the load forecasting model is evaluated; the robustness index is used to evaluate the robustness of the load forecasting model.

[0009] In one embodiment, the robustness index of the load forecasting model is evaluated according to the load forecasting sample, including:

[0010] Based on the first number of adversarial samples, the load forecast samples, and the initial load forecast data corresponding to the historical load monitoring data, interference sensitivity is determined; the robustness indicator includes the interference sensitivity.

[0011] In one embodiment, the robustness index of the load forecasting model is evaluated according to the load forecasting sample, including:

[0012] Based on the first number of adversarial samples, the historical load monitoring data, and the adversarial samples, the adversarial sample perturbation amplitude is determined; and the robustness indicator includes the adversarial sample perturbation amplitude.

[0013] In one embodiment, the method generates adversarial samples based on sample generation constraints and historical load monitoring data of the power system, including:

[0014] At least one adversarial sample generation algorithm is used to generate the adversarial sample based on the historical load monitoring data under the condition of the sample generation constraints.

[0015] In one embodiment, the sample generation constraint includes a target domain constraint, and the method further includes:

[0016] determining at least one initial domain constraint associated with the historical load monitoring data;

[0017] For each of the initial domain constraints, determining whether the historical load monitoring data satisfies the initial domain constraint;

[0018] If the historical load monitoring data satisfies the initial domain constraint, the initial domain constraint is determined as the target domain constraint.

[0019] In one embodiment, the robustness index of the load forecasting model is evaluated according to the load forecasting sample, including:

[0020] When a new adversarial sample generation algorithm is detected, using the new adversarial sample generation algorithm to generate a new adversarial sample based on the historical load monitoring data under the constraints of the sample generation;

[0021] Input the new adversarial sample into the load forecasting model for adversarial attack to obtain a new load forecasting sample corresponding to the new adversarial sample;

[0022] The robustness index of the load forecasting model is evaluated based on the load forecasting sample and the new load forecasting sample.

[0023] In a second aspect, the present application also provides a model evaluation device. The device comprises:

[0024] A generation module, used to generate adversarial samples based on sample generation constraints and historical load monitoring data of the power system;

[0025] An adversarial attack module, used for inputting the adversarial sample into the load forecasting model for adversarial attack, so as to obtain a load forecasting sample corresponding to the adversarial sample;

[0026] The evaluation module is used to evaluate the robustness index of the load forecasting model according to the load forecasting sample; the robustness index is used to evaluate the robustness of the load forecasting model.

[0027] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0028] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0029] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of any of the above methods when executed by a processor.

[0030] The above-mentioned model evaluation method, device, computer equipment, storage medium and program product generate adversarial samples based on sample generation constraints and historical load monitoring data of the power system, input the adversarial samples into the load forecasting model for adversarial attack to obtain load forecasting samples corresponding to the adversarial samples, and evaluate the robustness index of the load forecasting model based on the load forecasting samples, thereby realizing the evaluation of the robustness of the load forecasting model, and then when the robustness index shows that the robustness of the load forecasting model is poor, the load forecasting model can be optimized to improve the accuracy of the load forecasting model in load forecasting. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is an internal structure diagram of a computer device provided in an embodiment of the present application;

[0032] Figure 2 It is a flow chart of a model evaluation method provided in an embodiment of the present application;

[0033] Figure 3 It is a flowchart of a method for determining target domain constraints provided in an embodiment of the present application;

[0034] Figure 4 It is a flowchart of a robustness index evaluation method provided in an embodiment of the present application;

[0035] Figure 5 It is a flow chart of a method for evaluating the robustness of a load forecasting model provided in an embodiment of the present application;

[0036] Figure 6 It is a structural block diagram of a model evaluation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0038] Load forecasting plays a vital role in power system planning and operation. Load forecasting is essentially a forecast of electricity market demand. Through the analysis and research of historical data, the load of electricity demand is predicted, and the required amount of electricity supply is determined based on the predicted load.

[0039] At present, load forecasting is usually performed based on pre-trained deep learning models, so that the required amount of electricity is determined based on the predicted load to achieve a balance between power supply and demand. Therefore, the robustness of the deep learning model is crucial. Therefore, how to evaluate the robustness of the deep learning model used for load forecasting has become a technical problem that needs to be solved urgently in this field.

[0040] The model evaluation method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Figure 1 is an internal structure diagram of a computer device provided in an embodiment of the present application. The computer device may be a server, and its internal structure diagram may be as follows Figure 1 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a model evaluation method is implemented.

[0041] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0042] In one embodiment, Figure 2 As shown, Figure 2 is a flow chart of a model evaluation method provided in an embodiment of the present application, which can be applied to Figure 1 The computer device comprises the following steps:

[0043] S201, generating adversarial samples based on sample generation constraints and historical load monitoring data of the power system.

[0044] Exemplarily, the historical load monitoring data may be real load monitoring data acquired through sensors, monitoring systems, historical databases, etc., or may be historical load data obtained by correcting historical load forecasting samples output by a load forecasting model.

[0045] Optionally, the acquired historical load monitoring data may be preprocessed, including data cleaning, removal of missing values ​​and outliers, standardization and normalization of data, etc., and then adversarial samples may be generated based on the processed historical load monitoring data.

[0046] In the embodiment of the present application, sample generation constraints may include, for example, target domain constraints, preset power system constraints, and adversarial constraints. Among them, the target domain constraints are used to constrain the domain to which the generated adversarial sample belongs, so that the generated adversarial sample meets the requirements of the specific domain.

[0047] In addition, artificial rules related to power load forecasting developed by artificial intelligence experts of the power system can be obtained, and then the artificial rules can be converted into corresponding Boolean constraint forms to obtain the above-mentioned preset power system constraints.

[0048] In one embodiment, at least one adversarial sample generation algorithm may be used to apply a small perturbation sample to the historical load monitoring data under the condition of sample generation constraints to obtain an adversarial sample.

[0049] Optionally, the adversarial sample generation algorithm may include, for example, the fast gradient sign method (FGSM), the projected gradient descent method (PGD), the Carlini and Wagner attack (C&W), etc.

[0050] S202: Input the adversarial sample into the load forecasting model for adversarial attack to obtain a load forecasting sample corresponding to the adversarial sample.

[0051] In one embodiment, an adversarial sample may be input into a load forecasting model, and an adversarial attack may be performed on the load forecasting model through a tiny disturbance sample, and then a load forecasting sample corresponding to the adversarial sample output by the load forecasting model may be obtained.

[0052] S203: Evaluate the robustness index of the load forecasting model based on the load forecasting samples.

[0053] Among them, the robustness index is used to evaluate the robustness of the load forecasting model.

[0054] Optionally, the robustness indicator may include, for example, at least one of an adversarial attack success rate (ASR), a perturbation sensitivity (PS), and an adversarial sample perturbation magnitude (PM).

[0055] Among them, the adversarial attack success rate is used to characterize the proportion of successful adversarial samples to all adversarial samples. A successful adversarial attack means that the output error of the load forecasting model exceeds the preset threshold.

[0056] In one embodiment, the adversarial attack success rate ASR can be calculated by the following formula (1):

[0057] (1)

[0058] In formula (1), n ​​is the number of adversarial samples that are successfully countered, and sum is the total number of adversarial samples.

[0059] In one possible implementation, the robustness of the load forecasting model can be evaluated based on a robustness index. If the robustness index indicates that the robustness of the load forecasting model is poor, the load forecasting model can be optimized to improve the robustness of the load forecasting model, thereby improving the accuracy of load forecasting of the load forecasting model.

[0060] For example, the success rate of the adversarial attack can be calculated, and then the success rate of the adversarial attack can be used as a robustness indicator to evaluate the robustness of the load forecasting model based on the success rate of the adversarial attack. Alternatively, the interference sensitivity can be calculated, and then the interference sensitivity can be used as a robustness indicator to evaluate the robustness of the load forecasting model based on the interference sensitivity. Alternatively, the perturbation amplitude of the adversarial sample can be calculated, and then the perturbation amplitude of the adversarial sample can be used as a robustness indicator to evaluate the robustness of the load forecasting model based on the perturbation amplitude of the adversarial sample.

[0061] In addition, the success rate of adversarial attacks, interference sensitivity, and adversarial sample disturbance amplitude can also be calculated, and then the success rate of adversarial attacks, interference sensitivity, and adversarial sample disturbance amplitude can be used together as robustness indicators. The robustness of the load forecasting model can be jointly evaluated based on the success rate of adversarial attacks, interference sensitivity, and adversarial sample disturbance amplitude.

[0062] In an embodiment of the present application, an adversarial sample is generated based on sample generation constraints and historical load monitoring data of the power system, and the adversarial sample is input into the load forecasting model for an adversarial attack to obtain a load forecasting sample corresponding to the adversarial sample. According to the load forecasting sample, the robustness index of the load forecasting model is evaluated, thereby evaluating the robustness of the load forecasting model. When the robustness index shows that the load forecasting model has poor robustness, the load forecasting model can be optimized to improve the accuracy of the load forecasting model in load forecasting.

[0063] Based on the above embodiment, the above S203 may further include the following steps:

[0064] Interference sensitivity is determined based on a first number of adversarial samples, load forecast samples, and initial load forecast data corresponding to historical load monitoring data.

[0065] Among them, robustness indicators include interference sensitivity.

[0066] In the embodiment of the present application, interference sensitivity is used to characterize the sensitivity of the load forecasting model to input disturbances, that is, the change in the output of the load forecasting model for adversarial samples relative to the output of historical load monitoring data.

[0067] Exemplarily, the interference sensitivity PS may be determined by the following formula (2) based on the first number of adversarial samples, the load forecast samples, and the initial load forecast data corresponding to the historical load monitoring data:

[0068] (2)

[0069] In formula (2), N is the total number of adversarial samples, that is, the first number, For historical load monitoring data The corresponding initial load forecast data, For adversarial examples Corresponding load forecast samples.

[0070] In an embodiment of the present application, interference sensitivity is determined based on a first number of adversarial samples, load forecast samples, and initial load forecast data corresponding to historical load monitoring data, so that the robustness of the load forecast model can be evaluated based on the interference sensitivity, and then when the interference sensitivity shows that the robustness of the load forecast model is poor, the load forecast model can be optimized to improve the accuracy of the load forecast model in load forecasting.

[0071] Based on the above embodiment, the above S203 may further include the following steps:

[0072] Based on the first number of adversarial samples, the historical load monitoring data, and the adversarial samples, a disturbance amplitude of the adversarial samples is determined.

[0073] Among them, the robustness indicators include the perturbation amplitude of the adversarial samples.

[0074] In the embodiment of the present application, the perturbation amplitude of the adversarial sample is used to characterize the degree of change of the adversarial sample relative to the original sample.

[0075] For example, the adversarial sample perturbation amplitude can be calculated by the L2 norm, that is, the following formula (3): :

[0076] (3)

[0077] Alternatively, the adversarial sample perturbation amplitude can be calculated by the L∞ norm, that is, the following formula (4): :

[0078] (4)

[0079] In formula (3) and formula (4), N is the total number of adversarial samples, is the historical load monitoring data. For adversarial samples.

[0080] In an embodiment of the present application, the disturbance amplitude of the adversarial samples is determined based on the first number of adversarial samples, historical load monitoring data, and adversarial samples, so that the robustness of the load prediction model can be evaluated based on the disturbance amplitude of the adversarial samples. When the disturbance amplitude of the adversarial samples shows that the robustness of the load prediction model is poor, the load prediction model can be optimized to improve the accuracy of the load prediction model in load prediction.

[0081] Based on the above embodiment, the above S201 may further include the following steps:

[0082] At least one adversarial sample generation algorithm is used to generate adversarial samples based on historical load monitoring data under the condition of sample generation constraints.

[0083] Exemplarily, adversarial sample generation algorithms may include, for example, the fast gradient sign method (FGSM), the projected gradient descent method (PGD), the Carlini and Wagner attack (C&W), and the like.

[0084] Preferably, based on a variety of adversarial sample generation algorithms, under the condition of adversarial constraints, a small perturbation sample can be applied to the historical load monitoring data to obtain an intermediate adversarial sample, and then the intermediate adversarial sample that does not meet the target domain constraints is projected into the constraint space through the DPLL (Davis-Putnam-Logemann-Loveland) algorithm to obtain an adversarial sample.

[0085] In a possible implementation, if adversarial samples are generated based on multiple adversarial sample generation algorithms, each adversarial sample generation algorithm may be uniformly managed and standardized to improve the consistency of the adversarial samples.

[0086] In an embodiment of the present application, at least one adversarial sample generation algorithm is used to generate adversarial samples based on historical load monitoring data under the condition of sample generation constraints, thereby improving the diversity of adversarial samples and further improving the comprehensiveness and accuracy of robustness evaluation.

[0087] Reference Figure 3 , Figure 3 : is a flow chart of a method for determining a target domain constraint provided by an embodiment of the present application. Based on the above embodiment, the sample generation constraint includes a target domain constraint, and the method further includes the following steps:

[0088] S301, determining at least one initial domain constraint related to historical load monitoring data.

[0089] In one embodiment, initial domain constraints related to historical load monitoring data may be enumerated. Assuming that there are n features related to the historical load monitoring data, theoretically, the combination order of magnitude of the initial domain constraints related to the historical load monitoring data is O(2n).

[0090] For example, the features related to the historical load monitoring data are A and B, then the initial domain constraints related to the historical load monitoring data may include A and B, A or B, not A and B, not A or not B. These possible initial domain constraint combinations constitute the initial constraint space T, T = {A∨B), (¬A∨B), (A∨¬B), (¬A∨¬B)}.

[0091] S302: for each initial domain constraint, determine whether the historical load monitoring data satisfies the initial domain constraint.

[0092] Exemplarily, the values ​​of feature A and feature B corresponding to each historical load monitoring data can be substituted into the above initial domain constraints for checking. If the result is False, the historical load monitoring data does not meet the initial domain constraint; if the result is True, the historical load monitoring data meets the initial domain constraint.

[0093] S303: If the historical load monitoring data satisfies the initial domain constraint, the initial domain constraint is determined as the target domain constraint.

[0094] Exemplarily, the initial domain constraints that are not satisfied by the historical load monitoring data may be deleted, and then the initial domain constraints that are satisfied by the historical load monitoring data may be determined as the target domain constraints.

[0095] It should be noted that the target domain constraint should be the set of all initial domain constraints satisfied by the historical load monitoring data.

[0096] In an embodiment of the present application, at least one initial domain constraint related to the historical load monitoring data is determined, and for each initial domain constraint, it is judged whether the historical load monitoring data satisfies the initial domain constraint. If the historical load monitoring data satisfies the initial domain constraint, the initial domain constraint is determined as the target domain constraint, so that adversarial samples that meet the requirements of a specific domain can be generated based on the target domain constraint, thereby improving the targeted nature of adversarial attacks and further improving the accuracy of robustness evaluation.

[0097] Reference Figure 4 , Figure 4 1 is a flowchart of a robustness index evaluation method provided in an embodiment of the present application. This embodiment involves a possible implementation method of how to evaluate the robustness index of a load forecasting model based on a load forecasting sample. Based on the above embodiment, the above S203 includes the following steps:

[0098] S401, when a new adversarial sample generation algorithm is detected, a new adversarial sample is generated based on historical load monitoring data using the new adversarial sample generation algorithm under the condition of sample generation constraints.

[0099] S402: Input the new adversarial sample into the load forecasting model for adversarial attack to obtain a new load forecasting sample corresponding to the new adversarial sample.

[0100] In one embodiment, a monitoring module may be provided, through which the update of the adversarial sample generation algorithm is obtained at preset intervals. When a new adversarial sample generation algorithm is detected, a new adversarial sample is generated based on historical load monitoring data using the new adversarial sample generation algorithm under the condition of sample generation constraints. The new adversarial sample is then input into the load forecasting model for adversarial attack to obtain a new load forecasting sample corresponding to the new adversarial sample.

[0101] S403: Evaluate the robustness index of the load forecasting model according to the load forecasting samples and the new load forecasting samples.

[0102] In one embodiment, when the presence of a new adversarial sample generation algorithm is monitored, the robustness index of the load forecasting model can be calculated based on the load forecasting samples and the new load forecasting samples to evaluate the robustness of the load forecasting model through the robustness index.

[0103] In another embodiment, when a new adversarial sample generation algorithm is detected, the new adversarial sample generation algorithm can be used to generate new adversarial samples based on historical load monitoring data under the condition of sample generation constraints, and then the new adversarial samples and the original adversarial samples are input into the load forecasting model for adversarial attack to obtain new load forecasting samples. Then, the robustness index of the load forecasting model is evaluated based on the new load forecasting samples.

[0104] In an embodiment of the present application, when the existence of a new adversarial sample generation algorithm is monitored, a new adversarial sample is generated based on historical load monitoring data using the new adversarial sample generation algorithm under the condition of sample generation constraints, and the new adversarial sample is input into the load forecasting model for an adversarial attack to obtain a new load forecasting sample corresponding to the new adversarial sample. The robustness index of the load forecasting model is evaluated based on the load forecasting sample and the new load forecasting sample, thereby enabling dynamic updating of the adversarial sample generation algorithm and using the latest adversarial sample generation algorithm to generate new adversarial samples, so that the latest adversarial sample generation technology can always be integrated in the robustness evaluation process, so that the model can keep pace with the latest attack methods, thereby making up for the limitations of the prior art in the static evaluation process.

[0105] Reference Figure 5 , Figure 5 1 is a flow chart of a method for evaluating the robustness of a load forecasting model provided in an embodiment of the present application. The method comprises the following steps:

[0106] S501, determining at least one initial domain constraint related to historical load monitoring data.

[0107] S502: for each initial domain constraint, determine whether the historical load monitoring data satisfies the initial domain constraint.

[0108] S503: If the historical load monitoring data satisfies the initial domain constraint, the initial domain constraint is determined as the target domain constraint.

[0109] S504, using at least one adversarial sample generation algorithm, generating adversarial samples based on historical load monitoring data under the condition of sample generation constraints, wherein the sample generation constraints include target domain constraints.

[0110] S505: Input the adversarial sample into the load forecasting model to perform adversarial attack to obtain a load forecasting sample corresponding to the adversarial sample.

[0111] S506: Evaluate the robustness index of the load forecasting model based on the load forecasting samples.

[0112] S507, when a new adversarial sample generation algorithm is detected, a new adversarial sample is generated based on historical load monitoring data using the new adversarial sample generation algorithm under the condition of sample generation constraints.

[0113] S508: Input the new adversarial sample into the load forecasting model for adversarial attack to obtain a new load forecasting sample corresponding to the new adversarial sample.

[0114] S509: Evaluate the robustness index of the load forecasting model according to the load forecasting samples and the new load forecasting samples.

[0115] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0116] Based on the same inventive concept, the embodiment of the present application also provides a model evaluation device for implementing the model evaluation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more model evaluation device embodiments provided below can refer to the limitations of the model evaluation method above, and will not be repeated here.

[0117] In one embodiment, Figure 6 As shown, Figure 6 600 is a structural block diagram of a model evaluation device provided in an embodiment of the present application, and the device 600 includes:

[0118] The generation module 601 is used to generate adversarial samples based on sample generation constraints and historical load monitoring data of the power system.

[0119] The adversarial attack module 602 is used to input the adversarial sample into the load forecasting model for adversarial attack to obtain the load forecasting sample corresponding to the adversarial sample.

[0120] The evaluation module 603 is used to evaluate the robustness index of the load forecasting model according to the load forecasting samples; the robustness index is used to evaluate the robustness of the load forecasting model.

[0121] In one embodiment, the evaluation module 603 includes:

[0122] The first determination unit is used to determine interference sensitivity based on a first number of adversarial samples, load forecast samples, and initial load forecast data corresponding to historical load monitoring data; the robustness index includes interference sensitivity.

[0123] In one embodiment, the evaluation module 603 includes:

[0124] The second determination unit is used to determine the perturbation amplitude of the adversarial sample based on the first number of adversarial samples, historical load monitoring data, and the adversarial sample; the robustness index includes the perturbation amplitude of the adversarial sample.

[0125] In one embodiment, the generating module 601 includes:

[0126] The first generation unit is used to generate adversarial samples based on historical load monitoring data under the condition of sample generation constraints by using at least one adversarial sample generation algorithm.

[0127] In one embodiment, the sample generation constraint includes a target domain constraint, and the apparatus 600 further includes:

[0128] The first determination module is used to determine at least one initial domain constraint related to historical load monitoring data.

[0129] The judgment module is used to judge whether the historical load monitoring data meets the initial domain constraints for each initial domain constraint.

[0130] The second determination module is used to determine the initial domain constraint as the target domain constraint if the historical load monitoring data satisfies the initial domain constraint.

[0131] In one embodiment, the evaluation module 603 includes:

[0132] The second generation unit is used to generate new adversarial samples based on historical load monitoring data using the new adversarial sample generation algorithm under the condition of sample generation constraints when the existence of a new adversarial sample generation algorithm is monitored.

[0133] The adversarial attack unit is used to input the new adversarial sample into the load forecasting model for adversarial attack to obtain a new load forecasting sample corresponding to the new adversarial sample.

[0134] The evaluation unit is used to evaluate the robustness index of the load forecasting model according to the load forecasting samples and the new load forecasting samples.

[0135] Each module in the above-mentioned model evaluation device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0136] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0137] Generate adversarial samples based on sample generation constraints and historical load monitoring data of the power system;

[0138] Input the adversarial sample into the load forecasting model to conduct adversarial attack, so as to obtain the load forecasting sample corresponding to the adversarial sample;

[0139] According to the load forecasting samples, the robustness index of the load forecasting model is evaluated; the robustness index is used to evaluate the robustness of the load forecasting model.

[0140] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0141] Based on the first number of adversarial samples, the load forecast samples, and the initial load forecast data corresponding to the historical load monitoring data, interference sensitivity is determined; the robustness indicator includes interference sensitivity.

[0142] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0143] Based on the first number of adversarial samples, historical load monitoring data, and adversarial samples, a perturbation amplitude of the adversarial samples is determined; and the robustness indicator includes the perturbation amplitude of the adversarial samples.

[0144] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0145] At least one adversarial sample generation algorithm is used to generate adversarial samples based on historical load monitoring data under the condition of sample generation constraints.

[0146] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0147] determining at least one initial domain constraint associated with historical load monitoring data;

[0148] For each initial domain constraint, determine whether the historical load monitoring data meets the initial domain constraint;

[0149] If the historical load monitoring data satisfies the initial domain constraint, the initial domain constraint is determined as the target domain constraint.

[0150] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0151] When a new adversarial sample generation algorithm is detected, a new adversarial sample is generated based on historical load monitoring data using the new adversarial sample generation algorithm under the condition of sample generation constraints;

[0152] Input the new adversarial sample into the load forecasting model to conduct adversarial attack, so as to obtain a new load forecasting sample corresponding to the new adversarial sample;

[0153] Based on the load forecast samples and the new load forecast samples, the robustness index of the load forecast model is evaluated.

[0154] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0155] Generate adversarial samples based on sample generation constraints and historical load monitoring data of the power system;

[0156] Input the adversarial sample into the load forecasting model to conduct adversarial attack, so as to obtain the load forecasting sample corresponding to the adversarial sample;

[0157] According to the load forecasting samples, the robustness index of the load forecasting model is evaluated; the robustness index is used to evaluate the robustness of the load forecasting model.

[0158] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0159] Based on the first number of adversarial samples, the load forecast samples, and the initial load forecast data corresponding to the historical load monitoring data, interference sensitivity is determined; the robustness indicator includes interference sensitivity.

[0160] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0161] Based on the first number of adversarial samples, historical load monitoring data, and adversarial samples, a perturbation amplitude of the adversarial samples is determined; and the robustness indicator includes the perturbation amplitude of the adversarial samples.

[0162] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0163] At least one adversarial sample generation algorithm is used to generate adversarial samples based on historical load monitoring data under the condition of sample generation constraints.

[0164] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0165] determining at least one initial domain constraint associated with historical load monitoring data;

[0166] For each initial domain constraint, determine whether the historical load monitoring data meets the initial domain constraint;

[0167] If the historical load monitoring data satisfies the initial domain constraint, the initial domain constraint is determined as the target domain constraint.

[0168] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0169] When a new adversarial sample generation algorithm is detected, a new adversarial sample is generated based on historical load monitoring data using the new adversarial sample generation algorithm under the condition of sample generation constraints;

[0170] Input the new adversarial sample into the load forecasting model to conduct adversarial attack, so as to obtain a new load forecasting sample corresponding to the new adversarial sample;

[0171] Based on the load forecast samples and the new load forecast samples, the robustness index of the load forecast model is evaluated.

[0172] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0173] Generate adversarial samples based on sample generation constraints and historical load monitoring data of the power system;

[0174] Input the adversarial sample into the load forecasting model to conduct adversarial attack, so as to obtain the load forecasting sample corresponding to the adversarial sample;

[0175] According to the load forecasting samples, the robustness index of the load forecasting model is evaluated; the robustness index is used to evaluate the robustness of the load forecasting model.

[0176] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0177] Based on the first number of adversarial samples, the load forecast samples, and the initial load forecast data corresponding to the historical load monitoring data, interference sensitivity is determined; the robustness indicator includes interference sensitivity.

[0178] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0179] Based on the first number of adversarial samples, historical load monitoring data, and adversarial samples, a perturbation amplitude of the adversarial samples is determined; and the robustness indicator includes the perturbation amplitude of the adversarial samples.

[0180] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0181] At least one adversarial sample generation algorithm is used to generate adversarial samples based on historical load monitoring data under the condition of sample generation constraints.

[0182] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0183] determining at least one initial domain constraint associated with historical load monitoring data;

[0184] For each initial domain constraint, determine whether the historical load monitoring data meets the initial domain constraint;

[0185] If the historical load monitoring data satisfies the initial domain constraint, the initial domain constraint is determined as the target domain constraint.

[0186] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0187] When a new adversarial sample generation algorithm is detected, a new adversarial sample is generated based on historical load monitoring data using the new adversarial sample generation algorithm under the condition of sample generation constraints;

[0188] Input the new adversarial sample into the load forecasting model to conduct adversarial attack, so as to obtain a new load forecasting sample corresponding to the new adversarial sample;

[0189] Based on the load forecast samples and the new load forecast samples, the robustness index of the load forecast model is evaluated.

[0190] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0191] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0192] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A model evaluation method, characterized in that: The method comprises: Generate adversarial samples based on sample generation constraints and historical load monitoring data of the power system; Inputting the adversarial sample into the load forecasting model for adversarial attack to obtain a load forecasting sample corresponding to the adversarial sample; According to the load forecasting samples, a robustness index of the load forecasting model is evaluated; the robustness index is used to evaluate the robustness of the load forecasting model.

2. The method according to claim 1, characterized in that The step of evaluating the robustness index of the load forecasting model according to the load forecasting sample includes: Based on the first number of the adversarial samples, the load forecast samples, and the initial load forecast data corresponding to the historical load monitoring data, interference sensitivity is determined; the robustness indicator includes the interference sensitivity.

3. The method according to claim 1 or 2, characterized in that: The step of evaluating the robustness index of the load forecasting model according to the load forecasting sample includes: Based on the first number of adversarial samples, the historical load monitoring data, and the adversarial samples, the adversarial sample perturbation amplitude is determined; and the robustness indicator includes the adversarial sample perturbation amplitude.

4. The method according to claim 1 or 2, characterized in that: The method of generating adversarial samples based on sample generation constraints and historical load monitoring data of the power system includes: At least one adversarial sample generation algorithm is used to generate the adversarial sample based on the historical load monitoring data under the condition of the sample generation constraint.

5. The method according to claim 1 or 2, characterized in that: The sample generation constraint includes a target domain constraint, and the method further includes: determining at least one initial domain constraint associated with the historical load monitoring data; For each of the initial domain constraints, determining whether the historical load monitoring data satisfies the initial domain constraint; If the historical load monitoring data satisfies the initial domain constraint, the initial domain constraint is determined as the target domain constraint.

6. The method according to claim 4, characterized in that The step of evaluating the robustness index of the load forecasting model according to the load forecasting sample includes: When a new adversarial sample generation algorithm is detected, generating a new adversarial sample based on the historical load monitoring data using the new adversarial sample generation algorithm and under the condition of the sample generation constraint; Inputting the new adversarial sample into the load forecasting model for adversarial attack to obtain a new load forecasting sample corresponding to the new adversarial sample; The robustness index of the load forecasting model is evaluated according to the load forecasting sample and the new load forecasting sample.

7. A model evaluation device, characterized in that: The device comprises: A generation module, used to generate adversarial samples based on sample generation constraints and historical load monitoring data of the power system; An adversarial attack module, used for inputting the adversarial sample into the load forecasting model for adversarial attack, so as to obtain a load forecasting sample corresponding to the adversarial sample; An evaluation module is used to evaluate the robustness index of the load forecasting model according to the load forecasting sample; the robustness index is used to evaluate the robustness of the load forecasting model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images