A method, device, computer equipment and storage medium for generating adversarial samples

Abstract

The present application belongs to the field of computer technology, and discloses a method, device, computer equipment and storage medium for generating adversarial samples, which includes: collecting an initial sample set of a target battery to train a neural network and obtain an initial neural network; inputting each initial sample in the initial sample set into the initial neural network and obtaining an original output value; inputting each initial sample into a trained strategy network, and adding a perturbation amount to each dimension of the initial sample through the strategy network to obtain a corresponding perturbation sample; inputting each perturbation sample into a trained value network and obtaining a corresponding perturbation score; inputting the perturbation sample with the highest perturbation score into the initial neural network and obtaining a perturbation output value; judging whether the difference between the perturbation output value and the original output value corresponding to the perturbation sample is greater than a preset threshold; if so, putting the perturbation sample into the initial sample set as an adversarial sample. The present application can shorten the generation time of adversarial samples and improve the robustness of the initial neural network.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more particularly to a method, apparatus, computer device, and storage medium for generating adversarial samples. Background Art

[0002] In the calculation of battery parameters, the application of neural networks is becoming more and more popular. Since neural networks usually need to collect samples for training, and these samples cannot ensure that every possible situation of the battery can be collected when they are collected, the neural network's fitting accuracy for situations outside the samples is not high at the end of training, and there is even the possibility of large errors.

[0003] In the prior art, interference is usually applied to all collected samples to find adversarial samples. However, when the number of samples is large, the operation of interfering with all samples will take a long time, and the generation efficiency of adversarial samples is low. Summary of the invention

[0004] The present application provides an adversarial sample generation method, apparatus, computer device, and storage medium, which can shorten the generation time of adversarial samples and improve the robustness of initial neural network training.

[0005] In a first aspect, an embodiment of the present application provides a method for generating an adversarial sample, including:

[0006] Collecting an initial sample set of the target battery to train the neural network and obtain an initial neural network;

[0007] Input each initial sample in the initial sample set into the initial neural network to obtain the original output value;

[0008] Input each initial sample into the trained policy network, add the perturbation amount to each dimension of the initial sample through the policy network, and obtain the corresponding perturbation sample;

[0009] Input each perturbation sample into the trained value network to obtain the corresponding perturbation score;

[0010] Input the perturbation sample with the highest perturbation score into the initial neural network to obtain the perturbation output value;

[0011] Determine whether the difference between the perturbed output value and the original output value corresponding to the perturbed sample is greater than a preset threshold; if so, put the perturbed sample into the initial sample set as an adversarial sample.

[0012] Furthermore, the initial sample is the current power of the target battery, as well as current data and voltage data during charging.

[0013] Furthermore, the original output value or the perturbation output value output by the initial neural network is the remaining charging time.

[0014] Furthermore, the method also includes: after putting the perturbed samples as adversarial samples into the initial sample set, using the initial sample set to train the initial neural network to obtain a trained target neural network.

[0015] Furthermore, the method further comprises:

[0016] Input the initial sample into the initial strategy network, add perturbation to any dimension, and obtain the training sample;

[0017] Input the training samples into the initial neural network to obtain the training output value;

[0018] Calculate the change between the training output value and the original output value of the initial sample;

[0019] Input the training samples into the initial value network to obtain the corresponding training scores;

[0020] Train the initial policy network using the training scores as the supervision values ​​of the training samples;

[0021] The change amount is used as the supervision value of the training sample to train the initial value network;

[0022] Repeat all the above steps to obtain the trained policy network and the trained value network.

[0023] Furthermore, the above calculation of the change between the training output value and the original output value of the initial sample includes:

[0024] Let the disturbance amount add 1, and then multiply it by the training output value to get the first value;

[0025] Subtract the original output value from the first value to get the change.

[0026] Furthermore, the perturbation value ranges from -5% to 5% with a step of 1%.

[0027] In a second aspect, an embodiment of the present application provides an adversarial sample generation device, comprising:

[0028] An initial training module is used to collect an initial sample set of a target battery to train a neural network and obtain an initial neural network;

[0029] The original input module is used to input each initial sample in the initial sample set into the initial neural network to obtain the original output value;

[0030] The perturbation adding module is used to input each initial sample into the trained policy network, and add the perturbation amount to each dimension of the initial sample through the policy network to obtain the corresponding perturbation sample;

[0031] The perturbation scoring module is used to input each perturbation sample into the trained value network to obtain the corresponding perturbation score;

[0032] The disturbance input module is used to input the disturbance sample with the highest disturbance score into the initial neural network to obtain the disturbance output value;

[0033] The adversarial judgment module is used to judge whether the difference between the perturbed output value and the original output value corresponding to the perturbed sample is greater than a preset threshold; if so, the perturbed sample is put into the initial sample set as an adversarial sample.

[0034] In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of an adversarial sample generation method as in any of the above embodiments are performed.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for generating adversarial samples as in any of the above embodiments are implemented.

[0036] In summary, compared with the prior art, the technical solution provided in the embodiment of the present application has at least the following beneficial effects:

[0037] The embodiment of the present application provides a method for generating adversarial samples, which adds a perturbation to the initial sample based on a trained policy network, and uses a trained value network to score the perturbed sample after adding the perturbation, selects the perturbed sample with the highest perturbation score and judges the difference between it and the initial sample. If it is greater than a preset threshold, it means that the perturbed sample can be used as an adversarial sample and trains the neural network together with the initial sample. The above method realizes the perturbation of samples and the determination of adversarial samples through the trained policy network and value network, shortens the generation time of adversarial samples, and improves the robustness of the initial neural network training. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flowchart of a method for generating adversarial samples is provided as an exemplary embodiment of the present application.

[0039] Figure 2 A flowchart of the policy network and value network training steps provided for an exemplary embodiment of the present application.

[0040] Figure 3 A structural diagram of an adversarial sample generation device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0042] Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of this application.

[0043] See also Figure 1 , the embodiment of the present application provides a method for generating adversarial samples, including:

[0044] Step S11, collecting an initial sample set of the target battery to train a neural network and obtain an initial neural network.

[0045] Step S12, inputting each initial sample in the initial sample set into the initial neural network to obtain an original output value.

[0046] The initial sample is the current power of the target battery, as well as the current data and voltage data during charging.

[0047] Step S13: input each initial sample into the trained policy network, and add a perturbation amount to each dimension of the initial sample through the policy network to obtain a corresponding perturbation sample.

[0048] The perturbation value ranges from -5% to 5% with a step of 1%. Each dimension refers to each value in the vector of the initial sample; for example, if the input initial sample is [10, 20, 30], then there are 3 dimensions, and after applying +5% perturbation, it becomes [10.5, 21, 31.5]. Specifically, different perturbations can be applied to different dimensions of the same initial sample.

[0049] Step S14: input each disturbance sample into the trained value network to obtain a corresponding disturbance score.

[0050] Step S15, inputting the disturbance sample with the highest disturbance score into the initial neural network to obtain the disturbance output value.

[0051] Among them, the original output value or the perturbation output value of the initial neural network output can be the remaining charging time.

[0052] Specifically, the target battery is completely discharged, and then charged to full capacity. The current capacity, current data, and voltage data at different times during charging are collected as initial samples, and the charging time at the corresponding time is calculated. After charging is completed, the total charging time - the charging time of each current capacity = the remaining charging time corresponding to each current capacity is calculated, and this is used as the supervision value of each initial sample to train the neural network to obtain the initial neural network.

[0053] In the specific implementation process, in addition to the remaining charging time, other neural network adversarial sample generation processes related to battery parameter calculations (such as discharge time, remaining power, etc.) are all within the scope of disclosure of this application.

[0054] Step S16, determining whether the difference between the perturbed output value and the original output value corresponding to the perturbed sample is greater than a preset threshold; if so, placing the perturbed sample as an adversarial sample into the initial sample set.

[0055] Furthermore, the present application can also train the initial neural network using the initial sample set after putting the perturbation sample as an adversarial sample into the initial sample set to obtain a trained target neural network.

[0056] Furthermore, the adversarial sample can be matched with a corresponding learning rate according to the difference between the perturbed output value and the original output value. Using this learning rate during the initial neural network training can make the adversarial sample play a role to varying degrees during training.

[0057] The above embodiment provides a method for generating adversarial samples, which adds a perturbation to the initial sample based on a trained policy network, and uses a trained value network to score the perturbed sample after adding the perturbation, selects the perturbed sample with the highest perturbation score and determines the difference between it and the initial sample. If it is greater than a preset threshold, it means that the perturbed sample can be used as an adversarial sample and trains the neural network together with the initial sample. The above method realizes the perturbation of samples and the determination of adversarial samples through the trained policy network and value network, shortens the generation time of adversarial samples, and improves the robustness of the initial neural network training.

[0058] See also Figure 2 In some embodiments, the method further comprises:

[0059] Step S21, input the initial sample into the initial strategy network, add a disturbance to any dimension thereof, and obtain a training sample.

[0060] The initial policy network can adopt a feedforward neural network or a fully connected neural network. Specifically, when training the initial policy network, it is not necessary to add a perturbation to each dimension of the initial sample, which will increase the training time. Therefore, it is sufficient to randomly select one dimension of the initial sample for perturbation.

[0061] Step S22, input the training sample into the initial neural network to obtain the training output value.

[0062] Step S23, calculating the change between the training output value and the original output value of the initial sample.

[0063] Specifically, add 1 to the disturbance amount and multiply it by the training output value to obtain a first value; subtract the original output value from the first value to obtain a change; that is, change = training output value*(1+disturbance amount)-original output value.

[0064] Here, the disturbance amount itself has positive and negative signs. For example, when the disturbance amount is +2%, the first value is +2%+100%=102%; when the disturbance amount is -2%, the first value is -2%+100%=98%.

[0065] Step S24, input the training samples into the initial value network to obtain corresponding training scores.

[0066] Among them, the initial value network, also known as the Q-value neural network, can adopt a feedforward neural network or a fully connected neural network.

[0067] Specifically, the scoring formula of the Q-value neural network for the disturbance amount is:

[0068] Q(s_t,a_t)←Q(s_t,a_t)+α(r_t+γ*max(Q(s_t+1,a_t))-Q(s_t,a_t)).

[0069] Among them, Q(s_t,a_t) is the training score of the initial sample, Q(s_t+1,a_t) is the training score of the training sample, r_t is the change, α is the learning rate, γ is the discount factor, α can be 0.2, and γ can be 0.8.

[0070] Step S25, using the training scores as the supervision values ​​of the training samples to train the initial strategy network.

[0071] Step S26, using the variation as the supervision value of the training sample to train the initial value network.

[0072] Step S27, repeat all the above steps (S21-S26) to obtain a trained policy network and a trained value network.

[0073] The above steps S21-S27 may be performed after the above step S12.

[0074] The policy network and value network in this application are trained alternately. When training the policy network, one dimension of the initial sample is selected for perturbation. After the perturbation, the value network calculates the score, and the policy network is trained based on this score as the supervision value. After the training, the policy network is a complete network (that is, it knows how to add perturbation so that the difference in the perturbation output value exceeds the preset threshold). Subsequently, the initial sample is perturbed in each dimension in the formal application.

[0075] See also Figure 3Another embodiment of the present application provides an adversarial sample generation device, including:

[0076] The initial training module 101 is used to collect an initial sample set of a target battery to train a neural network and obtain an initial neural network.

[0077] The original input module 102 is used to input each initial sample in the initial sample set into the initial neural network to obtain an original output value.

[0078] The disturbance adding module 103 is used to input each initial sample into the trained policy network, and add a disturbance amount to each dimension of the initial sample through the policy network to obtain a corresponding disturbance sample.

[0079] The disturbance scoring module 104 is used to input each disturbance sample into the trained value network to obtain a corresponding disturbance score.

[0080] The disturbance input module 105 is used to input the disturbance sample with the highest disturbance score into the initial neural network to obtain a disturbance output value.

[0081] The adversarial judgment module 106 is used to judge whether the difference between the perturbed output value and the original output value corresponding to the perturbed sample is greater than a preset threshold; if so, the perturbed sample is put into the initial sample set as an adversarial sample.

[0082] In some embodiments, the device further includes: a target training module, which is used to train the initial neural network using the initial sample set after putting the perturbed sample as an adversarial sample into the initial sample set to obtain a trained target neural network.

[0083] Furthermore, the device also includes: a training adding module, which is used to input the initial sample into the initial strategy network, add a disturbance amount to any dimension thereof, and obtain a training sample.

[0084] The training output module is used to input the training samples into the initial neural network to obtain the training output values.

[0085] The change calculation module is used to calculate the change between the training output value and the original output value of the initial sample.

[0086] The training scoring module is used to input the training samples into the initial value network to obtain the corresponding training scores.

[0087] The strategy training module is used to train the initial strategy network using the training scores as the supervision values ​​of the training samples.

[0088] The value training module is used to train the initial value network by taking the variation as the supervision value of the training sample.

[0089] The repeated training module is used to repeatedly execute all the above modules to obtain the trained policy network and the trained value network.

[0090] For the specific limitations on an adversarial sample generating device provided in this embodiment, please refer to the above embodiment on an adversarial sample generating method, which will not be repeated here.

[0091] Each module in the above-mentioned adversarial sample generation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0092] An embodiment of the present application provides a computer device, which may include a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the processor executes the steps of a method for generating an adversarial sample as in any of the above embodiments.

[0093] The working process, working details and technical effects of the computer device provided in this embodiment can be found in the above embodiment of an adversarial sample generation method, which will not be described in detail here.

[0094] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for generating an adversarial sample as in any of the above embodiments are implemented. Wherein, the computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash drive and / or a memory stick (Memory Stick), etc., and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The working process, working details and technical effects of the computer-readable storage medium provided in this embodiment can be found in the above embodiment of a method for generating an adversarial sample, which will not be repeated here.

[0095] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0096] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above-mentioned embodiments only express several implementation methods of the present application, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several deformations and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the patent of this application shall be based on the attached claims.

Claims

1. A method for generating adversarial samples, characterized in that: include: Collecting an initial sample set of the target battery to train the neural network and obtain an initial neural network; Inputting each initial sample in the initial sample set into the initial neural network to obtain an original output value; Input the initial sample into the initial strategy network, add a disturbance to any dimension thereof, and obtain a training sample; Inputting the training sample into the initial neural network to obtain a training output value; Calculating a change between the training output value and the original output value of the initial sample; Inputting the training samples into the initial value network to obtain corresponding training scores; Using the training score as a supervision value of the training sample to train the initial strategy network; Using the variation as the supervision value of the training sample to train the initial value network; Repeat all the above steps to obtain the trained strategy network and the trained value network; Inputting each of the initial samples into a trained policy network, adding a perturbation amount to each dimension of the initial samples through the policy network, and obtaining a corresponding perturbation sample; Input each of the disturbance samples into the trained value network to obtain a corresponding disturbance score; Inputting the disturbance sample with the highest disturbance score into the initial neural network to obtain a disturbance output value; Determine whether a difference between the disturbance output value and the original output value corresponding to the disturbance sample is greater than a preset threshold; If yes, the perturbed sample is put into the initial sample set as an adversarial sample; Among them, the initial sample is the current power of the target battery, as well as the current data and voltage data during charging; the original output value or the disturbance output value output by the initial neural network is the remaining charging time.

2. The adversarial sample generation method according to claim 1, characterized in that: Also includes: After the perturbation samples are put into the initial sample set as adversarial samples, the initial neural network is trained using the initial sample set to obtain a trained target neural network.

3. The adversarial sample generation method according to claim 1, characterized in that: The calculating the change between the training output value and the original output value of the initial sample comprises: Add 1 to the disturbance amount, and then multiply it by the training output value to obtain a first value; The change is obtained by subtracting the original output value from the first value.

4. The adversarial sample generation method according to claim 3, characterized in that: The disturbance value ranges from -5% to 5% with a step of 1%.

5. An adversarial sample generation device, characterized in that: include: An initial training module is used to collect an initial sample set of a target battery to train a neural network and obtain an initial neural network; An original input module, used for inputting each initial sample in the initial sample set into the initial neural network to obtain an original output value; A training adding module, used for inputting the initial sample into the initial strategy network, adding a disturbance amount to any dimension thereof, and obtaining a training sample; A training output module, used for inputting the training samples into the initial neural network to obtain training output values; A change calculation module, used for calculating the change between the training output value and the original output value of the initial sample; A training scoring module, used for inputting the training samples into the initial value network to obtain corresponding training scores; A strategy training module, used for training an initial strategy network using the training score as a supervision value of a training sample; A value training module, used for training an initial value network by taking the variation as a supervision value of a training sample; Repeated training module, used to repeatedly execute all the above modules to obtain the trained policy network and the trained value network; A disturbance adding module, used for inputting each of the initial samples into a trained policy network, adding a disturbance amount to each dimension of the initial samples through the policy network, and obtaining a corresponding disturbance sample; A disturbance scoring module, used for inputting each disturbance sample into the trained value network to obtain a corresponding disturbance score; A disturbance input module, used for inputting the disturbance sample with the highest disturbance score into the initial neural network to obtain a disturbance output value; A confrontation judgment module, used to judge whether the difference between the disturbance output value and the original output value corresponding to the disturbance sample is greater than a preset threshold; If yes, the perturbed sample is put into the initial sample set as an adversarial sample; Among them, the initial sample is the current power of the target battery, as well as the current data and voltage data during charging; the original output value or the disturbance output value output by the initial neural network is the remaining charging time.

6. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the adversarial sample generation method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the adversarial sample generation method according to any one of claims 1 to 4 are implemented.

Images