Information processing device

Abstract

The information processing apparatus of the present invention is capable of generating a deep neural network that can predict specific robustness when using adversarial training. It includes one or more processors and a memory storing one or more programs. When one or more programs are executed by one or more processors, the information processing apparatus performs the following steps: acquiring user settings regarding the characteristics of adversarial training; performing adversarial training; using learning data including adversarial examples and correct answer data representing the original classification category, along with the user settings, to enable the neural network to learn; the adversarial training enables the neural network that outputs a misclassified category when given adversarial examples to learn to output the original classification category when given adversarial examples; and executing a formal verification algorithm to determine that no adversarial examples exist within a specified noise range of specific data in the neural network using weight coefficients obtained through adversarial training.

Description

Technical Field

[0001] This invention relates to an information processing apparatus that uses adversarial training and formal methods to improve the robustness of deep neural networks. Background Technology

[0002] In recent years, deep learning techniques have been used to identify subjects in images with a high probability. It is known that deep neural networks performing classification problems such as subject recognition can be fed images with noise (perturbations) of an imperceptible degree, known as adversarial examples, to intentionally induce misclassification.

[0003] On the other hand, a technique called adversarial training was developed to improve the robustness of deep neural networks by teaching them to learn against attacks that use adversarial examples to cause misclassification (Aleksander Madry and 4 others, “Towards Deep Learning Models Resistant to Adversarial Attacks” ICLR 2018, June 19, 2017, URL: https: / / arxiv.org / abs / 1706.06083).

[0004] In adversarial training, adversarial examples are used as training data to enable deep neural networks to learn without misclassifying data even when inputting these adversarial examples. Therefore, deep neural networks generated through adversarial training, like those of regular deep neural networks, do not guarantee correct classification results even if they achieve high accuracy on the data used in training and testing, just like regular deep neural networks.

[0005] On the other hand, since there are cases where misclassification of adversarial examples can lead to serious consequences, it is preferable to be able to predict the degree of robustness of the learned neural network during adversarial training. Summary of the Invention

[0006] The present invention was made in view of the above-mentioned problems, and its object is to realize a technique for generating deep neural networks that can predict and meet specific robustness when using adversarial training.

[0007] To address this issue, for example, the information processing apparatus of the present invention has the following structure:

[0008] An information processing apparatus comprising:

[0009] More than one processor; and

[0010] A memory that stores more than one program.

[0011] When one or more programs are executed by one or more processors, the information processing device performs the following steps:

[0012] Obtain user settings regarding the characteristics of adversarial training.

[0013] The adversarial training is performed using learning data including adversarial examples and correct answer data representing the original classification categories, along with the user settings, to enable the neural network to learn. The adversarial training causes the neural network to learn to output the misclassified category when given the adversarial examples, so that it can output the original classification category when given the adversarial examples.

[0014] A formal verification algorithm is executed to determine whether there are adversarial examples in the neural network using weight coefficients learned through the adversarial training within a specified noise range of specific data.

[0015] Invention Effects

[0016] According to the present invention, it is possible to generate deep neural networks that can predict specific robustness when using adversarial training. Attached Figure Description

[0017] Figure 1 This is a diagram illustrating the outline of an information providing system according to embodiments of the present invention.

[0018] Figure 2 This is a block diagram illustrating an example of the hardware structure of the information processing apparatus in this embodiment.

[0019] Figure 3 This is a block diagram illustrating an example of the functional structure of an information processing device.

[0020] Figure 4 This is one of the diagrams used to illustrate the difference between sample classification achieved through conventional learning and sample classification achieved through adversarial training.

[0021] Figure 5 This is Figure 2, used to illustrate the difference between sample classification achieved through conventional learning and sample classification achieved through adversarial training.

[0022] Figure 6 This is one example used to illustrate the application of formal verification to a regression model.

[0023] Figure 7 This is the second example used to illustrate the application of formal verification to a regression model.

[0024] Figure 8 This is one example used to illustrate the application of formal verification to a classification model.

[0025] Figure 9 This is the second example used to illustrate the application of formal verification to a classification model.

[0026] Figure 10 This is a flowchart representing the actions of robustness improvement processing in an information processing device.

[0027] Figure 11 This is a diagram representing an example of the data structure for learning and testing data.

[0028] Figure 12 This is a diagram representing an example of the data structure of adversarial sample data.

[0029] Figure 13 This is a diagram representing an example of the data structure of model data 334. Detailed Implementation

[0030] The embodiments are described in detail below with reference to the accompanying drawings. Furthermore, the following embodiments do not limit the invention as claimed, and not all combinations of the features described in the embodiments are necessarily necessary for the invention. Two or more features from the plurality of features described in the embodiments may be arbitrarily combined. Additionally, the same reference numerals are used for the same or identical structures, and repeated descriptions are omitted.

[0031] <Overview of Information Processing Systems>

[0032] refer to Figure 1 An example of an information processing system according to this embodiment will be described. The information processing system 10 comprises, for example, an information processing device 100, an electronic device 101, and a communication terminal 102, which can communicate with each other via a network. The information processing device 100 generates a learned model (also called a validated deep neural network) that has undergone adversarial training and formal verification by performing robustness improvement processing. Robustness improvement processing, adversarial training, and formal verification will be described later. The information processing device 100 is, for example, a server device, but it can also be an edge node configured in a network, or a node constituting a P2P network. In this embodiment, for simplicity, the example is a server device consisting of a single computer, but it can also consist of multiple computers, or it can be implemented using a virtual machine running on a host machine.

[0033] Communication terminal 102 is a communication terminal that allows user 103 to specify the data and settings required for robustness improvement processing, or to upload necessary data to information processing device 100. Communication terminal 102 may be, for example, a personal computer, but may also be a tablet device or a smartphone.

[0034] Electronic device 101 is an electronic device capable of configuring a validated neural network generated through robustness improvement processing. Electronic device 101 can, for example, download the validated neural network from information processing device 100 in response to user instructions or specific triggers such as system updates. Figure 1 The example shown illustrates an electronic device 101, such as a mobile device, but it could also be any other electronic device performing image recognition processing. The electronic device could also be, for example, a vehicle. Here, a structure downloading to electronic device 101 is shown, but instead of electronic device 101, a network application server with a neural network installed could be used. Alternatively, the information processing device 100 could be built on a cloud service platform, or the neural network could be installed in the information processing device 100 itself or in other devices on the cloud service platform.

[0035] Adversarial training and formal verification of the neural network are performed through the robustness improvement processing described in detail later in this embodiment. Therefore, the robustness of the neural network improved by this robustness improvement processing is enhanced against adversarial examples. Adversarial examples are samples (input data) generated in a manner that causes the neural network to misclassify by adding noise (also called perturbation) at a level imperceptible to the human eye. For neural networks with improved robustness against adversarial examples, in one example of the use case described below, the security risks caused by misclassification can be mitigated.

[0036] For example, let's take a neural network that classifies the animal in an image as an input. This neural network, for instance, when given an image of a panda, outputs a classification result stating that the subject is a panda. For example, an adversarial example is generated by adding mathematically calculated noise (perturbation) to the panda image. This adversarial example (the noisy panda image) appears to the human eye as a panda image. However, when this adversarial example is input into a neural network that has not undergone adversarial training, the neural network outputs an incorrect classification result, such as a gibbon. In contrast, in an electronic device using a validated neural network that has undergone the robustness improvement processing of this embodiment, a correct classification result (i.e., panda) can be obtained for adversarial examples with noise within a range that meets the robustness requirements.

[0037] As another example, let's consider a neural network that classifies the individuals in an image containing a captured face. Such face recognition technology is useful, for example, in entering and exiting rooms and analyzing surveillance images. When a face recognition process is performed using a neural network that has not undergone adversarial training, the person wearing sunglasses with a pattern of added noise (i.e., an adversarial example) will be identified as someone else. In contrast, in an electronic device using a validated neural network that has undergone the robustness improvement processing of this embodiment, the correct person can be identified as the classification result even for adversarial examples with noise within a range that meets the robustness requirements. That is, it is possible to prevent malfunctions of the electronic device (disguise using adversarial examples).

[0038] As another example, let's consider a neural network that classifies whether an image (or motion picture) of a face (or person's face) generated using deepfake technology that does not exist in reality is a deepfake. Such a classifier is useful for identifying deepfake-generated ID photos and account images, or for identifying motion pictures generated for political manipulation. However, there are security risks if adversarial examples are used to circumvent the judgment of such a neural network. In contrast, in an electronic device using a verified neural network that has undergone the robustness improvement processing of this embodiment, a correct judgment result on whether a deepfake has been obtained can be obtained for adversarial examples with noise within a range that meets the robustness requirements.

[0039] As another example, we will illustrate this with a neural network that identifies legal speeds based on speed signs within an image. In this example, there is a case where this neural network is installed in a vehicle. Such a neural network identifies legal speeds based on speed signs within an image, enabling vehicles to travel at the legal speed. When adversarially sampled by adding patterns to the signs, misclassifying them as signs of different speeds or signs that have stopped occurs when the recognition process is performed using a neural network that has not undergone adversarial training. In contrast, in an electronic device using a validated neural network that has undergone robustness improvement processing according to this embodiment, correct sign classification results can be obtained for adversarial examples with noise within a range that meets the robustness requirements.

[0040] There are many scenarios in which electronic devices using the verified neural network of this embodiment can be applied to practical applications. However, the scenarios and effects of applying the verified neural network of this embodiment to practical applications are not limited to the examples described above.

[0041] <Example of Hardware Structure of Information Processing Device>

[0042] refer to Figure 2The hardware structure of the information processing device 100 will be described as an example. The information processing device 100 includes a memory 202, a processor 204, a communication interface 206, a storage 208, and an input interface 210. These elements are connected to a bus 214 and communicate with each other via a bus 370.

[0043] Memory 202 is a volatile storage medium such as DRAM, which temporarily stores data and programs. Memory 208 is a non-volatile storage medium that persistently stores data and programs. Memory 208 can be, for example, a semiconductor memory or a hard disk. Memory 208 can store various data required for performing adversarial training and formal verification, such as learning data for enabling the neural network to learn, test data for testing the learned neural network, and adversarial example data, as described later.

[0044] Processor 204 may include, for example, a central processing unit (CPU) or other computing circuitry. Processor 204 may also consist of more than one processor. Processor 204 may further include computing circuitry (e.g., GPU) and dedicated hardware for performing statistical processing such as machine learning at higher speeds, and may also include internal memory. Processor 204 implements various functions of information processing device 100 by deploying and executing programs stored in memory 208 to memory 202.

[0045] Communication interface 206 is an interface for sending and receiving data with external devices of information processing device 100. Communication interface 206 may include communication circuitry capable of communicating in various standard-compliant communication methods. Communication interface 206 is connected to a network and exchanges data with communication terminal 102 and electronic device 101 via the network. Input interface 210 is, for example, a device for receiving input from the administrator of information processing device 100, but may be omitted.

[0046] Power supply 212 is a circuit or module used to provide power for the operation of various parts of the information processing device 100. Power supply 212 may also be configured to include a battery.

[0047] <Explanation of Competitive Training>

[0048] Before describing the functional structure of the information processing device, the adversarial training in this embodiment will be explained. As mentioned above, adversarial examples are images or moving images with added noise that cause the neural network before adversarial training to output misclassified categories instead of the original classification categories. The robustness of adversarial examples can be defined by the minimum amount of noise required to change the classification result of the neural network with specific data. The magnitude of the added noise is also called the norm. For the original sample, the greater the noise required to make it an adversarial example, the harder it is to fool a human. For example, if the original sample X is arranged in a two-dimensional space, the magnitude of the noise (i.e., the norm) can be represented by the radius of distance X. That is, when a permissible norm is specified by user settings, the less the classification result of each sample in a specific dataset (e.g., training data) changes within that norm, the higher the robustness of the model can be considered.

[0049] There are various methods to improve robustness against adversarial examples. In this embodiment, we will use adversarial training as an example to illustrate this method. Adversarial training is learning using adversarial examples as training data. In this case, the correct answer data in the training data does not represent the category to which the adversarial example was misclassified, but rather represents the original correct answer category before adversarial samplerization. That is, the neural network is trained to not misclassify (i.e., classify) the input adversarial example as the original correct answer category.

[0050] Figure 4 and Figure 5 The diagram illustrates the difference between classifying samples using conventional learning and classifying samples using adversarial training. Figure 4 The left and right sides, with boundary line 400, schematically illustrate the boundary where the neural network correctly classifies the original learning data before adversarial training. Because the samples have not yet been adversarially sampled, the neural network is able to classify each sample correctly.

[0051] Figure 4 The right side schematically illustrates misclassification occurring when adversarial examples exist within a certain norm range from each of the examples. (Enclosed) Figure 4 The squares on the right side of the sample schematically represent regions within a certain norm of each original learning data point (black dot 412 and white dot 414). Additionally, black triangle 416 and white triangle 418 represent adversarial samples that caused misclassification. Figure 4In the right-hand figure, the norm range of some samples overlaps with the regions of other categories (beyond the boundary line 400), so there are adversarial examples (black triangle 416 and white triangle 418) in the "regions within a certain norm of the original learning data".

[0052] exist Figure 5 The diagram illustrates the difference between the boundary line 400 formed by learning the original samples and the boundary line 500 formed by adversarial training. Figure 5 The left side represents Figure 4 The right side shows the situation regarding the existence of adversarial examples. Adversarial training learns by using adversarial examples in a way that there are no adversarial examples in the "region within a certain norm of the original learning data," thus making the boundary for classifying samples a boundary line of 500. Therefore, since the "region within a certain norm of the original learning data" does not exceed the boundary line of 500, there are no adversarial examples.

[0053] in addition, Figure 4 and Figure 5 The "region within a certain norm of the original learning data" shown corresponds to the noise level within the range that satisfies the robustness requirement. That is, when generating a neural network that has undergone adversarial training, it is necessary to pre-define the robustness requirement, specifying the maximum norm of robustness, and then use adversarial examples of that norm to train it.

[0054] Furthermore, it is preferable to ensure that adversarial examples are absent in regions within a certain norm of the learning data, but this does not guarantee that learning will proceed in this manner. Additionally, like learning with a typical deep neural network, even if a neural network trained adversarially achieves a high accuracy rate on the samples used in learning and testing, it is not guaranteed that it will always output the correct classification result. On the other hand, there is a possibility that misclassification of adversarial examples can have serious consequences. Therefore, in this embodiment, through formal verification as described below, it is possible to predict the degree of robustness of the neural network trained adversarially.

[0055] <Explanation of Formal Verification>

[0056] Next, refer to Figures 6 to 9 The verification of deep neural networks using formal verification in this embodiment will be described. Formal verification ensures, for example, that the system is correct by mathematically proving whether the hardware and software satisfy specific properties. In the verification of this embodiment, formal verification is used to prove whether the deep neural network satisfies specific properties.

[0057] Figure 6 An example is shown to illustrate the application of formal verification to a regression model. Figure 6In the example shown, nodes 600 and 602 represent neurons in the input layer. Nodes 604 and 606 represent neurons in the intermediate layer. Furthermore, node 608 represents a neuron in the output layer. In this regression model example, the output layer outputs a predicted value (in this case, a credit score) corresponding to the input value. Additionally, the numerical values ​​associated with the arrows between nodes represent the weight coefficients connecting the nodes. For simplicity, the bias is set to 0 and omitted in this example. Figure 6 In the example shown, under the constraint that input x2 is less than 3, we verify whether there exists a combination of inputs (x1, x2) whose output (prediction result) is greater than 5. In this example, we verify whether there exists a combination of a credit score greater than 5 and a parameter representing years of service with a parameter representing annual income.

[0058] Figure 7 An example is shown where a solution exists that satisfies the above constraints. In the case where the group (x1, x2) is (5, 0), the sum of the products of the input values ​​and weight coefficients at each node is calculated, and the result is processed using the activation function of the intermediate layer (e.g., ReLU), thus obtaining a credit score of 5.

[0059] Figure 8 An example is shown to illustrate the application of formal verification to a classification model. Figure 8 In the example shown, nodes 800 and 802 represent neurons in the input layer. Additionally, nodes 804 and 806 represent neurons in the intermediate layer. Furthermore, nodes 808, 810, and 812 represent neurons in the output layer. In this classification model example, the output layer outputs the category (in this case, a credit rating) for the input value. The numerical values ​​associated with the arrows between nodes represent the weight coefficients connecting the nodes. In this example, the bias is also set to 0 and omitted. Figure 8 In the example shown, under the constraint that input x1 is less than 3, we verify whether there exists a combination of inputs (x1, x2) that maximizes the value of node 808 in the output layer. In this example, we verify whether there exists a combination of parameters representing years of service and parameters representing annual income that has the highest probability of a credit rating of A.

[0060] Figure 9 An example is shown where a solution exists that satisfies the above constraints. When the group (x1, x2) is (1, 0), the sum of the products of the input values ​​and weight coefficients at each node is calculated, and the result is processed using the activation function of the intermediate layer (e.g., ReLU), thus achieving the highest credit rating of A.

[0061] The above examples demonstrate that the robustness of a neural network trained adversarially can be guaranteed by establishing the absence of adversarial examples within a certain norm of specific data as an attribute, and by proving this attribute. For instance, the weight coefficients shown in the classification model correspond to the weight coefficients obtained through adversarial training, and the constraints in the input correspond to the range of the norm at a specific sample. Furthermore, the classification of the output node corresponds to the output layer (classification result) of the adversarially trained neural network, and the constraints on the output correspond to the probability that the probability of classes other than the original classification is not the highest. Thus, through formal verification, it can be verified (determined) that no adversarial examples exist in the neural network using weight coefficients obtained through adversarial training within a certain norm of the specific data.

[0062] <Example of the functional structure of an information processing device>

[0063] Next, refer to Figure 3 The functional structure of the information processing device 100 will be explained. Figure 3 The functional structure shown can be implemented, for example, by having the processor 204 deploy one or more programs stored in the memory 208 to the memory 202 and execute them. Furthermore, the functional modules described in this embodiment can be combined or separated, and the described functions can also be implemented using other modules. Additionally, what is described as hardware can also be implemented in software, or vice versa.

[0064] The data acquisition unit 310 acquires, for example, data (hyperparameters regarding the structure of the neural network), learning data, and test data of the model to be applied for adversarial training. The data acquisition unit 310 can receive this data sent from the communication terminal 102, or it can acquire data stored in the storage 208 as learning data, test data 330, and model data 334, in accordance with prior user instructions. Furthermore, it can also acquire this data stored in external storage of the information processing device 100, in accordance with user instructions.

[0065] The user setting acquisition unit 312 acquires, for example, user settings regarding the characteristics of adversarial training from the communication terminal 102. These settings include, for example, settings for the adversarial example generation algorithm, the norm (noise level) set during adversarial training, and the number of epochs during adversarial training.

[0066] The processing unit 314 controls each part of the information processing device 100 and controls various operations of the information processing device 100. In addition, the processing unit 314 controls the operations of the adversarial sample generation unit 316, the adversarial training control unit 318, and the formal verification control unit 320 to achieve robustness improvement processing.

[0067] The adversarial example generation unit 316 adversarially samples the learning data acquired by the data acquisition unit 310. Then, the adversarial example generation unit 316 generates adversarial training learning data by associating the generated adversarial examples with the correct answer data representing the original classification categories. Additionally, the adversarial example generation unit 316 also adversarially samples the test data acquired by the data acquisition unit 310. Then, the adversarial example generation unit 316 associates the adversarially sampled data with the correct answer data of the test data, using it as test data for testing the neural network generated using adversarial training. The adversarial example generation unit 316 stores the generated adversarial training learning data and test data, for example, as adversarial example data 332, in storage 208.

[0068] The adversarial training control unit 318 performs adversarial training based on the learning data for adversarial training generated by the adversarial example generation unit 316, the user settings acquired by the user setting acquisition unit 312, and the model data. Furthermore, when the adversarial training ends due to reaching a set number of cycles and accuracy, the adversarial training control unit 318 uses the adversarial training test data to evaluate the degree of misclassification performed by the trained neural network. If the evaluation result obtained using the adversarial training test data does not exceed a certain accuracy, the processing unit 314 can perform adversarial training again, repeating this process until the evaluation result obtained using the adversarial training test data exceeds a certain accuracy.

[0069] The formal verification control unit 320 verifies (determines) that in the neural network with weight coefficients obtained through adversarial training by the adversarial training control unit 318, there are no adversarial examples within a certain norm of the specific data. In this case, the formal verification control unit 320 performs verification using learning data or test data that has not yet been adversarially sampled. The processing unit 314 can terminate the formal verification process if the proportion of data satisfying robustness constraints is greater than a preset value; otherwise, it can perform adversarial training with modified conditions. The formal verification control unit 320 can attach labels to the data of the formally verified model and store it as model data 334 in storage 208.

[0070] The model configuration unit 322 responds to a request from the electronic device 101 or an operation from the administrator of the information processing device 100 by sending the data of the formally verified model to the electronic device 101 and deploying the model data in the electronic data.

[0071] The learning data and test data 330 include learning data containing raw correct answer data and raw test data. Figure 11The diagram illustrates an example of the data structure for learning data and test data 330. Learning data and test data 330 include data IDs, image data, correct answer labels, and data categories. Data IDs represent identifiers for the data. Image data refers to the image data within the learning data and test data. Figure 11 In the example, this image data could be either a deepfake image of a non-existent person's face or an image of an actual person's face. The correct answer label corresponds to the original correct answer data. For example, if the image data is a deepfake image, it is represented as "1," and if it is an image of an actual person's face, it is represented as "0." Those skilled in the art will understand that in the case of a neural network recognizing a person's name, the correct answer label is the name of the person corresponding to the image data. The correct answer label can be recorded in a One-Hot encoding form independent of this data structure. The data type indicates whether the image data is used as training data or as test data. For the data type, it can be randomly assigned during each learning iteration, with a fixed ratio of training data to test data, or it can be fixed. Alternatively, a column for data types can be omitted, and a data structure can be provided for both training and test data.

[0072] Adversarial example data 332 includes learning and testing data used for adversarial training of adversarial examples. Figure 12 The image shows an example of the data structure for adversarial example data 332. Adversarial example data 332 includes data IDs, image data, correct answer labels, data types, generation algorithms, and norms. Additionally, although... Figure 12 Not shown, but may further include the data ID of the original image data used to generate adversarial examples. The data ID is the identifier of the adversarialized image, and the image data is the adversarialized image data. The correct answer label represents the value of the correct answer label of the original image data as is. That is, the deep neural network learns to derive the original correct answer label even when given an adversarial example, without misclassification. The data type is the same as the training data and test data 330. The generation algorithm indicates the type of algorithm used to generate adversarial examples. The types of algorithms can be diverse, such as FGSM (Fast Gradient SignMethod), C&W Attack, JSMA (Jacobian Saliency Map Attack), Boundary Attack, etc. The norm indicates the magnitude of the noise applied when generating adversarial examples. Alternatively, the norm can also be other metrics representing the magnitude of the noise in the adversarial examples, such as the Wasserstein distance.

[0073] Model data 334 includes data (hyperparameters regarding the structure of the neural network) of the model to be subjected to adversarial training, either pre-acquired or sent from communication terminal 102. It also includes data (hyperparameters regarding the structure of the neural network) and weight coefficients of the model after adversarial training generated by adversarial training control unit 318. Alternatively, model data 334 may also include data of a labeled model whose formal verification has been completed using formal verification control unit 320. Figure 13 The diagram illustrates an example of the data structure for model data 334. Model data 334 includes a model ID, weight coefficients, hyperparameters, generation date and time, and generation stage. The model ID is an identifier for the neural network. The weight coefficients are the weights obtained during the learning of the neural network. Hyperparameters may include data such as the number of layers, the number of nodes in each layer, and the network structure (e.g., CNN, kernel size, etc.). The generation date and time indicate the date and time the model was generated. The generation stage is information used to identify whether the model has undergone formal validation, adversarial training, or is a raw model.

[0074]

[0075] Next, for the series of actions of robustness improvement processing performed in the information processing device 100, refer to Figure 10 This will be explained in more detail. Furthermore, this process is implemented by the processor 204 deploying the computer program stored in the storage 208 to the memory 202 and executing it. In the following description, for ease of explanation, the processor 204 will be described as the main processing unit for each step; however, the corresponding processing is performed by the processing unit 314 and other units according to the processing content.

[0076] First, after the processor 204 acquires the learning data and the structure of the neural network (hyperparameters, etc.) (S1001), it acquires the user settings (S1002). The processing order of S1001 and S1002 can also be reversed, or data from both sides can be acquired simultaneously from the communication terminal 102. The user settings include the norm (noise level), the adversarial example generation algorithm, the number of epochs for adversarial training, the learning rate for adversarial training, the learning algorithm for adversarial training, and the threshold for evaluating the test data used in adversarial training. The user settings may also include specifying which data norm does not contain adversarial examples during formal verification. The learning algorithm for adversarial training includes specifying algorithms such as gradient descent and Adam.

[0077] Next, processor 204 generates adversarial examples (S1003). The generation of adversarial examples is performed based on adversarial example generation algorithms included in the user settings. If no algorithm is specified in the user settings, algorithms that can be used as adversarial example generation algorithms can be executed sequentially to generate adversarial examples generated by various algorithms. In this way, adversarial training that improves robustness not only for adversarial examples with specific characteristics but also for a wide variety of adversarial examples can be implemented.

[0078] Processor 204 performs adversarial training (S1004) based on the learning data acquired in S1001 and the user settings acquired in S1002. Processor 204 performs adversarial training to enable the neural network to learn using learning data including adversarial examples and correct answer data representing the original classification category. Furthermore, the original neural network prior to adversarial training outputs a misclassified category instead of the original classification category when given an adversarial example as input.

[0079] Next, the processor 204 uses the test data obtained in S1001 to test the actions of the neural network generated by adversarial training (S1005), and determines whether the accuracy obtained by the test is above a threshold (S1006). If the accuracy is not above the threshold, the processor 204 advances the processing to S1007, changes one of the settings (S1007), and executes the processing of S1003 to S1005 again according to the changed settings.

[0080] In S1007, the processor 204 changes one of the setpoints. For example, if the test performed using the test data in S1006 fails because it does not achieve a specified accuracy, the norm size can be reduced (i.e., one setpoint). In other cases, for example, only the accuracy of adversarial training on the test data can be set as a user setting, and the norm size (i.e., one setpoint) can be changed from a smaller value to a larger value each time S1007 is executed. In this way, the norm size can be gradually increased by maintaining the adversarial training test results above the specified accuracy using the processing in S1003 to S1005. Thus, adversarial training can achieve the maximum possible increase in norm (i.e., improved robustness) while maintaining the accuracy of the deep neural network, considering the relationship between the decrease in accuracy relative to the original samples (which are not adversarial samples) when improving robustness through adversarial training. In this case, in S1006, the processor 204 can determine whether the norm change has been performed a specified number of times while maintaining an accuracy above the specified level; if so, the processing proceeds to S1008. Alternatively, it is possible to use the above method to perform adversarial training with a single adversarial example generation algorithm that exceeds the maximum norm of the specified accuracy, and to perform adversarial training with multiple other adversarial example generation algorithms that also exceed the maximum norm of the specified accuracy. From the deep neural networks generated by these adversarial trainings, the user can select the desired deep neural network structure from the perspective of accuracy or the characteristics of the adversarial example generation algorithm.

[0081] In addition, the processor 204 can also repeatedly perform the processing steps S1003 to 1005 while keeping other settings fixed among multiple user settings and changing one setting. For example, by repeatedly processing by changing the adversarial example generation algorithm, adversarial examples with various characteristics can be generated. By performing adversarial training on these adversarial examples, the robustness of the generated deep neural network can be improved. Furthermore, there are many cases where it is difficult to know which setting value will yield the best results, such as the number of epochs during adversarial training, the learning rate during adversarial training, and the learning algorithm during adversarial training. Therefore, it is more advantageous for the user to determine the value that yields the best results based on the results obtained by gradually changing these setting values ​​by the processor, rather than setting them based on intuition and experience.

[0082] On the other hand, if the accuracy is above a threshold, the processor 204 performs a robustness evaluation using formal verification (S1008). Furthermore, the robustness evaluation using formal verification can be implemented using known techniques. For example, the techniques for proving properties of deep neural networks described in the known document "Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks" (Guy Kats, et al., May 19, 2017, URL: https: / / arxiv.org / pdf / 1702.01135.pdf) can be used. The processor 204 takes the deep neural network generated through adversarial training as the object and sets the attribute that the maximum output of the output layer is outside the category of the correct answer data within the specified norm (the value of the norm specified in adversarial training) of the original learning data or test data as the query, thereby performing formal verification on the absence of a combination of inputs corresponding to this attribute. If the result is that no such combination exists, the processor 204 proves that in the neural network using weight coefficients learned through adversarial training, there are no adversarial examples within the specified noise range of the specific data. If a combination of inputs corresponding to the attribute is found, it proves the existence of an adversarial example within the specified norm of the specific data. Furthermore, in this case, the existing combination of inputs is returned as a counterexample. During formal verification, if the proportion of robust data exceeds a pre-set value, the processor 204 determines that no adversarial examples exist within the specified noise range of the specific data, and terminates the robustness improvement process.

[0083] As explained above, this embodiment, after obtaining user-defined settings regarding the characteristics of adversarial training, uses learning data including adversarial examples and correct answer data representing the original classification categories, along with the user settings, to perform adversarial training that enables the neural network to learn. Furthermore, before adversarial training, the neural network outputs a misclassified category instead of the original classification category when given an adversarial example. Then, by executing a formal verification algorithm, it is determined that no adversarial examples exist in the neural network using the weight coefficients learned through adversarial training within a specified noise range for the specific data. In this way, a deep neural network capable of predicting specific robustness when using adversarial training can be generated.

[0084] The invention is not limited to the above-described embodiments, and various modifications and alterations can be made within the scope of the invention's intent.

Claims

1. An information processing apparatus for image recognition, characterized by, include: More than one processor; and A memory that stores more than one program. When one or more programs are executed by one or more processors, the information processing device performs the following steps: Obtain user settings regarding the characteristics of adversarial training, including the magnitude of noise in the adversarial examples. The adversarial training is performed using learning data including the adversarial examples and correct answer data representing the original classification categories, along with the user settings, to enable the neural network to learn. The adversarial training causes the neural network to learn to output misclassified categories when given the adversarial examples, so that it can output the original classification categories when given the adversarial examples. A formal verification algorithm is executed to determine whether there are adversarial examples in the neural network using the weight coefficients learned through the adversarial training within a specified noise range of the specific data. The one or more processors, The steps include repeatedly performing the adversarial training by changing the noise range of the adversarial examples, and determining whether the neural network learned through the adversarial training meets the specified accuracy rate. Corresponding to the determination that the neural network learned through adversarial training meets the specified accuracy, the formal verification algorithm is used to determine that there are no adversarial samples in the neural network using the weight coefficients learned through the most recent adversarial training within a specified noise range of specific data.

2. The information processing apparatus as described in claim 1, characterized in that: The step of determining whether the neural network learned through the adversarial training meets the specified accuracy rate includes determining whether the accuracy rate of the test performed using the neural network learned through the adversarial training and the test data exceeds a specified threshold.

3. The information processing apparatus as described in claim 1, characterized in that: The settings include an algorithm for generating adversarial examples. The one or more processors perform the adversarial training by modifying the algorithm for generating the adversarial examples.

Images