Model parameter recovery device, method, and program

Abstract

To provide a technology that eliminates the need to re-learn parameters of a model in order to return the parameters of the model to their pre-attack state.SOLUTION: A model parameter recovery device comprises a change determination unit 2 that determines whether a predetermined change has occurred in parameters of a model based on an error correction code assigned to the parameters, and a recovery unit 3 that recovers the parameters based on the error correction code when the change determination unit 2 determines that a predetermined change has occurred in the parameters.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The disclosed technology relates to a technology for recovering model parameters.

Background Art

[0002] Regarding AI, or in other words, attacks on model parameters and countermeasures against them, various studies have been conducted. Although many countermeasures for preventing attacks have been proposed, there are not many technologies for continuing to use the model parameters that have been attacked.

[0003] As technologies for continuing to use the model parameters that have been attacked, the technologies of Non-Patent Documents 1 and 2 are known.

[0004] The technology of Non-Patent Document 1 is a technology for detecting a backdoor attack and optimizing a retraining dataset for returning the model parameters to the state before the attack when the backdoor attack is detected.

[0005] The technology of Non-Patent Document 2 is a technology for detecting the state of being attacked by a backdoor attack and returning to the state before the attack by changing the parameters.

Prior Art Documents

Non-Patent Documents

[0006]

Non-Patent Document 1

Non-Patent Document 2

[0007] In the technologies described in Non-Patent Documents 1 and 2, it is necessary to retrain the model parameters to return them to their state before the attack.

[0008] The disclosed technology aims to eliminate the need to retrain the model's parameters in order to return them to their pre-attack state. [Means for solving the problem]

[0009] A model parameter recovery device, which is one embodiment of the disclosed technology, comprises a change determination unit that determines whether a predetermined change has occurred in the model parameter based on an error correction code assigned to the parameter, and a recovery unit that recovers the parameter based on the error correction code if the change determination unit determines that a predetermined change has occurred in the parameter. [Effects of the Invention]

[0010] According to the disclosed technology, retraining of the model parameters is unnecessary to return the model parameters to their pre-attack state. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 shows an example of the functional configuration of a model parameter recovery device. [Figure 2] Figure 2 shows an example of the processing procedure for a model parameter recovery method. [Figure 3] Figure 3 shows an example of the functional configuration of a model learning device with recovery capabilities. [Figure 4] Figure 4 shows an example of the processing procedure for a model learning method with recovery capabilities. [Figure 5] Figure 5 shows an example of a computer's functional configuration. [Modes for carrying out the invention]

[0012] Embodiments of the disclosed technology will be described below with reference to the drawings. Note that components having the same function are numbered identically in the drawings, and redundant explanations are omitted.

[0013] [Model parameter recovery device and method] As shown in Figure 1, the model parameter recovery device includes, for example, a model storage unit 1, a change determination unit 2, and a recovery unit 3.

[0014] The model parameter recovery method can be implemented, for example, by having each component of the model parameter recovery device perform the processing shown in steps S2 to S3 in Figure 2.

[0015] <Model Memory Unit 1> The model memory unit 1 stores multiple parameters that make up the model. The model is, for example, a deep neural network.

[0016] If the model is a deep neural network, each parameter is a weight vector corresponding to the edges connecting the nodes.

[0017] An error correction code is assigned to all or part of a plurality of parameters that constitute a model. In the model storage unit 1, together with the parameters, the error correction code assigned to the parameters is stored. An example of the error correction code is a cyclic redundancy check.

[0018] For example, an example of part of a plurality of parameters that constitute a model is at least one predetermined parameter among the plurality of parameters that constitute the model.

[0019] When the model is a deep neural network, the parameter that is a weight vector may be an element of a matrix. In this case, an example of part of a plurality of parameters that constitute the model may be the parameters included in a predetermined matrix among the plurality of parameters that constitute the model. [[ID=?]]

[0020] [[ID=?]] Note that the generation of a plurality of parameters stored in the model storage unit 1 and the assignment of an error correction code are performed before the process of step S1 is performed by the model learning device with a recovery function described later.

[0021] <Change determination unit 2> The change determination unit 2 reads a certain parameter from among the plurality of parameters stored in the model storage unit 1 and the error correction code assigned to the certain parameter.

[0022] The change determination unit 2 performs the following processing using the read parameter and the error correction code assigned to the parameter.

[0023] The change determination unit 2 determines whether a predetermined change has occurred in the parameter based on the error correction code assigned to the parameter of the model (step S1). The determination result is output to the recovery unit 3.

[0024] It seems there are some "?" in the original text's ID numbers in the middle part which might be a mistake. I've translated as accurately as possible based on the provided content. If there are any specific requirements or corrections regarding those "?", please let me know.For example, the change determination unit 2 first calculates the amount of change in the parameter based on the error correction code assigned to the model parameter. Of course, the change determination unit 2 may also calculate the amount of change in the parameter using the parameter itself, not just the error correction code. Then, the change determination unit 2 compares the calculated amount of change with a predetermined threshold, and if the calculated amount of change is greater than the predetermined threshold, it determines that a predetermined change has occurred in the parameter.

[0025] An example of a predetermined threshold is a predetermined absolute value. Another example of a predetermined threshold is a representative value of the change in the parameter over the past n times, where n is a predetermined positive integer. An example of a representative value is the mean, maximum, minimum, or mode.

[0026] The amount of change in the parameters over the past n trials can be obtained from the information about the amount of change in the parameters during each learning trial, which is obtained by the learning unit 4 described later. The information about the amount of change in the parameters during each learning trial, obtained by the learning unit 4, may also be input to the change determination unit 2.

[0027] The change determination unit 2 may also determine that a predetermined change has occurred in the parameter if the absolute value of the calculated change is greater than a predetermined threshold, and the calculated change is either positive or negative.

[0028] Furthermore, the change determination unit 2 may determine that a predetermined change has occurred in the parameter if the past n changes in the parameter were positive and the change calculated this time is negative.

[0029] Furthermore, the change determination unit 2 may determine that a predetermined change has occurred in the parameter if the absolute value of the calculated change is greater than a predetermined threshold, and the past n changes in the parameter have been positive, while the change calculated this time is negative.

[0030] The change determination unit 2 may also determine whether a predetermined change has occurred in the parameter by combining it with an existing parameter change detection method based on vector distance or the like.

[0031] For example, the change determination unit 2 may make a final determination that a predetermined change has occurred in the parameter if it determines that a predetermined change has occurred in the parameter based on the calculated change amount described above, and also if it determines that a predetermined change has occurred in the parameter based on an existing parameter change detection method.

[0032] <Recovery section 3> The recovery unit 3 receives the judgment result from the change judgment unit 2.

[0033] If the change determination unit 2 determines that a predetermined change has occurred in the parameter, the recovery unit 3 recovers the parameter based on the error correction code (step S3).

[0034] Recovering parameters based on error correction codes means restoring the parameters to their original values ​​based on the error correction codes.

[0035] The recovery unit 3 may recover only the parameters that have been determined to have undergone a predetermined change, or it may recover not only the parameters that have been determined to have undergone a predetermined change but also other parameters of the model, or it may recover all parameters of the model.

[0036] Other parameters of the model include parameters adjacent to the parameter that has been judged to have undergone a predetermined change, parameters at a distance L from the parameter that has been judged to have undergone a predetermined change, and parameters belonging to the same matrix as the parameter that has been judged to have undergone a predetermined change. L is a predetermined positive integer.

[0037] The parameters adjacent to a parameter that has been determined to have undergone a predetermined change are, in the case of a deep neural network, the parameters corresponding to each edge connected to the node to which the edge corresponding to that parameter is connected.

[0038] Here, if the model is a deep neural network, for example, the distance between parameter P1 and parameter P2 is the number of nodes between the edge corresponding to parameter P1 and the edge corresponding to parameter P2.

[0039] In this case, the parameter at a distance L from the parameter that is judged to have undergone a predetermined change is the parameter that corresponds to the edge between which there are L nodes and the edge corresponding to that parameter.

[0040] When L=1, the parameter at a distance L from the parameter that is judged to have undergone a predetermined change is the parameter adjacent to the parameter that is judged to have undergone a predetermined change.

[0041] In this way, by incorporating recovery mechanisms such as error correction codes into the model parameters themselves, it becomes unnecessary to retrain the model parameters to return them to their pre-attack state.

[0042] After the multiple parameters constituting the model, which have been learned by the learning unit 4 and to which error correction codes have been assigned by the error correction code assignment unit 5, are stored in the model storage unit 1, the learning unit 6, shown by the dashed line in Figure 1, may retrain the multiple parameters constituting the model stored in the model storage unit 1.

[0043] In this case, if the training data used by the learning unit 6 is contaminated with noise or triggers, then changes may occur in multiple parameters that constitute the model stored in the model storage unit 1 as a result of the learning by the learning unit 6, changes that were not intended by the creator of the model stored in the model storage unit 1. We will refer to such unintended changes as a contamination attack.

[0044] Even if multiple parameters constituting the model stored in the model storage unit 1 are changed due to such a contamination attack, the parameters can be restored to their original state using the model parameter recovery device and method described above.

[0045] [Model learning device and method with recovery function] As illustrated in Figure 3, the model learning device with recovery function includes, for example, a model storage unit 1, a learning unit 4, and an error correction code assignment unit 5.

[0046] A model learning method with recovery functionality can be implemented, for example, by performing the steps S4 to S5 shown in Figure 4.

[0047] <Learning Section 4> Learning data is input into learning unit 4.

[0048] The learning unit 4 performs learning based on the input learning data and generates multiple parameters that constitute the model (step S4). The multiple parameters that constitute the generated model are stored in the model storage unit 1.

[0049] The learning unit 4 divides the training data into multiple groups and performs learning based on each of the multiple groups. In other words, the learning unit 4 performs learning multiple times. These multiple learning times may include minor learning, known as fine-tuning learning.

[0050] When performing multiple learning sessions, the learning unit 4 may obtain information about the amount of change in parameters during each learning session.

[0051] Information about the amount of parameter change during each learning session may be output to the change determination unit 2. In this case, the change determination unit 2 determines a predetermined threshold based on the information about the amount of parameter change during each learning session.

[0052] <Error Correction Code Assignment Unit 5> The error correction code assignment unit 5 reads multiple parameters that constitute the model from the model storage unit 1.

[0053] The error correction code assignment unit 5 assigns error correction codes to all or some of the multiple parameters that constitute the read model (step S5). The error correction codes are stored in the model storage unit 1 along with the corresponding parameters.

[0054] In this way, by assigning error correction codes to all or some of the multiple parameters that make up the model, it becomes unnecessary to retrain the model parameters in order to return them to their state before the attack.

[0055] It should be noted that before the error correction code is assigned by the error correction code assignment unit 5, learning may be performed using contaminated learning data with noise or triggers. For example, if the learning unit 4 performs learning multiple times, at least one of those learning sessions may be using contaminated learning data with noise or triggers. Even in this case, if the change determination unit 2 of the model parameter recovery device uses a representative value of the past n changes in the parameter as a predetermined threshold to determine whether a predetermined change has occurred, and n is sufficiently larger than the number of times learning has been performed using contaminated learning data with noise or triggers, the change determination unit 2 can determine whether a predetermined change has occurred.

[0056] [Differentiation] The specific configuration of the embodiments of the disclosed technology is not limited to the configuration described above. The specific configuration of the embodiments of the disclosed technology can be modified as appropriate, without departing from the spirit of the embodiments of the disclosed technology.

[0057] The various processes described in the embodiments of the disclosed technology may be performed not only in chronological order according to the order described, but also in parallel or individually as required by the processing capacity of the device performing the processes.

[0058] For example, data exchange between components of the model parameter recovery device may be performed directly, or it may be performed via a storage unit (not shown).

[0059] [Programs, recording media] The processing of each part of the above-mentioned devices may be implemented by a computer. In this case, the processing content of the functions that each device should have is described by a program. This program is then loaded into the memory unit 1020 of the computer 1000 shown in Figure 5, and the arithmetic processing unit 1010, input unit 1030, output unit 1040, display unit 1060, etc. are operated, thereby realizing the various processing functions of each of the above-mentioned devices on the computer.

[0060] The program describing this process can be recorded on a computer-readable recording medium. Computer-readable recording media are, for example, non-temporary recording media, specifically magnetic recording devices, optical discs, etc.

[0061] Furthermore, this program may be distributed, for example, by selling, transferring, or lending portable recording media such as DVDs or CD-ROMs on which the program is recorded. Alternatively, the program may be stored in the storage device of a server computer and distributed by transferring the program from the server computer to other computers via a network.

[0062] A computer executing such a program first stores the program recorded on a portable recording medium or transferred from a server computer in its own non-temporary storage device, the auxiliary recording unit 1050. Then, when processing is to be executed, the computer reads the program stored in the auxiliary recording unit 1050 into the storage unit 1020 and executes the processing according to the loaded program. Alternatively, the computer may directly read the program from the portable recording medium into the storage unit 1020 and execute the processing according to that program. Furthermore, each time a program is transferred to this computer from a server computer, it may sequentially execute the processing according to the received program. Alternatively, the above processing may be executed by a so-called ASP (Application Service Provider) type service, where the server computer does not transfer programs to this computer, but the processing function is realized only by execution instructions and result acquisition. In this embodiment, the program includes information used for processing by an electronic computer that is equivalent to a program (data that is not a direct instruction to the computer but has the property of defining the processing of the computer).

[0063] Furthermore, in this configuration, the device is configured by executing a predetermined program on a computer, but at least a part of these processing contents may be implemented in hardware. For example, the change detection unit 2, the recovery unit 3, the learning unit 4, and the error correction code assignment unit 5 may be configured by processing circuits.

[0064] It goes without saying that the invention may be modified as appropriate without departing from its spirit.

[0065] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually described as being incorporated by reference.

Claims

1. A change determination unit that determines whether a predetermined change has occurred in the model parameters based on an error correction code assigned to the parameters, If the change determination unit determines that a predetermined change has occurred in the parameter, the recovery unit restores the parameter based on the error correction code, A model parameter recovery device that includes this.

2. A model parameter recovery device according to claim 1, The change determination unit determines that a predetermined change has occurred in the parameter when the amount of change in the parameter is greater than a predetermined threshold. Model parameter recovery device.

3. A model parameter recovery device according to claim 1, The recovery unit recovers not only the parameter but also other parameters of the model. Model parameter recovery device.

4. A change determination step in which the change determination unit determines whether a predetermined change has occurred in the model parameters based on an error correction code assigned to the parameters, If the recovery unit determines that a predetermined change has occurred in the parameter, the recovery unit performs a recovery step of recovering the parameter based on the error correction code. A model parameter recovery method that includes this.

5. A program for causing a computer to function as a component of any of the model parameter recovery devices according to claims 1 to 3.

Images