Systems and methods for creating and executing secure neural networks

Abstract

Disclosed implementations include a method for creating a secure neural network model. Programmed generated transcoding layers may be added to the inputs and outputs of existing neural network models. The transcoding may be generated using a deterministic algorithm, and the input data may be protected with known data transformations by obscuring and transforming the input and output data of the model.

Description

Background Art

[0001] Artificial intelligence (AI) refers to a computer model that simulates the cognitive process of human thinking. Recently, AI has found many applications. For example, ChatGPT is an AI model that interacts with users to provide information and creative works in a conversational manner. In addition, autonomous and semi-autonomous vehicles can use AI to identify objects (such as pedestrians, traffic signs, and other vehicles), and ride-sharing applications can use AI to determine waiting times and real-time ride pricing. One method of AI is machine learning (ML), which is used to find the probability of a certain result using analytical experiments. ML uses a large set of historical "training" data fed into a statistical model to "learn" one or more specific tasks such as facial recognition. The more training data used, the more accurate the ML probability estimate will be. The corollary is that if, for example, corrupted and / or abnormal data is input into the ML model by an attacker, the ML model can be rendered inaccurate and / or inoperable. Of course, this presents security issues in ML applications.

[0002] Various ML algorithms are known (e.g., ADAP and RMSProp). ML models can be implemented using "neural networks," also known as "artificial neural networks" (ANNs). Neural networks mimic the way biological neurons in the human brain signal each other. Neural networks consist of multiple layers of nodes, including an input layer, one or more internal / hidden layers, and an output layer. Each node or artificial "neuron" is connected to another and has an associated weight and threshold. If the output of any individual node is above a specified threshold, the node is activated, sending data to the next layer of the network.

[0003] Neural network models represent mathematical functions. In this way, they are similar to traditional computer software, but they are expressed in a different language (not human-readable) and are often discovered through computation rather than authored (machine learning, not engineering). Often, these models perform operations on sensitive data, such as making a cancer diagnosis or calculating loan default risk. Furthermore, models implemented by neural networks represent a considerable investment in intellectual property that is worth protecting.

[0004] As noted above, neural networks are typically architected as a collection of layers, where each layer includes neurons that perform computational operations. Neurons in one layer are connected to neurons in the proceeding layer using weighted synapses, and in this way, data flows forward through the neural network. The first layer in which data enters the neural network is called the "input layer." The values ​​held in the input neurons are propagated forward to the neurons in the hidden layers. The final layer, called the "output layer," reports the final numerical results of the neural network processing.

[0005] The input layer of a neural network can accept numerical data, which corresponds to various real-world features, for example. For image processing neural networks, these features may be pixel intensities (such as numbers between 0 and 255) corresponding to X and Y coordinates in an image. For cancer diagnostic models, the input may be, for example, a description of a tissue or even a blood test result. Each feature typically corresponds to an input neuron. The same is true at the output. In other words, conventional neural networks typically have a one-to-one mapping between data and meaning. For example, a cancer screening neural network will generally output the presence or absence of a tumor through a neuron (where 0.0 means absence and 1.0 means presence).

[0006] In conventional neural network architectures, there are no mechanisms for protecting data. The one-to-one mapping between data and meaning represents a significant security vulnerability. For example, if Bob knows what neural network is being used to process data about Alice, Bob can easily determine details about Alice by looking at what numbers are assigned to the input neurons (even if Bob does not have access to the model). If Bob can see the output (even without knowing what the input is), he may even be able to learn some very private things about Alice (such as a cancer diagnosis or credit score). A skilled attacker can even reverse engineer the model implemented by a neural network by using an ML algorithm to "infer" the behavior of the model by passing in a lot of data and examining the output.

[0007] Data "masking" is the process of modifying sensitive data in such a way that it has little or no value to an unauthorized intruder, yet is still usable by software or authorized personnel. Data obfuscation can be used in programmed computational algorithms to protect information that is classified as personally identifiable information or mission critical data, or that otherwise needs to be maintained as secure. However, the data must remain available for the purpose of conducting an effective test cycle. Due to the one-to-one mapping between data and meaning noted above, conventional neural network architectures are not easily adaptable to use obfuscated data. Adapting existing data obfuscation methods (such as finite ring encoding or even simple XOR-based methods) is difficult because a) we are limited in what mathematical operations we can apply to the data, and b) we are restricted to working with numbers as continuous scalar representations (as opposed to finite sequences of bits). For this reason, existing masking methods are not practical for use in neural networks.

[0008] In summary, conventional neural network architectures have no practical mechanism for accepting protected data. Incoming data is almost always "clear", and this presents a security / privacy issue. Service providers who may want to use neural networks must deal with the privacy requirements of their end users (which are often mandated by law, such as GDPR in the EU) and the risks of exposing their models to the outside world through public interfaces (e.g., APIs). This is especially true in a hosted environment like AMAZON WEB SERVICES (AWS), where neural networks perform computations off-site in the cloud.

[0009] While there are many known solutions for protecting data flows (such as SSL and obfuscation), these protections end when the data is presented to the runtime environment that executes the neural network model (e.g., KERAS, TORCH, TENSORFLOW, etc...), as the models themselves are designed to accept "raw" or clear data. Summary of the invention

[0010] The disclosed implementation extends the protection boundaries of neural networks and is therefore more compliant with principals for data security and privacy. The disclosed implementation includes a method for embedding an implementation of an encoding mechanism (not created with machine learning) into an existing neural network to provide a more secure data interface to the neural network.

[0011] A disclosed implementation includes a method for protecting a model executed by a neural network, the method comprising: receiving an input specification data structure specifying inputs to an original input layer of an original neural network, wherein the original neural network is constructed to execute the model; fractionalizing the inputs; specifying weightings that define how much each fragmented input contributes to a neuron in the original input layer; storing the weights in a weight table; creating a protected input layer based on the fragmented inputs and the weight table; integrating the protected input layer into the original first hidden layer; and then adding the remainder of the original neural network as is (or with further processing as needed, such as other mechanisms to support other data protection strategies). BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The foregoing summary of the invention and the following detailed description of the present invention will be better understood when read in conjunction with the accompanying drawings. For the purpose of illustrating the present invention, various illustrative embodiments are shown in the accompanying drawings. However, it should be understood that the present invention is not limited to the precise arrangements and means shown.

[0013] Figure 1 is a block diagram of the architecture of a protected neural network according to a disclosed implementation.

[0014] Figure 2 is a block diagram illustrating an example of an input decoder layer according to a disclosed implementation.

[0015] Figure 3 is a flow chart of a method for creating a protected neural network according to a disclosed implementation. DETAILED DESCRIPTION

[0016] Certain terms used in the following description are for convenience only and are not limiting. Unless specifically set forth herein, the terms "a", "an", and "the" are not limited to one element, but should instead be interpreted as meaning "at least one". The term includes the words noted above, their derivatives, and words of similar meaning.

[0017] The disclosed implementation adds programmatically generated "transcoding" layers to the input of an existing neural network model (model definition only, not runtime). These layers can be generated using deterministic algorithms and can operate similarly to data transformations used in conventional data obfuscation techniques in that they protect data by obfuscating and transforming it (similar to encryption algorithms).

[0018] The architecture of the disclosed implementation allows processing of inputs that are encoded representations of data. The encoding embedded in one model is not necessarily the same as the encoding embedded in another model. Instead of each input corresponding to a single value (e.g., 3.14159), the input can be re-encoded as a set of multiple values ​​(e.g., [-0.9987, 0.3350, -0.5608, 8.0133, -0.5724] = 3.14). This is of course extendable to any number of inputs. The matrix for decoding the encoded values ​​can be integrated into the first hidden layer of the protected neural network so that the original data is not reconstructed in its original form (only the hidden neuron accumulator is constructed).

[0019] Since the inputs to the model are obfuscated, protected models built according to public implementations are more difficult to reverse engineer. Cloning and adversarial attacks against models protected in this way are more difficult to accomplish, since additional (secret) information is required to infer what the underlying model is doing. Side-channel attacks (such as monitoring memory consumption to determine what is in memory) will also be more difficult with the proposed approach, since the original unobfuscated form of the data is not directly present.

[0020] Unique protected instances of existing neural networks can be generated in a way that keeps user data private. Each user (or even session) can have a unique corresponding private and protected version of a neural network that uses an obfuscated set of parameters (embedded weights and biases). This locks the neural network model to a specific use case and makes it incompatible with all other instances of the model. While it is possible that someone with access to the weights and biases of a protected model could reverse engineer everything to extract the original model, this is time consuming and resource intensive (depending on the situation). Furthermore, an attacker who could reverse engineer the protections of one instance of a neural network still does not gain access to all other instances of that model. The protections, and therefore the interface, are unique for each model.

[0021] For example, an AI / ML-based service that can provide medical diagnosis or monitor patients for vital signs can provide medical information to third parties, thus requiring an entire chain of actors that may or may not be trustworthy and / or able to protect the data. However, medical institutions, patients, and government regulators will want to maintain the privacy of the data. Even if an attacker captures the data that goes into a protected model (e.g., after the data has been decrypted and stored in memory), the attacker will not know what the data means and the attacker may not send the data to another copy of the protected model to check the output because the data is tied to a specific instance. This also limits the possibility of accidental disclosure (such as inadvertently saving model inputs to disk). The disclosed implementation can be used in conjunction with other cryptographic solutions and is useful for AI / ML deployments where it is important to maintain the privacy of user data and / or limit access to the model.

[0022] Figure 1 The architecture of a protected AI / ML model 100 including a neural network according to a disclosed implementation is illustrated. A method for creating a protected model is described later. Instead of accepting "clear" unencoded data in its original form, the protected model 100 accepts an encoded data representation. In other words, the data can be encoded before it is sent to the neural network. The encoding can be done in a conventional manner on any device such as a smartphone, a desktop PC, or an edge device. The protected model 100 includes a neural network 102, which can include a conventional neural network 102a. The protected model 100 also includes an input decoder layer 102b and an output encoder layer 102c. The input data can be sent to any service that hosts the protected model 100, and the processing can occur in a manner similar to a conventional AI / ML model (e.g., no special runtime or other requirements are required). However, the neural network of the protected model 100 has a built-in decoder layer 102b that converts the encoded input data into data compatible with the internal workings of the neural network 102a. The input data may be encoded in a manner compatible with the transcoding layer using, for example, an encoder AI 104a executing on a device such as a smartphone, edge device, or other computing device.

[0023] The neural network 102 can process data in the same way that an unprotected neural network model would process data. Instead of producing a "raw" prediction, the output encoder layer 102c encodes the output into a "cloaked" form. The hidden model output is decoded into a clear form at its destination using a decoder API 104b executed, for example, on a device such as a smartphone, edge device, or other computing device. All of this can happen in situ relative to the model and is fully compatible with existing AI / ML frameworks. Converting data to and from a protected format (input and output of the neural network) can be done via the API, so from an engineering perspective, the only real change to the workflow is that all "client-side" - i.e. data needs to be encoded and decoded in the conventional way.

[0024] The disclosed implementation may use a two-part approach for encoding and decoding that can be integrated into existing neural network models in a manner that does not require any changes to the existing runtime environment of AI / ML. Additionally, the method does not require the cooperation of data scientists building the protected model. In the first part of the encoding method, the representation of data entering and exiting the protected neural network is changed by redistributing what would normally be the value of a single neuron across multiple neurons (referred to herein as "fragmentation"). In the second part of the encoding method, the order in which the neurons are connected is randomized. These two encoding method parts will be discussed in more detail below. The protected model 100 receives data that looks more like random noise than a structured signal to an external observer, thereby hiding the processing that is occurring. New encodings can be generated in as many unique instances as desired.

[0025] The input decoder layer 102b and the output encoder layer 102c can be fragmented layers, that is, layer 102b can accept many encoded inputs and combine them into a single decoded value, and layer 102c can accept a single input and encode it into multiple inputs. There are many ways to do this. For example, each "fractional" neuron can correspond to a power of two (1, 2, 4, 8, 16, 32...), similar to how binary encoding works. In another example, each of the fragmented neurons can correspond to a fragment using the value π (1 / π, 2 / π, 3 / π,...). The fragmented neuron can even correspond to an arbitrary random value (e.g., 1, -4.6, 1235, 0.234234, 170778, 98). Since a single number can be represented as the sum of states, the number can be represented in a manner compatible with the architecture of many conventional neural networks. Once the fragmentation mechanism is selected, an interface to an existing neural network can be constructed by adding a fully connected (dense) layer to the input and assigning weights in a way that replicates the fragmentation.

[0026] As an example, Figure 2 As shown in , the unprotected original model can be implemented by a neural network 202a, which expects "weights" W and "height" H as input layer 202a 输入 The input encoder layer 202b can use a simple "power of two" encoding function (similar to binary) to fragment the input across four neurons for each of the features W and H. In other words, there will be four neurons to capture the input "weight" and four neurons to capture the input "height". In this example, a new "de-fractionalizing" input encoder layer 202b consisting of 8 input neurons is created. The first four de-fragmented input neurons use X f1 =1.0,X f2 =2.0, X f3 =4.0 and X f4 =8.0 to the original "weight" W input neuron. The connection weights for these first four inputs to the original "height" H neuron may all be set to 0, and thus have no contribution to the H neuron. A second set of 4 defragmented neurons may be connected to the original "weight" neuron with a value of 0, but to the original "height" neuron with values ​​of 1.0, 2.0, 4.0, and 8.0, respectively, thereby contributing to the H neuron instead of the W neuron.

[0027] exist Figure 2 In the example above, passing a fragmented input of 1, 0, 0, 1, 0, 0, 1, 1 would correspond to an input of weight = 9 (1x1 + 1x8) and an input height = 12 (1x4 + 1x8). To manage the output, we just work backwards – adding a new fully connected output encoder layer where the weights from the original output to the fragmented output encoder layer determine how the numbers are distributed.

[0028] Consider the example of a neural network that accepts "true" and "false" values ​​- even if the state is spread across an increasing number of neurons corresponding to some power of two, all the information will still be concentrated in one or two places. The following table illustrates similar inputs, but the inputs are expressed as different numbers of fragmented inputs.

[0029]

[0030] It is more desirable to use an encoding scheme that "spreads" values ​​across input neurons in a way that does not reveal anything about how decoding might work. This can be achieved by carefully choosing how to assign weights within the fragmented layers. Some specific examples are discussed below. One approach would be to use a random number generator to find the encoding for a particular value. In this example, the same value can produce multiple encodings, and the encoded value gives little information about the encoding method.

[0031] A number N may be chosen to represent the number of fragments for a source number. A weight table may be created, e.g. a table of N+1 entries, where each entry is just a random number between, e.g., -1.0 and +1.0 (although this parameter range may be tuned for a particular application). The last entry at N+1 may be a special bias term. A source number x may be transcoded by performing the following steps:

[0032] Select a set of N+1 numbers (weight table) as weights. Call these w1 to w n And the last number is called b (for bias).

[0033] To encode x, we randomly select N-1 numbers e1 to e n-1 And e n Calculated as:

[0034] e n =[x-sum(i=1..n-1,w i e i )+b] / w n

[0035] Choose the numbers in e, the contribution table, to represent the encoded values ​​relative to the weights and biases.

[0036] Following is an implementation of this method written in Python.

[0037]

[0038] Figure 3A method 300 for creating a protected model implemented by a neural network according to the disclosed implementation is illustrated. The method 300 creates a unique fuzzy interface to an existing neural network to be protected. At 302, the inputs of the neural network are fragmented by, for example, expanding the number of inputs accepted by the neural network. Where the original neural network may only accept two inputs (A and B), the new protected neural network may have many inputs so that the original data is hidden inside multiple values. This also provides greater flexibility when considering encoding / decoding input values. For example, each original input can be fragmented to be represented by 5 values. This means that the new interface in this example will expect 10 inputs; five corresponding to input A and five corresponding to input B. At step 304, weights can be set to define how much a fragmented input neuron contributes to the neurons in the next layer (the original input layer). Each weight of the first hidden layer is connected to the encoded input by changing their existing weights to compensate for the new fragmented weights. At step 306, the weights and inputs can be stored as a weight table data structure to be used as a basis for creating input decoder layers and output decoder layers. Both steps 302 and 304 may be accomplished based on an input specification data structure that specifies the input variables expected by the original input layer.

[0039] A table of weights or other encoded indicators is used as a shared secret. Since a many-to-one relationship has been defined, values ​​can be assigned to weights that connect each expanded input to the corresponding destination in the existing neural network. These weights act to recombine the input into its original form with respect to each hidden layer neuron. Data that has been encoded with this knowledge will produce valid inputs for the original neural network, while data that has not been prepared with this knowledge will not produce valid inputs for the original neural network.

[0040] The table of weights can be a collection of random numbers, or something that is engineered to exhibit a specific property. In one example, the weight table can be a collection of multiple "0"s and a single "1", so that only one of the extended inputs actually contributes to the original neural network, but it will be difficult for an attacker to know which of the extended inputs is the contributing input. Additional pseudo-random numbers between a certain range of values ​​can be used as weightings, where the range is selected to ensure that the encoding leads to many large numbers (large number) (or in other cases, many small numbers). In another example, the table can be designed to force "guessing" to lead to a certain failure condition within the neural network, such as by designing a weight table in which guessing "1" as an input will always produce a "negative" model output. In addition, the weight table can be configured to require the presence of specific attributes (such as signatures).

[0041] An example of code for creating a weight table is set forth below.

[0042] # Make a "secret key" set def makeRandomFractionalTable(tableLen=10, minVal=-1.0, maxVal=1.0): ft=[] for i in range(0, tableLen): v=random.uniform(minVal, maxVal) ft.append(v) return ft --END-

[0043] At step 308, the weights can be embedded into the neural network as a new first layer (input decoder layer), wired directly into the first hidden layer of the unprotected network. Embedding this table into an existing network can be done using an API that allows parsing the original network (such as the Python API Keras). The following is an example of code for embedding a weight table.

[0044]

[0045] The following code can be used to fragment the values ​​v across the weight table. These become the encoded inputs to the already protected ANN.

[0046] fractionalizeInputValue(weightTable, v, bias=0, minC=-1.0, maxC=1.0, targetIndex=0): 4tableLen=len(weightTable)5e=[0]*tableLen 6v=v+bias 7 8iftargetIndex===-1: 9targetIndex=random.randint(0, tableLen-1)10 11cSum=012for i in range(0, tableLen): 13if i! =targetIndex:14e[i]=random.uniform(minC,maxC)15cSum+=e[i]*weightTable[i]16 17e[targetIndex]=(v-cSum) / weightTable[targetIndex]18return e--END-

[0047] The result is a list of numbers e that serve as input to the protected neural network. Passing this contribution table to the neural network will cause the new first layer to decode the numbers into inputs (features) in the hidden layer. Thereafter, the network can operate in the conventional manner.

[0048] In the disclosed implementation, a new set of inputs and biases is created. This becomes the new input layer for the network and is connected to the old first hidden layer. As an example, if we want to send the number 3.14 to the network, we can complete the steps above to calculate the "contribution table". Because random numbers are used, the same value can be encoded in an almost infinite number of ways. For example, e = fractionalizeInput (weights, 3.14) -> [0.1586, 0.2986, -0.8057, 8.3496, -0.9053]. When decoded by recombining with the selected weights, this will = 3.14 when decoded. Other protections can be added to the network. For example, the order of the inputs can be randomized so that consecutive encoded inputs no longer correspond to consecutive original inputs (e.g., A3, B2, B1, A1, A4, B4, A2, B3).

[0049] If we only have two (or a few) neurons to shuffle, randomizing the order would not be practical. However, since we can make the network consist of any number of inputs we want, randomization can be very useful. In other words, because encoding maps a single scalar to an arbitrary length vector, other methods (such as order randomization) are more useful. In step 310, an output encoding layer is created and connected to the last hidden layer of the network (the output layer of the original network).

[0050] Various known encoding techniques, such as data transformations, can be applied to accomplish encoding of the input to the protected neural network. The disclosed implementation is compatible with conventional neural networks and AI / ML runtimes. A "protected" model is still just a model. The methods of the disclosed implementation can be automated. The only change to the end user is an additional encoding step that occurs before the model has been run.

[0051] The disclosed implementation methods may be accomplished by one or more computing devices including functional "modules" consisting of code executable by a computer processor to perform the above functions. The computing devices implementing the disclosed implementations may include a variety of tangible computer-readable media. Computer-readable media may be any available tangible media that can be accessed by a device and includes both volatile and non-volatile media, removable and non-removable media. Tangible, non-transitory computer storage media include volatile and non-volatile, and removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data.

[0052] Various data and codes may be stored in an electronic storage device, which may include a non-transitory storage medium that stores information electronically. The electronic storage medium of the electronic storage device may include one or both of a system storage device and / or a removable storage device that is provided integrally (i.e., substantially non-removable) with the computing device, the removable storage device being removably connectable to the computing device via, for example, a port (e.g., a USB port, a FireWire port, etc.) or a drive (e.g., a disk drive, etc.). The electronic storage device may include one or more of the following: an optically readable storage medium (e.g., an optical disk, etc.), a magnetically readable storage medium (e.g., a magnetic tape, a magnetic hard drive, a floppy disk drive, etc.), a charge-based storage medium (e.g., an EEPROM, a RAM, etc.), a solid-state storage medium (e.g., a flash drive, etc.), and / or other electronically readable storage medium.

[0053] The processor(s) of a computing device may be configured to provide information processing capabilities and may include one or more of the following: a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and / or other mechanisms for electrically processing information. As used herein, the term "module" may refer to any component or collection of components that perform the functionality attributed to the module. This may include one or more physical processors, processor-readable instructions, circuits, hardware, storage media, or any other components during the execution of processor-readable instructions.

[0054] Those skilled in the art will appreciate that changes may be made to the disclosed implementations without departing from the broad inventive concept thereof. Therefore, it will be understood that the invention is not limited to the disclosed implementations, but is intended to cover modifications within the spirit and scope of the invention as defined by the appended claims.

Claims

1. A method for protecting a model implemented as a neural network, the method comprising: an input specification data structure receiving inputs specifying an original input layer of an original neural network, wherein the original neural network is constructed to execute the model; fragmenting the input to create fragmented input; Specify weights that define how much each fragmented input contributes to a neuron in the original input layer; storing the weights in a weight table; creating a protected input layer based on the fragmented input and the weighting table, whereby the protected input layer is operable to decode the encoded input; and Connect the protected input layer to the original input layer to create a security model.

2. The method according to claim 1, wherein: Fragmentation consists of selecting e1 to e n N values ​​are used to represent the input value.

3. The method according to claim 2, wherein: The weighted table consists of N+1 fields, which store weights w1 to w n and deviation value b.

4. The method according to claim 3, wherein: The protected input layer performs a decoding operation based on the following formula to decode the encoded input e n decoding: en=[X-sum(i=1..n-1,Wiei)+b] / Wn 5. A method according to any one of the preceding claims, further comprising randomising the order of the inputs.

6. A method according to any one of the preceding claims, wherein: The weighted table is used as a shared secret for encoding the data input into the security model.

7. A method according to any one of the preceding claims, wherein: The weights are embedded into the neural network as a new input layer of the security model.

8. A security model implemented as a neural network, the security model comprising: an unsafe model implemented as a neural network having an original input layer, an output layer, and at least one hidden layer; and A protected input layer is connected to the original input layer, wherein the protected input layer is created by receiving an input specification data structure that specifies the input of the original input layer, fragmenting the input to create fragmented inputs, specifying weights that define how much each fragmented input contributes to a neuron in the original input layer, and storing the weights in a weighted table, whereby the protected input layer is operable to decode the encoded input.

9. The security model according to claim 8, wherein: Fragmentation consists of selecting e1 to e n N values ​​are used to represent the input value.

10. The security model according to claim 9, wherein: The weighted table consists of N+1 fields, which store weights w1 to w n and deviation value b.

11. The security model according to claim 10, wherein: The protected input layer performs a decoding operation based on the following formula to decode the encoded input e n decoding: en=[X-sum(i=1..n-1,Wiei)+b] / Wn 12. A security model according to any one of claims 8 to 11, wherein: The order of inputs is randomized in the secure input layer.

13. A security model according to any one of claims 8 to 12, wherein: The weighted table is used as a shared secret for encoding the data input into the security layer.

14. A security model according to any one of claims 8 to 13, wherein: The weighted embedding is inside the secure input layer.

15. A system arranged to carry out the method of any one of claims 1 to 7.

Images