Modifying machine learning models to obfuscate their characteristics

Abstract

Methods, systems, and apparatuses, including a computer program encoded on a computer storage medium, for obfuscating operations of a neural network. One method includes receiving data representing a neural network. The neural network includes parameters specifying a sequence of network layers and a plurality of nodes in each layer of the sequence of network layers. The neural network is compiled to generate instructions that, when executed, cause one or more computing units of a hardware device to perform obfuscation operations associated with an inference operation of the neural network. The obfuscation operations, when executed, obfuscate one or more measurable characteristics of the neural network. Compiling includes determining a target layer in the sequence of network layers, determining an obfuscation network structure to associate with the target layer, and compiling the neural network with the associated obfuscation network structure to generate instructions for performing the obfuscation operations specified by the obfuscation network structure.

Description

[Technical Field]

[0001] This specification relates generally to machine learning. In particular, this specification describes techniques for modifying machine learning models to include structures that obfuscate the inference operations of the machine learning models as they are executed. [Background technology]

[0002] Artificial intelligence (AI) is the intelligence exhibited by machines and refers to the ability of a computer program or machine to think and learn. One or more computers can be used to perform calculations to train machine learning models for each task. Neural networks belong to the subfield of machine learning models.

[0003] A neural network can use one or more layers of nodes representing multiple operations, such as vector or matrix operations. One or more computers can be configured to perform the operations or calculations of the neural network to generate an output, e.g., a classification, prediction, or segmentation of the received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e., the next hidden layer or output layer. Each layer of the network generates an output from the received input according to the current values ​​of a respective set of network parameters.

[0004] Specially designed hardware accelerators can perform certain functions and operations, including operations or calculations specified within neural networks, faster and more efficiently than operations performed by a general-purpose central processing unit (CPU). Hardware accelerators may include graphics processing units (GPUs), tensor processing units (TPUs), video processing units (VPUs), field programmable gate arrays (FPGAs), or application-specific integrated circuits (ASICs). Summary of the Invention

[0005] After being properly trained, a machine learning model (e.g., a neural network) can be compiled and deployed on a hardware device configured to perform inference operations to process input data. The inference operations are defined by parameters of the neural network, which are updated during the training process. The parameters define (i) the node operations (e.g., linear and nonlinear operations) of the nodes in each network layer of the neural network and (ii) the structure of the neural network. For example, parameters defining the node operations include parameters defining activation functions for each network layer and parameters defining node weights for the nodes in the network layer. As another example, parameters defining the structure (also referred to as hyperparameters) include at least one of the number of nodes in a network layer, the number of network layers in the machine learning model (e.g., in a neural network), or node connections across adjacent layers (e.g., fully connected layers, convolutional layers, or transposed convolutional layers). For simplicity, the following specification is described in terms of neural networks, but may be applied to other types of machine learning models.

[0006] It is extremely important to keep the parameters of a trained neural network secret. First, training a neural network, especially a deep neural network with sufficient accuracy to generate predictions, requires significant computational cost and time. Furthermore, some neural networks have applications in security sensitive authentication, such as face unlock tasks, where the neural network is configured to recognize faces to conveniently unlock a device. Therefore, it is extremely important to keep the structure and parameters of the neural network unbreakable, or at least difficult to break, to prevent malicious actors from learning the parameters and using them to unlock unauthorized devices.

[0007] However, especially when the neural network is implemented on a hardware device accessible by a third party (e.g., an edge device such as a smartphone, smart watch, smart tablet, or other edge device), various techniques can be applied to "decode" the trained neural network. For example, one technique may measure the characteristics of the trained neural network when the hardware device performs an inference operation on the trained neural network. More specifically, the technique may determine the parameters and structure of the neural network by collecting data, such as power consumption, electromagnetic waves, or time, when the hardware device performs the inference operation and analyzing a characteristic profile generated based on the collected data. This technique is also known as a side-channel attack.

[0008] The techniques described herein can enhance the security of neural networks implemented in hardware devices, such as edge hardware devices. For example, the described techniques defend against side-channel attacks by determining, at compile time, one or more obfuscated network structures associated with one or more original network layers in the neural network and generating instructions that, when executed by the hardware device, cause the hardware device to perform obfuscation operations specified by the obfuscated network structures in parallel and / or sequentially with inference operations of the one or more original network layers.

[0009] One or more original network layers are generally referred to as "target layers" of the neural network. In some circumstances, the system may determine one or more critical layers of the neural network as target layers for obfuscating the operations of the target layers. Target layers (e.g., critical layers) typically require significant time and cost (e.g., computational resource costs) to train. Alternatively or additionally, the target layers (e.g., critical layers) may generally include network layers that are substantially important to the network, such as layers that are important for improving the performance of the neural network. Performance may include hardware resource requirements, time requirements, power requirements, or other requirements for performing the neural network's inference operations for various tasks. A system (or compiler) performing the described techniques may determine the target layer by determining whether a network layer is a critical layer based on one or more characteristics associated with the neural network. Exemplary characteristics may include the type of layer, the size of the layer, the inputs and / or outputs of the layer, or other suitable characteristics. In some implementations, the system may determine that a layer is a critical layer if one or more criteria, such as a threshold memory bandwidth, a threshold power consumption, or other criteria, are met by the layer. In this manner, the described techniques can thwart side-channel attacks or at least raise the bar for the computational and / or time costs of cracking a deployed neural network using side-channel attacks. While these techniques are primarily described with respect to a target layer (e.g., a critical layer), obfuscation layers can be added to or near other non-critical layers. Furthermore, while the term "critical layer" is used for simplicity in the following specification, it should be noted that a "critical layer" may be equivalent to, or determined or selected as, a target layer when operations at the target layer are obfuscated by introducing obfuscation operations executed on a hardware device.

[0010] As used throughout this specification, the term "obfuscation operation" generally refers to an operation that, when performed in conjunction with a machine learning operation (e.g., an inference operation) of a neural network deployed by a hardware device, causes a change in one or more measurable characteristics of the neural network such that at least one parameter of the neural network, e.g., at least one of the number of network layers of the neural network, the number of nodes in the network layers, the node operations of the nodes in the network layers, or the weights associated with the nodes in the network layers, is obscured. Note that different types of machine learning models have different types of parameters that define the model. The techniques described herein can obfuscate any type of parameter that affects the measurable characteristics of a machine learning model.

[0011] The one or more measurable characteristics of a neural network generally refer to data that can be measured when a hardware device performs an inference operation on the neural network. The measurable data may include data or profiles related to power consumption, time, electromagnetic emissions, or other measurable data, as described above.

[0012] It should also be noted that the obfuscation operation may be performed sequentially and / or in parallel with the machine learning operation, depending on the determined obfuscation network structure. As used throughout this specification, the term "parallel" generally refers to a common period when both the obfuscation operation and the inference operation are performed by a hardware device. For example, the common period can be exactly the same period, substantially the same period (e.g., within a threshold period of each other), or two different periods having an overlapping area. As used throughout this specification, the term "sequentially" generally refers to the obfuscation operation and the inference operation being performed in sequence at different periods. For example, the obfuscation operation may be performed before or after one or more inference operations are performed. Different periods generally refer to periods that do not have an overlapping area.

[0013] In situations where obfuscation operations are performed in parallel with inference operations, the obfuscation network structure may include, for example, one or more obfuscation nodes added to one or more critical layers. The obfuscation operations specified by the obfuscation nodes in the critical layers may be performed in parallel with the inference operations of the original nodes in the critical layers. As another example, when a neural network is compiled, obfuscation network structures may be additionally included in the instructions. However, these obfuscation network structures do not change the original structure or parameters of the neural network. Rather, the obfuscation operations specified by these obfuscation network structures are executed by the hardware device in parallel with the inference operations in one or more critical layers. The obfuscation network structure may mimic the operations performed by the critical layers with similar data flow and / or data operations.

[0014] In a situation where obfuscation operations are executed sequentially with inference operations, the obfuscation network structure may include, for example, one or more obfuscated network layers added immediately before or after a critical layer. Even if the output from an obfuscated network layer is not used in a subsequent operation specified in a subsequent original layer, the compiler determines the sequence for executing the obfuscation operations in the obfuscated layer and the inference operations in the original layer as if the subsequent layer were waiting for the output from the preceding obfuscation layer. In this way, it is possible to prevent, or at least increase the "cost" of, using a side-channel attack to distinguish between critical layers and obfuscated network layers.

[0015] Examples of obfuscation operations may include linear or nonlinear operations of any suitable type. In situations where obfuscation operations are performed in parallel with specific inference operations of a critical layer, these obfuscation operations may mimic the actual inference operations. For example, obfuscation operations performed in parallel with node linear operations of a particular node may be linear operations such as addition, multiplication, and binary operations. As another example, obfuscation operations performed in parallel with node nonlinear operations of a particular node may be nonlinear operations such as activation functions, e.g., ReLU, sigmoid, Tanh, or other suitable nonlinear operations. As another example, obfuscation operations may include tensor reduction operations that mimic the action-weighted multiplication of a network layer. In situations where obfuscation operations are performed sequentially with inference operations of a critical layer, the obfuscation operations may be any suitable node or inter-layer operations, e.g., linear matrix operations, nonlinear node activation, pooling, or other suitable operations. These operations may be independent of the inference operations performed by the associated critical layer. Additional examples of obfuscation operations are described below.

[0016] Generally, a host or a compiler included in the host can compile a machine learning model (e.g., a neural network) and generate instructions that, when executed by a hardware device, cause the hardware device to perform at least an inference operation of the neural network. Compiling a neural network generally refers to converting program code in a high-level programming language (e.g., C++, Python, JAVA, or other programming language) that represents the neural network into a machine-readable low-level programming language (e.g., binary code). The compiled neural network can be deployed to one or more hardware devices to perform operations according to the corresponding instructions. During the compilation step, the described techniques can determine an obfuscated network structure associated with one or more critical layers of the neural network and generate instructions that, when executed by the hardware device, cause the hardware device to execute (i) an inference operation of the neural network's original inference operation and (ii) an obfuscated operation specified by the obfuscated network structure, in parallel and / or sequentially.

[0017] The hardware device may include one or more processing elements configured to process each assigned operation according to the instructions. Each of the processing elements may further include a plurality of computational units specifically arranged to perform the assigned operations. The assigned operations may include (i) machine learning operations, for example, in a manner that accelerates the performance of the machine learning operations, and / or (ii) obfuscation operations specified by an associated obfuscation network structure. Note that the described techniques are compatible with and independent of any type of hardware device suitable for performing machine learning operations (e.g., different types of accelerators, such as GPUs, TPUs, VPUs, FPGAs, or ASICs). This is because the described techniques are performed during a compilation step, and instructions for obfuscating parameters of a machine learning model may be determined differently for different hardware devices.

[0018] The described techniques may determine one or more characteristics of a neural network before compilation and, based on the one or more characteristics, determine whether to associate an obfuscated network structure with the neural network, and if so, where and how to associate the obfuscated network structure with the neural network. More specifically, the described techniques may determine whether to compile a neural network to run in a "standard mode" (no obfuscation operations are added to the instructions) or a "secure mode" (obfuscation operations are added). Compiling a neural network to run in secure mode may be referred to as performing compilation in secure mode. Similarly, compiling a neural network to run in standard mode may be referred to as performing compilation in standard mode.

[0019] In general, the system may determine to perform compilation in a manner that prepares the neural network to run in secure mode if the neural network is associated with a security-sensitive task, such as a facial recognition task. Alternatively, the system may determine to perform compilation in secure mode if the neural network requires significant time and cost for training. In some embodiments, the system may determine characteristics of the neural network by metadata associated with the neural network or by data included in requests from one or more applications that utilize the neural network. In some embodiments, the system may determine whether to compile the neural network in secure mode based on user input. For example, a developer of the neural network may specify that the neural network be compiled and run in secure mode.

[0020] After determining to compile the neural network in secure mode, the described techniques can determine one or more critical layers in a sequence of network layers in the neural network and determine an obfuscated network structure associated with the one or more critical layers. A critical layer may be a network layer that requires significant time and cost (e.g., computational cost) to train. Alternatively, a critical layer may be a layer with operations that significantly affect the performance of the neural network, such as a particular type of network layer in the neural network sequence, specific inter-layer connections between one layer and an adjacent layer, or a layer with specially designed node operations.

[0021] The obfuscation network structure may include obfuscation nodes included in the original critical layer, obfuscation layers added immediately before or after the critical layer, and / or nodes or inter-layer operations that are separate from the neural network but run in parallel with the inference operations of the critical layer. These obfuscation nodes and layers and corresponding obfuscation operations, when executed, do not affect the original inference operations of the neural network, for example, because the outputs of these obfuscation operations are not used by the original inference operations.

[0022] The subject matter described herein may be implemented in particular embodiments to achieve one or more of the following advantages: Compiling a neural network in a secure mode to obfuscate the parameters of the neural network deployed on a hardware device can prevent the neural network from being deciphered. More specifically, the obfuscation operation can cause changes in one or more measurable characteristics of the neural network executed by the hardware device, making it more difficult and requiring more time and resources to decipher the corresponding neural network parameters based on the measurable characteristics. Thus, the described techniques enhance data security by preventing the leakage of potentially sensitive machine learning data.

[0023] The subject matter described herein is further advantageous from the perspective of model compilation. For example, the described techniques are general and independent of the hardware device selected for deploying the compiled neural network. Instructions specifying inference and obfuscation operations are generated at hardware device compilation time, and these operations are scheduled and assigned to different computational units and / or processing elements before the instructions are sent to the hardware device. Thus, the described techniques do not require specially designed hardware devices, thereby enabling neural networks to be deployed on different types of hardware devices.

[0024] Furthermore, because the described techniques do not require modification of the neural network before compilation, users or engineers do not need to modify the structure of the neural network in a high-level programming language to add obfuscation structures / operations. The obfuscation network structure and corresponding obfuscation operations are automatically determined and compiled at compile time. In this manner, the described techniques significantly reduce research and development time required to update neural networks (e.g., determine obfuscation operations before compiling the neural network). This also prevents errors in neural networks that may occur during the development process due to the inclusion of associated obfuscation operations.

[0025] Other implementations of this and other aspects include corresponding systems, apparatus, and computer programs encoded on a computer storage device and configured to perform the actions of the methods. One or more computer systems may be so configured by software, firmware, hardware, or a combination thereof installed on the system that, during operation, causes the system to perform the actions. One or more computer programs may be so configured by having instructions that, when executed by a data processing device, cause the device to perform the actions.

[0026] The details of one or more embodiments of the subject matter described herein are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims.

[0027] Like reference symbols and designations in the various drawings refer to like elements. [Brief explanation of the drawings]

[0028] [Figure 1] FIG. 1 is a block diagram of an example system including an example secure machine learning model compiler. [Figure 2]1 illustrates an exemplary secure machine learning model compiler. [Figure 3] 1 illustrates an exemplary process of obfuscating a network structure associated with an input neural network. [Figure 4] 10 illustrates another exemplary process of obfuscating a network structure associated with an input neural network. [Figure 5] 10 illustrates another exemplary process of obfuscating a network structure associated with an input neural network. [Figure 6] 1 is an example flowchart of a process for compiling an input neural network for an obfuscated network structure. DETAILED DESCRIPTION OF THE INVENTION

[0029] The subject matter described herein relates to modifying a machine learning model such that, upon execution, obfuscation operations are performed that obfuscate measurable characteristics of the machine learning model. More specifically, the described technique relates to determining an obfuscation network structure associated with one or more specific layers of an input neural network (also referred to as an original neural network) before compiling the machine learning model, compiling the original neural network with the determined obfuscation network structure, and transmitting data including the compiled neural network with the obfuscation network structure to one or more hardware devices to perform operations on the neural network. The obfuscation network structure represents multiple obfuscation operations that, when executed by the corresponding hardware devices, can obfuscate one or more parameters of the original neural network. The obfuscation operations generally prevent, or at least increase the cost and / or time, for a malicious entity to decipher the network parameters based on a characteristic profile associated with the observed original neural network. The obfuscation network structure added to the machine learning model can be predetermined or pre-configured by a user or received as input to the system.

[0030] The term "hardware device," as used throughout the specification, generally refers to a hardware processor, e.g., a hardware accelerator, deployed in an edge device such as a smartphone, smart tablet, smart watch, or other suitable edge device. The hardware device may include one or more processing elements, each of which may include one or more computational units. The hardware device may perform operations in different processing elements and / or different computational units of each processing element as scheduled by instructions received from a host. Each computational unit of a hardware computing system is self-contained and may independently perform at least a portion of the computations required for a given layer of a multilayer neural network or at least a portion of the obfuscation operations specified by an obfuscation network structure.

[0031] One exemplary machine learning model may be a neural network trained to perform an inference task. The trained neural network includes multiple parameters that define the neural network and an inference operation within the neural network. The neural network parameters may include the number of network layers within the neural network, the number of nodes within the neural network, a node operation for each node, and / or a node weight for each node. The neural network computes inferences to process inputs by performing the neural network's inference operation. In particular, the neural network's layers each include multiple nodes, each having a respective node operation and weight. The node operations may include linear operations (e.g., multiplication and addition) or nonlinear operations (e.g., activation operations including ReLu activation functions, Tanh functions, and sigmoid functions). In some implementations, the parameters further include data determining node connections between adjacent network layers, e.g., fully connected layers, convolutional layers, or transposed convolutional layers.

[0032] The hardware device may perform inference operations on the neural network and / or obfuscation operations on the obfuscated network structure associated with the neural network based on instructions received from the host. A compiler on the host may schedule how these operations are distributed or assigned to portions of the computing units of the hardware device when compiling the neural network in "secure mode." In some embodiments, one or more hardware components in the edge device may include a dynamic scheduling mechanism that allows operations to be scheduled to the computing units at runtime.

[0033] An example of a computational unit is a tile, with each tile having a computational unit or processing engine and / or one or more caches and switches. An exemplary computational process performed on a neural network layer may include multiplying an input tensor with an input activation function and a parameter tensor with a weight. This computation involves multiplying the input activation function by the weight in one or more cycles and performing product accumulation over many cycles. The results of the network layer computation can be written to an output bus or stored in memory. Similarly, an obfuscation operation can mimic the inference operation described above, although the results of the obfuscation operation are generally not used by the inference operation. In some embodiments, the results of the obfuscation operation are discarded or stored in memory without being fetched in subsequent operations.

[0034] Upon performing an inference operation on the deployed neural network, the hardware device generates one or more measurable characteristics of the neural network. Various techniques, such as side-channel attacks, may be used to reverse engineer the parameters of the trained neural network. In some situations where the hardware device is included in an edge device, such as a smartphone, smartwatch, smart tablet, or other suitable edge device, the hardware device may be used to repeatedly perform test operations to determine the parameters of the neural network.

[0035] Maintaining confidentiality of neural network parameters is desirable for a variety of reasons. One example is from a security perspective. A neural network may be configured to perform a human face recognition task to conveniently unlock a device. If a third party were to successfully reverse engineer the neural network and figure out how to trick the face unlock mechanism, it would be insecure to use the face unlock mechanism. For example, a malicious actor could use an adversarial attack to trick the neural network into unlocking a device without the actor's authorization.

[0036] The described techniques can address the above security concerns by obfuscating the inference operations of a neural network. More specifically, when compiling a neural network in secure mode, the described techniques can issue instructions including obfuscation operations represented by an obfuscated network structure associated with the neural network. These instructions, when executed on a hardware device, can cause the hardware device to perform both the inference operation of the neural network and an obfuscation operation that disguises the original parameters of the neural network. These obfuscation operations can be performed in parallel and / or sequentially with the inference operation, depending on the type of obfuscated network structure associated with the neural network. In this manner, the hardware device can modify at least one of multiple measurable characteristics of the neural network to effectively conceal or mask the measurable characteristics of the actual machine learning operations performed by the hardware device, rendering the decryption process infeasible, inoperable, or ultimately more difficult by analyzing the masked or modified measurable characteristics. Details of performing the obfuscation operations are described below.

[0037] Figure 1 is a block diagram of an example system 100 including an example secure machine learning model compiler 105. As shown in Figure 1, the system 100 includes a host 108 and a hardware device 102 communicatively coupled to the host 108, e.g., via one or more networks. Generally, the host 108 can be communicatively coupled to multiple hardware devices, although for simplicity, only one hardware device 102 is shown in Figure 1.

[0038] The host 108 is configured to receive machine learning models, compile the received machine learning models using the secure machine learning model compiler 105, send instructions or data including the compiled machine learning models to the hardware device 102, and receive output data from the hardware device 102. The secure machine learning model compiler 105 is configured to compile one or more trained machine learning models (e.g., neural networks) from a high-level programming language (e.g., C++, Python, Java) into a machine-readable program (e.g., binary code). The binary code of the trained machine learning models generally includes at least all parameters that define the trained machine learning models. The binary code specifies, for example, the number of network layers in the compiled neural network, the type of each network layer, the number of nodes in each network layer, node operations for each node in each network layer, node weights determined for each node in each network layer, and inter-layer connectivity. The binary code may include any other parameters of the machine learning models.

[0039] The host 108 also generates instructions that, when executed by the hardware device 102, cause the hardware device 102 to perform the inference operation specified by the binary code. In some circumstances, the compiler 105 determines to include an obfuscation operation in the binary code that, when executed by the hardware device 102, causes one or more computing units or processing elements of the hardware device 102 to perform the obfuscation operation. Generally, an obfuscation operation is an operation that can hide or mask a measurable characteristic of a neural network. In other words, the obfuscation operation can include an operation that causes the hardware device 102 and / or an edge device that includes the hardware device 102 to generate or adjust one or more measurable characteristics. The obfuscation operation can include any suitable machine learning operation and / or other operation that generates or adjusts a measurable characteristic.

[0040] In some embodiments, the secure machine learning model compiler 105 is configured to determine whether to compile the machine learning model in standard mode (no obfuscation operations are added) or secure mode (obfuscation operations are added) based on characteristics of the machine learning model. Generally, if the secure machine learning model compiler 105 determines to compile the machine learning model in standard mode, the secure machine learning model compiler 105 does not include obfuscation operations in the instructions. On the other hand, if the secure machine learning model compiler 105 determines to compile the machine learning model in secure mode, the secure machine learning model compiler 105 determines one or more obfuscated network structures associated with one or more specific network structures of the machine learning model (e.g., one or more critical layers of a neural network) and generates instructions that schedule computational components on the hardware device 102 to perform the respective inference operations specified by the original machine learning model and the obfuscation operations specified by the obfuscated network structures. The components and operations of the secure machine learning model compiler 105 are described in more detail with reference to FIG. 2, and details of determining the obfuscated network structures are described with reference to FIGS. 3-5.

[0041] To determine whether to compile a machine learning model in secure mode, the secure machine learning model compiler 105 may analyze the nature or characteristics of the machine learning model. For example, the secure machine learning model compiler 105 may determine to perform the operation in secure mode if the machine learning model is a large, deep neural network that requires significant time and cost (e.g., computational cost) to train and the hardware device 102 is located within an edge device. In this example, the secure machine learning model compiler 105 may select the secure mode based on the size (e.g., by comparing the size of the model to a threshold) and / or the training time (e.g., by comparing the time required to train the model to a threshold, and such information may be included in metadata transmitted to the secure machine learning model compiler 105, for example). As another example, the secure machine learning model compiler 105 may determine to perform the operation of the machine learning model in secure mode if the machine learning model is used for security-sensitive applications, such as face unlock, voice unlock, signature verification, personal information access or prediction using a machine learning model, or other security-sensitive applications. In this example, the metadata associated with the machine learning model may include a label or metadata indicating whether the machine learning model is sensitive or which (e.g., secure or normal) mode to use to perform the machine learning model's operations. Thus, the secure machine learning model compiler 105 may determine whether to compile the machine learning model under secure mode or normal mode based on the metadata.

[0042] The hardware device 102 may be a hardware processor, e.g., a hardware accelerator such as a graphics processing unit (GPU), a vision processing unit (VPU), a tensor processing unit (TPU), or other suitable hardware accelerator. To accelerate the execution of inference operations, the hardware device 102 includes one or more processing elements 104A-N, each including one or more computational units 106A-N, also referred to as computational units 106 for brevity. Each computational unit 106 is a self-contained unit for executing an assigned inference operation (e.g., a linear or nonlinear operation for a node within or across a network layer). The number of processing elements 104A-N and corresponding computational units per processing element may vary based on different computational requirements. For example, the hardware device 102 may include 4, 8, 16, or more processing elements, each having 4, 8, 16, or more computational units. Additionally, different hardware devices 102 may have different arrangements and interconnections for processing elements 104A-N.

[0043] After receiving instructions and / or data from host 108, hardware device 102 may store the instructions received from host 108. The instructions may include data representing parameters of the assigned inference operations in the neural network in memory 110 (details of memory 110 are described below). The neural network parameters may include the number of nodes and network layers assigned to hardware device 102, node weights, and corresponding input activation functions from previous network layers. If the neural network is compiled under secure mode, the instructions may further include parameters specifying an obfuscation network structure, obfuscation operations associated with the obfuscation network structure, a method for associating the obfuscation network structure with the neural network, input data for performing the obfuscation operations, memory locations for storing computational results obtained from performing the obfuscation operations, or other parameters. For example, computational results from the inference operations and / or obfuscation operations specified by the obfuscation network structure may be stored in memory 110, and the results from the obfuscation operations may not be used in subsequent operations.

[0044] In some embodiments, the hardware device 102 may include a data bus configured to communicatively couple multiple computing units of a processing element in a sequence. The data bus may include different types of data buses for communicating respective instructions indicating different operations to be performed by different computing units (e.g., 106A-N in the processing element 104A), input data used for the operations in the different computing units, and results generated for the input data in the different computing units. For example, the data bus may include a ring bus originating from a controller (not shown) in the hardware device 102, providing communication coupling the computing units 106-106N sequentially in a ring shape and through a bus data path back to the controller. In some embodiments, the data bus may include a mesh bus, providing communication paths coupling or connecting each computing unit to its corresponding adjacent computing unit both horizontally and vertically. The mesh bus may be used to transfer input activations between one or more memory units in adjacent computing units.

[0045] Generally, instructions received at the hardware device 102 are broadcast to corresponding processing elements 104A-N to process the respective operations. The instructions generally specify a first portion of the computing units for performing the inference operation and a second portion of the computing units for performing the obfuscation operation. The instructions may further specify a memory unit for storing the results. For example, the instructions may indicate that the results produced by the second portion of the computing units for performing the obfuscation operation are to be discarded or saved (e.g., in a memory unit within the computing unit 106) and not accessed for further computation. The obfuscation operation may be performed in parallel or sequentially with the inference operation.

[0046] In some embodiments, instructions issued from host 108 may specify a processing element or computational unit of a processing element to be added to hardware device 102 or an edge device including hardware device 102 for dedicatedly performing obfuscation operations. The term “dedicatedly” generally refers to one or more processing elements or computational units incorporated into a hardware device in addition to (e.g., in addition to other elements or units in the hardware device for performing inference operations) and configured to perform substantially only obfuscation operations and not inference operations associated with a deployed machine learning model. A processing element or computational unit dedicated to performing obfuscation operations may include, for example, a processor, a multiplication unit, a multiplexer, a vector reduction unit, a logic gate, or other suitable processing element or computational unit.

[0047] The measurable characteristics include an electromagnetic profile of one or more inference operations, a time profile for performing one or more inference operations, or a power consumption profile of a computing unit performing the inference operations. In some implementations, the measurable characteristics may further include an audio profile and / or a temperature profile for performing one or more inference operations. The electromagnetic profile may represent, for example, a measure of electromagnetic emissions versus capacitor charge on a hardware device as the hardware device performs operations of a machine learning model. In some implementations, the characteristic profile may be represented by a graph having a horizontal axis representing time and a vertical axis representing a particular characteristic (e.g., electromagnetic emissions, power consumption, audio, or temperature).

[0048] An obfuscation network structure associated with a neural network may generally include an obfuscation node in a critical network layer, one or more obfuscation network layers immediately preceding or immediately following the critical network layer, or one or more parallel obfuscation network layers running in parallel with the critical network layer. The term “obfuscating a network layer immediately preceding a critical layer” generally refers to a situation in which an output from an obfuscation network layer is received as input to a critical layer, and the term “obfuscating a network layer immediately following a critical layer” generally refers to a situation in which the input to an obfuscation layer is an output from a critical layer. Obfuscation operations associated with the obfuscation node and the parallel obfuscation network are scheduled by the secure machine learning model compiler 105 to run in parallel with the inference operations of the associated critical network layer, and obfuscation operations associated with one or more obfuscation network layers immediately preceding or immediately following a critical network layer are scheduled to run sequentially with the inference operations specified by the critical network layer. For example, the obfuscation operations of an obfuscation layer preceding a critical layer are run before the critical layer, as if the critical layer received output from the preceding obfuscation layer. Similarly, the obfuscation operations of an obfuscation layer following a critical layer are performed after the critical layer, as if the output of the critical layer were received as input to the subsequent obfuscation layer.

[0049] In some implementations, obfuscation operations may mimic corresponding inference operations. For example, inference operations by nodes in a critical layer and obfuscation operations specified by obfuscation nodes in the critical layer may be associated with activation functions and executed in parallel. As another example, obfuscation operations in obfuscation layers before and after the critical layer may be similar to the operations specified by the critical layer. As a result, the combined data profile (e.g., power consumption profile) will be different from that for performing only inference operations, and the measurable characteristics will deviate from the true measurable characteristics of the neural network. Therefore, any reverse-engineered neural network parameters will differ from the true neural network parameters. For example, a reverse-engineered neural network may include obfuscation nodes and obfuscation network layers not included in the original neural network.

[0050] In some embodiments, an obfuscation operation can be unrelated (e.g., independent) from a corresponding inference operation, but performing the obfuscation operation can render any measurable data from the hardware device 102 meaningless. For example, if the inference operation is related to matrix reduction, the obfuscation operation can be a specific logical operation or a scalar addition or multiplication such that the true measurable profile is altered to lose its pattern or character, making it meaningless for determining the true parameters of the neural network. As another example, an obfuscation network structure can include a parallel obfuscation layer that includes obfuscation operations that run in parallel with the inference operations specified by one or more critical layers, such that the measurable data from the hardware device 102 becomes meaningless.

[0051] The computational results of the inference operation are stored in memory 110 and provided to host 108 for other inference operations that depend on the computational results. However, the computational results of the obfuscation operation are not used by any inference operation. In some embodiments, the obfuscation results are discarded without being written to any memory. Alternatively, the obfuscation results are written to a memory unit where other computational components of the inference operation do not receive any data stored in the memory unit. In other examples, the obfuscation results are written to the same memory as the inference operation, but are not accessed by the inference operation.

[0052] 2 illustrates an exemplary secure machine learning model compiler 200. The secure machine learning model compiler may be equivalent to or may be used to implement the secure machine learning model compiler 105 of FIG.

[0053] 2, the secure machine learning model compiler 105 is configured to process input data and generate output data by processing the input data. As described above, the input data may include one or more machine learning models (e.g., neural networks) encoded in a high-level programming language. The output data may include compiled machine learning models encoded in a machine-readable low-level programming language (e.g., binary code).

[0054] The secure machine learning model compiler 200 may include a security engine 210 configured to determine whether to compile a machine learning model in the input data under secure mode or standard mode, for example, based on characteristics of the machine learning model. The characteristics of the machine learning model may represent the time and cost to train the machine learning model. The time and cost to train a machine learning model may generally be related to the size of the machine learning model, the complexity of designing the structure of the machine learning model, the size of the training examples for training the machine learning model, the accuracy level set for training the machine learning model, the method by which the machine learning model is trained (individual training with additional machine learning models or end-to-end training), or other factors. For machine learning models that require high costs, the security engine 210 may determine to compile the machine learning model under secure mode. Otherwise, the machine learning model is compiled by the secure machine learning model compiler 200 under standard mode.

[0055] Alternatively or additionally, the characteristics of the machine learning model may indicate whether the machine learning model is security-sensitive. For example, as described above, a machine learning model trained for face unlock is security-sensitive. For security-sensitive machine learning models, the security engine 210 may determine to compile the machine learning models in secure mode. Otherwise, the machine learning models are compiled by the secure machine learning model compiler 200 in standard mode.

[0056] The characteristics of the machine learning model may be provided to the security engine 210 for analysis in different ways. For example, the characteristics of the machine learning model may be stored in metadata associated with the machine learning model included in the input data.

[0057] In some implementations, the secure machine learning model compiler 200 may receive instructions regarding whether to compile a machine learning model in secure mode. The instructions may be included in instruction data stored in memory, embedded in or associated with a program representing the machine learning model, or provided by a user via a user interface. For example, the instructions may include a flag value embedded in the program representing the machine learning model. The flag value may be interchangeably set by a user to cause the secure machine learning model compiler 200 to compile the corresponding machine learning model in standard mode or secure mode. The flag value may be a binary value, a logical value, a string, a real number, or other suitable value. For example, a flag value of “0” may cause the compiler 200 to compile the machine learning model in standard mode, and a flag value of “1” may cause the compiler 200 to compile in secure mode. The secure machine learning model compiler 200 further includes a modification engine 220 that determines an obfuscation structure associated with the machine learning model. For machine learning models that the security engine 210 determines to compile in standard mode, the secure machine learning model compiler 200 (or modification engine 220) does not determine an obfuscation network structure, and the machine learning model is compiled as is. For a machine learning model that security engine 210 determines to compile under secure mode, modification engine 220 determines one or more critical structures of the machine learning model (e.g., critical layers of a neural network) and corresponding obfuscation structures associated with the one or more critical structures.

[0058] For example, if the machine learning model is a neural network, the modification engine 220 determines critical layers in a sequence of network layers included in the neural network and determines different obfuscation network structures associated with the critical layers. The obfuscation network structures specify multiple obfuscation operations for modifying observed characteristics of the neural network, ultimately preventing the neural network parameters from being deciphered by unauthorized parties. For simplicity, the following description uses a "neural network" as an exemplary machine learning model for purposes of explanation. It should be noted that the machine learning model may include any other suitable model other than a neural network, and the described techniques may be applied to different machine learning models as appropriate.

[0059] The modification engine 220 can use various techniques to determine a critical layer within a neural network. For example, the modification engine 220 can determine whether a layer is a critical layer based on one or more criteria. The criteria can include, for example, (i) whether the layer within the neural network includes one or more node weights that were updated above a threshold during training, (ii) whether the number of node weights updated in the network layer exceeds a threshold, (iii) whether the number of node weights in the network layer that are dropped out or determined to be zero exceeds a threshold, or other suitable criteria. In response to determining that a layer satisfies one or more of the above criteria, the modification engine 220 can determine such a layer as a critical layer. As a simple example, the trained machine learning model can be a pre-trained neural network after fine-tuning, and the modification engine 220 can determine as a critical layer the layer with the largest change in node weights or a layer whose change in node weights meets a threshold. As another example, the modification engine 220 can determine whether a layer is a critical layer based on the input data and / or output data of the layer. The modification engine 200 may compare one or more characteristics (e.g., data type, data size, or other characteristics) of the input data and / or output data with a respective threshold or a set of predetermined characteristics. In some circumstances, the modification engine 200 may further compare a data read / write level associated with the input / output data of a network layer with a threshold. For example, the modification engine 200 may determine a layer as a critical layer if the data read / write level associated with the layer meets a threshold.

[0060] After determining the critical layer, the modification engine 220 may determine one or more obfuscation network structures for the critical layer. The modification engine 220 may determine the obfuscation network structures based on different obfuscation tasks. For example, the obfuscation network structure may include an obfuscation node having an obfuscation node operation added to the critical layer. As another example, the obfuscation network structure may include an obfuscation network layer immediately before and / or after the critical layer. Alternatively or additionally, the obfuscation network structure may include one or more parallel obfuscation network layers that specify obfuscation operations scheduled to run in parallel with the inference operations of the critical layer. Details of generating the obfuscation network structure are described in conjunction with Figures 3-5.

[0061] Scheduler 230 is configured to schedule the execution of both the inference operations and the obfuscation operations on the computing units of the hardware device. For example, scheduler 230 may assign one or more inference operations to a first portion of the computing units of the hardware device and assign one or more obfuscation operations to a second portion of the computing units of the hardware device. Scheduler 230 may further determine when to execute the obfuscation operations and the inference operations. For example, scheduler 230 may assign portions of the computing units to execute the inference operations and the obfuscation operations in parallel or sequentially. In situations where the hardware device includes dedicated computing units and / or processing elements for executing obfuscation operations, scheduler 230 may assign the obfuscation operations to the dedicated computing units and / or processing elements. In this manner, the hardware device may maximize utilization of the processing elements for inference operations.

[0062] 3 illustrates an example process 300 for obfuscating a network structure associated with an input neural network. The obfuscated network structure may be determined by a secure machine learning model compiler when the compiler compiles the input neural network. The secure machine learning model compiler may be equivalent to the secure machine learning model compiler 105 of FIG. 1 and / or the secure machine learning model compiler 200 of FIG. 2.

[0063] 3, a secure machine learning model compiler included in the described system can determine critical layers of an input neural network. The neural network can include multiple network layers (e.g., network layers 312, 314, 316) arranged in a sequence. The secure machine learning model compiler can determine whether to compile the neural network in a secure mode, and in response to determining to compile the neural network in a secure mode, the secure machine learning model compiler can determine critical layers from the sequence of network layers.

[0064] As described above, the secure machine learning model compiler can determine a critical layer based on one or more criteria, such as comparing the number of weights changed for a layer to a threshold number or comparing a measurement of value changes in the layer's node weights to a threshold. If a network layer meets one or more criteria, the secure machine learning model compiler determines the network layer as a critical layer. For example, as shown in FIG. 3 , the secure machine learning model compiler can determine a critical layer 312 in an input neural network. The neural network further includes one or more preceding network layers 314 that precede the critical layer in sequence and one or more succeeding network layers 316 that follow the critical layer in sequence. The critical layer 312 can include one or more nodes 306A-N, each having a node weight value and a corresponding node operation (e.g., the node activation function described above).

[0065] The obfuscation network structure in the example process 300 includes one or more obfuscation nodes 370A-N that are associated with (e.g., add obfuscation nodes to) the critical layer 312. The modified critical layer 352 represents both the original inference operation and the obfuscation operation specified by the added obfuscation nodes 370A-N.

[0066] When compiling the original neural network, the secure machine learning model compiler can incorporate one or more obfuscation nodes 370A-N into the critical layer in various ways. For example, the secure machine learning model compiler can add an obfuscation node 370A between two adjacent nodes (e.g., 306A and 306B). Alternatively or additionally, the secure machine learning model compiler can add two or more obfuscation nodes (e.g., 370C, 370D, 370E, and 370F) between two adjacent nodes (e.g., 306C and 306D). Each obfuscation node 370A, 370B, ..., or 370N represents an obfuscation node weight and an obfuscation node operation, and further specifies the corresponding obfuscation operation. For example, an obfuscation node can include an obfuscation node weight having a predetermined value, e.g., zero, one, or other value. The obfuscation node weight can be the same as one of the two adjacent original nodes, for example. The node operation of the obfuscated nodes 370A-N may include any suitable node operation, such as Tanh, Sigmoid, or other node operations as described above. The obfuscated node operation may be, for example, similar to one of two adjacent nodes.

[0067] The obfuscation operation may include an operation on values ​​associated with these obfuscation nodes 370A-N. For example, the obfuscation operation may include an appropriate operation (e.g., matrix multiplication, tensor reduction, pooling, or other appropriate operation) on one or more obfuscation node weights, one or more obfuscation node operations, or both. As noted above, input values ​​for the obfuscation operation may be predetermined and stored in one or more memory units within the hardware device. Output values ​​from performing the obfuscation operation are typically not used in subsequent inference operations. Rather, these output values ​​are discarded or stored in one or more memory addresses that are not accessed for inference operations.

[0068] Furthermore, obfuscation operations are specified by obfuscation nodes 370A-N within the critical layer, and therefore these obfuscation operations are sent to the hardware device in the form of data instructions to be scheduled by the secure machine learning model compiler and executed in parallel with the inference operations specified by the original nodes and the critical layer.

[0069] 4 illustrates another exemplary process 400 for obfuscating a network structure associated with an input neural network. The obfuscated network structure may be determined by a secure machine learning model compiler when the compiler compiles the input neural network. The secure machine learning model compiler may be equivalent to the secure machine learning model compiler 105 of FIG. 1 and / or the secure machine learning model compiler 200 of FIG. 2.

[0070] A secure machine learning model compiler included in the described system may determine a critical layer 412 of an input neural network to compile from a sequence of network layers (414, 412, and 416), as shown in Figure 4. One or more network layers 414 are preceding neural network layers that precede the critical layer 412 in the sequence, and one or more network layers 416 are succeeding network layers 316 that follow the critical layer 412 in the sequence. The critical layer 412 may include one or more nodes 406A-N, each having a node weight value and a corresponding node operation (e.g., a node activation function as described above).

[0071] The obfuscation network structure of the exemplary process 400 includes one or more obfuscation network layers 464 and / or 462. The secure machine learning model compiler is configured to modify the input neural network by adding one or more obfuscation network layers 464 and / or 462 before and / or after the critical layer 412 according to the sequence. For example, one or more obfuscation network layers 464 are added before the critical layer 412 and after the preceding network layer 414. Alternatively or additionally, another obfuscation network layer 462 is further added immediately after the critical layer 412 and before the subsequent network layer 416. It should be noted that the obfuscation layers may include any suitable number of obfuscation layers before and / or after the critical layer 412, for example, 1, 2, 3, 5, 10, or any other suitable number of obfuscation layers.

[0072] Each obfuscation network layer may include one or more obfuscation nodes. Each obfuscation node in an obfuscation network layer may include an obfuscation node weight value and an obfuscation node operation. Each obfuscation network layer may include one or more obfuscation operations associated with the obfuscation layer. For example, an obfuscation operation may include one or more inter-layer matrix operations (e.g., multiplying a layer output generated from a previous layer by an obfuscation node weight of the obfuscation layer). As another example, an obfuscation operation may include one or more pooling operations of layer outputs of a previous layer. In some implementations, the layer and node operations in an obfuscation layer may be similar to those specified by a critical layer. Alternatively, the obfuscation operation specified by an obfuscation layer may be of a different type than that of the critical layer. For example, an obfuscation operation may include a matrix multiplication while a critical layer includes a pooling or softmax operation, or vice versa.

[0073] As described above, the input values ​​of the obfuscation operation specified by the obfuscation layer may be predetermined and stored in one or more memory units within the hardware device. The output values ​​from performing the obfuscation operation specified by the obfuscation layer are typically not used in subsequent inference operations. Rather, these output values ​​are discarded or stored in one or more memory addresses that are not accessed for inference operations.

[0074] Furthermore, the obfuscation operations specified by the obfuscation layer are scheduled by the secure machine learning model compiler to be executed in accordance with the sequence of network layers in the modified neural network. Although the output of the obfuscated network layer is not typically used for inference operations specified by subsequent network layers (e.g., critical layers), the inference operations specified by the original network layer and the obfuscation operations specified by the obfuscation layer are still executed sequentially according to the layer sequence, as if the original network layer received the output from the obfuscation layer. This makes it more difficult for an unauthorized third party to determine which layers in the input neural network are obfuscated layers or actual layers based on the data profile observed from the hardware device.

[0075] 5 illustrates another exemplary process 500 for obfuscating a network structure associated with an input neural network. The obfuscated network structure may be determined by a secure machine learning model compiler when the compiler compiles the input neural network. The secure machine learning model compiler may be equivalent to the secure machine learning model compiler 105 of FIG. 1 and / or the secure machine learning model compiler 200 of FIG. 2.

[0076] 4, the obfuscation network structure of the exemplary process 500 also includes obfuscation network layers associated with critical layers. However, the secure machine learning model compiler does not use the obfuscation network layers to modify the structure (or parameters) of the input neural network. Instead, the secure machine learning model compiler determines one or more obfuscation layers that specify obfuscation operations to be performed in parallel with the inference operations specified by the critical layers. These obfuscation layers are also referred to as parallel obfuscation layers in the following description.

[0077] More specifically, when compiling an input neural network, the secure machine learning model compiler determines one or more parallel obfuscation layers 572 associated with a critical layer 512 in the sequence of network layers 512, 514, and 516 of the input neural network.

[0078] The one or more parallel obfuscation network layers 572 each include one or more obfuscation nodes 574. For example, a first parallel obfuscation network layer can include multiple obfuscation nodes 580A-N, and a second parallel obfuscation network layer can include multiple obfuscation nodes 585A-N. As described above, each obfuscation node in an obfuscation network layer can include an obfuscation node weight value and an obfuscation node operation. Each parallel obfuscation network layer can include one or more obfuscation operations associated with the parallel obfuscation layer. For example, an obfuscation operation can include one or more inter-layer matrix operations (e.g., multiplying a layer output generated from a previous layer by an obfuscation node weight of the obfuscation layer). As another example, an obfuscation operation can include one or more pooling operations of layer outputs of a previous layer. In some implementations, the layer and node operations in a parallel obfuscation layer can be similar to those specified by a critical layer. Alternatively, the obfuscation operations specified by a parallel obfuscation layer can be of a different type than those of the critical layer. For example, the obfuscation operation may include a pooling or softmax operation while the critical layer includes a matrix multiplication, or vice versa.

[0079] As described above, the input values ​​of the obfuscation operations specified by the parallel obfuscation layer may be predetermined and stored in one or more memory units within the hardware device. The output values ​​from performing the obfuscation operations specified by the parallel obfuscation layer are typically not used in subsequent inference operations. Rather, these output values ​​are discarded or stored in one or more memory addresses that are not accessed for inference operations.

[0080] Additionally, the obfuscation operations specified by the parallel obfuscation layers are scheduled by the secure machine learning model compiler to execute in parallel with the inference operations specified by the corresponding critical network layers, thereby altering the observed profile of one or more characteristics of the neural network, thereby further impeding, or at least increasing the time and cost, of deciphering one or more parameters of the neural network (e.g., parameters specifying one or more critical layers of the neural network).

[0081] It should be noted that while only one critical layer is shown in Figures 3-5 for simplicity of explanation, the secure machine learning model compiler may be configured to determine multiple critical layers within a neural network. Furthermore, while only one type of obfuscation network structure is shown in each of Figures 3-4, it will be appreciated that one or more obfuscation network structures may be combined in various ways for a critical network layer.

[0082] 6 is an exemplary flowchart of a process 600 for compiling an input neural network for an obfuscated network structure. For convenience, the process 600 is described as being performed by one or more computer systems located at one or more locations. For example, the process 600 can be performed on a host or by a compiler located on a host, such as the secure machine learning model compiler 105 shown in FIG. 1 or the secure machine learning model compiler 200 shown in FIG. 2. The order of steps in the process 600 is exemplary only and can be performed in a different order. In some implementations, the process 600 can include additional or fewer steps, or some of the steps can be divided into multiple steps.

[0083] The system receives 610 data representing a machine learning model. The machine learning model may include various types of machine learning models, such as a neural network. The data representing the neural network may specify multiple inference operations. The neural network may include parameters specifying a sequence of multiple network layers, multiple nodes in each of the multiple layers, node weights and node operations for each node, and other structures associated with the neural network. In some implementations, the data may further include metadata indicating additional information related to the machine learning model. For example, the metadata may indicate whether the machine learning model is security-sensitive and / or the time and / or cost to train the machine learning model. For simplicity, the following description is based on a neural network received by the system.

[0084] The system generally compiles a neural network to generate instructions that, when executed, cause one or more computing units of a hardware device to perform inference operations of the neural network and, in some circumstances, also perform obfuscation operations associated with the neural network.

[0085] The obfuscation operation, when performed, can obfuscate at least one or more measurable characteristics of the neural network. More specifically, the obfuscation operation can alter at least one or more measurable characteristics of the neural network. For example, as described above, the obfuscation operation, when performed, can be configured to obscure at least one of: a number of network layers of the neural network; a number of nodes in a network layer of the neural network; node operations of nodes in a network layer of the neural network; or weight values ​​associated with nodes in a network layer of the neural network.

[0086] The measurable characteristics may include at least one of a power profile, an electromagnetic profile, or a temporal profile. When the measurable characteristics are altered, it becomes more difficult to determine the parameters of the neural network based on the measurable characteristics. Obfuscation operations may include operations similar to inference operations, such as activation function operations, tensor multiplication, and reduction. For example, an obfuscation operation may include an obfuscation node operation specified by an obfuscation node in a network layer that is executed in parallel with other node operations in the network layer. The obfuscation node operation may be node addition or node multiplication. Alternatively, the obfuscation node operation of an obfuscation node may specify an activation function for a particular node that is different from the actual activation function of an actual node that is executed in parallel with the obfuscation node operation in the same network layer. In some implementations, the obfuscation operation may include operations unrelated to and / or different from the inference operation. An obfuscation operation may be executed in parallel with a corresponding inference operation or sequentially. Further details regarding obfuscation operations are provided above.

[0087] To compile the neural network, the system determines whether to obfuscate one or more measurable characteristics of the neural network (620). As described above, the system determines whether to obfuscate one or more measurable characteristics of the neural network based on the characteristics or properties of the neural network. Based on input data associated with the neural network, the system determines whether the neural network is security-sensitive and / or whether training required a significant amount of time and / or cost to meet a certain threshold. In response to determining that the neural network is security-sensitive and / or required a significant amount of resources, the system determines to compile the neural network in a “secure mode,” as described above. In the “secure mode,” the system can obfuscate one or more measurable characteristics of the neural network by determining one or more obfuscation operations to include in instructions in addition to the inference operations specified by the neural network when compiling the neural network, where the one or more obfuscation operations, when executed together with the inference operations, can obfuscate the one or more measurable characteristics of the neural network. In some implementations, step 620 is optional. For example, the system or host may receive user instructions or input data that instructs the host to compile the input neural network under "secure mode" without having to make a decision.

[0088] The system determines (630) a critical layer of the sequence of network layers. The critical layer is determined based on characteristics of the neural network. For example, the system may determine the critical layer based on the type of layer (e.g., pooling layer, fully connected layer, softmax layer). As another example, the system may determine the critical layer based on updates to parameters of the network layer during training. More specifically, the system may compare the parameter updates (e.g., the number of node updates in the network layer, changes in the values ​​of nodes within the network layer, the number of dropout nodes in the network layer, or changes in inter-layer connectivity between the network layer and adjacent layers) with one or more criteria (e.g., a threshold number or value). Alternatively, the system may receive metadata associated with the neural network indicating one or more critical layers. Determining critical layers is described in detail above.

[0089] The system determines (640) an obfuscation network structure associated with the critical layer. As described above, the system determines various obfuscation network structures associated with the critical layer. For example, the obfuscation network structure may include obfuscation nodes specifying respective obfuscation node weights and obfuscation node operations. The system may modify the neural network by adding one or more obfuscation nodes to the critical layer. The obfuscation operations specified by the obfuscation nodes added to the critical layer are generally executed in parallel with the inference operations specified by the original nodes in the critical layer. As another example, the obfuscation network structure may include one or more obfuscation network layers. The system may modify the neural network by inserting one or more obfuscation network layers immediately before and / or after the critical layer in the sequence of network layers. Each obfuscation network layer includes multiple obfuscation nodes as described above. The obfuscation operations specified by the obfuscation network layers are generally executed in order. For example, the order may be determined based on the sequence of modified network layers in the modified network. Alternatively or additionally, the obfuscation network structure includes one or more parallel obfuscation network layers. Unlike obfuscation network layers inserted into the original sequence of network layers of the neural network, the system does not modify the neural network using the parallel obfuscation layers. Instead, the system keeps the neural network structure unchanged and generates instructions to a hardware device to execute the obfuscation operations of the parallel obfuscation layers in parallel with one or more corresponding critical layers.

[0090] The system compiles the neural network using the associated obfuscation network structure (650). During compilation, the system generates instructions for executing the obfuscation operations specified by the obfuscation network structure and the inference operations specified by the compiled neural network. In some embodiments, the obfuscation operations are scheduled to run in parallel with the inference operations, e.g., the obfuscation operations specified by the obfuscation network structure, e.g., the obfuscation nodes and the parallel obfuscation network layers. Alternatively or additionally, the obfuscation operations and the inference operations are scheduled to run sequentially in order, e.g., the obfuscation operations specified by the obfuscation network structure, e.g., the obfuscation network layers.

[0091] In some embodiments, the system further determines a schedule in the instructions for one or more corresponding hardware devices. The schedule generally specifies a sequence for the hardware devices to perform the obfuscation operations specified by the corresponding obfuscation network structure and the inference operations specified by the neural network. For each hardware device of the one or more hardware devices, the schedule further indicates a different portion of a set of computing units of the hardware device for performing the respective inference and / or obfuscation operations.

[0092] In some implementations, the system may transmit data including the instructions to one or more hardware devices. Exemplary hardware devices may include edge devices such as a smartphone, a smart watch, a smart tablet, or a laptop. In some embodiments, the system may further generate data to trigger instructions to be executed by the one or more hardware devices, such that the inference operations specified by the neural network and the obfuscation operations specified by the obfuscation network structure are executed on the corresponding hardware devices (660).

[0093] This specification uses the term "configured" in the context of systems, devices, and computer program components. To say that one or more computer systems are configured to perform certain operations or actions means that the system has installed thereon software, firmware, hardware, or a combination thereof that causes the system to perform those operations or actions during operation. To say that one or more computer programs are configured to perform certain operations or actions means that the one or more programs contain instructions that, when executed by a data processing device, cause the device to perform those operations or actions. To say that special purpose logic circuitry is configured to perform certain operations or actions means that the circuitry has electronic logic that performs those operations or actions.

[0094] The term "data processing apparatus" refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. An apparatus may also be or further include special-purpose logic circuitry (e.g., an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit)). In addition to hardware, an apparatus may also optionally include code that creates an execution environment for a computer program, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0095] A computer program, which may be called or described as a program, software, software application, module, software module, script, or code, can be written in any form of programming language, including compiled or interpreted, declarative or procedural, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in part of a file that holds other programs or data, for example, one or more scripts stored in a markup language document, in a single file dedicated to the program, or in multiple cooperating files, for example, files that store one or more modules, subprograms, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communications network.

[0096] The processes and logic flows described herein may be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0097] A computer suitable for executing a computer program includes, by way of example, a general-purpose or special-purpose microprocessor, or both, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from a read-only memory, a random-access memory, or both. The basic elements of a computer are a central processing unit for implementing and executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes, or is operably coupled to, one or more mass storage devices for storing data, such as, for example, magnetic disks, magneto-optical disks, or optical disks, or to receive and / or transfer data therefrom. However, a computer need not have such devices. Furthermore, a computer can be embedded in other devices, such as a mobile phone, a smartphone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name a few.

[0098] Computer-readable media suitable for storing computer program instructions and data include all types of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks such as internal or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0099] Embodiments of the subject matter described herein can be implemented in a computing system that includes a back-end component, e.g., a data server, or a computing system that includes a middleware component, e.g., an application server, or a front-end component, e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an embodiment of the subject matter described herein, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communications network. Examples of communications networks include a local area network (LAN), a wide area network (WAN), e.g., the Internet.

[0100] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data, e.g., HyperText Markup Language (HTML) pages, to user devices, e.g., for the purpose of displaying the data to and receiving user input from users interacting with the user devices acting as clients. Data generated at the user devices (e.g., results of user interactions) can be received by the server from the user devices.

[0101] In addition to the above embodiments, the following embodiments are also innovative. Embodiment 1 is a method comprising receiving data representing a neural network including an inference operation, the neural network including parameters specifying a sequence of network layers and a plurality of nodes in each layer of the sequence of network layers, the method further comprising compiling the neural network to generate instructions that, when executed, cause one or more computing units of a hardware device to perform an obfuscation operation associated with the neural network and the inference operation of the neural network, the obfuscation operation, when executed, obfuscates one or more measurable characteristics of the neural network, the compiling comprising: determining a critical layer in the sequence of network layers; determining an obfuscation network structure to associate with the critical layer; and compiling the neural network with the associated obfuscation network structure to generate instructions for performing the obfuscation operation specified by the obfuscation network structure, the method further comprising causing the inference operation and the obfuscation operation to be executed on the hardware device.

[0102] Embodiment 2 is the method of embodiment 1, wherein the compiling further comprises determining to obfuscate the one or more measurable characteristics of the neural network.

[0103] Embodiment 3 is the method of embodiment 1 or 2, wherein causing the hardware device to perform the inference operation and the obfuscation operation includes causing the hardware device to perform the inference operation and the obfuscation operation in parallel.

[0104] Embodiment 4 is a method according to any one of embodiments 1 to 3, in which causing the hardware device to execute the inference operation and the obfuscation operation includes causing the hardware device to execute the inference operation and the obfuscation operation sequentially.

[0105] Embodiment 5 is a method according to any one of embodiments 1 to 4, wherein the obfuscation operation is configured to obscure at least one of the number of network layers of the neural network, the number of nodes in a network layer of the neural network, node operations of nodes in a network layer of the neural network, or weight values ​​associated with nodes in a network layer of the neural network.

[0106] Embodiment 6 is the method of any one of embodiments 1 to 5, wherein the one or more measurable characteristics of the neural network include at least one of a power profile, an electromagnetic profile, or a temporal profile.

[0107] Embodiment 7 is a method according to any one of embodiments 1 to 6, wherein determining a critical layer in the sequence of network layers includes determining the critical layer based on at least one of a type of network layer, an update of a parameter of the network layer, or metadata associated with the network layer.

[0108] Embodiment 8 is a method according to any one of embodiments 1 to 7, wherein determining the obfuscated network structure associated with the critical layer includes adding an obfuscated network layer immediately before and / or after the critical layer in the sequence of network layers.

[0109] Embodiment 9 is a method according to any one of embodiments 1 to 8, wherein determining the obfuscation network structure associated with the critical layer includes determining an obfuscation network layer with an obfuscation operation that is performed in parallel with the inference operation of the critical layer.

[0110] Embodiment 10 is a method according to any one of embodiments 1 to 9, wherein determining the obfuscation network structure associated with the critical layer includes adding an obfuscation node to a set of nodes of the critical layer, and the obfuscation node includes an obfuscation operation that is executed in parallel with the inference operation of the critical layer.

[0111] An eleventh embodiment is the method according to any one of the first to tenth embodiments, wherein the specific hardware device comprises an edge device.

[0112] Embodiment 12 is a method according to any one of embodiments 1 to 11, wherein compiling the neural network to generate instructions includes determining a schedule in the instructions for the particular hardware device, the schedule specifying a sequence for executing the obfuscation operation and the inference operation on each set of computing units of the particular hardware device.

[0113] Embodiment 13 is a system comprising one or more computers and one or more storage devices that store instructions, which, when executed by one or more computers, cause the one or more computers to perform respective operations, and the operations include a method described in any one of embodiments 1 to 12.

[0114] Embodiment 14 is one or more computer-readable storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform respective operations, each of which includes a method described in any one of embodiments 1 to 12.

[0115] While many specific implementation details are described herein, these should not be construed as limitations on the scope of what may be claimed, but rather as descriptions of features that may be unique to particular embodiments. Certain features described herein in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features of the invention that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, even if features may be described above as functioning in a particular combination and originally claimed as such, one or more features from a claimed combination may, in some cases, be deleted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0116] Similarly, while the figures may depict operations in a particular order, this should not be understood as requiring that such operations be performed in the particular order or sequential order depicted, or that all of the depicted operations be performed, to achieve desirable results. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems may generally be integrated into a single software product or packaged into multiple software products.

[0117] In each instance where an HTML file is mentioned, other file types or formats may be substituted. For example, an HTML file may be substituted with an XML, JSON, plain text, or other type of file. Additionally, where a table or hash table is mentioned, other data structures (such as a spreadsheet, relational database, or structured file) may be used.

[0118] Specific embodiments of the present invention have been described. Other embodiments are within the scope of the following claims. For example, the steps recited in the claims, described herein, or illustrated in the figures can be performed in a different order and still achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. 1. A method comprising: receiving data representing a neural network including an inference operation, the neural network including parameters specifying a sequence of network layers and a plurality of nodes in each layer of the sequence of network layers, the method further comprising: compiling the neural network to generate instructions that, when executed, cause one or more computing units of a hardware device to perform an obfuscation operation associated with the neural network and the inference operation of the neural network, the obfuscation operation, when executed, obfuscates one or more measurable characteristics of the neural network; determining a target layer in the sequence of network layers; determining an obfuscated network structure to associate with the target audience; and compiling the neural network together with the associated obfuscation network structure to generate instructions for performing the obfuscation operations specified by the obfuscation network structure, the method further comprising: causing the inference operation and the obfuscation operation to be performed in a hardware device.

2. The method of claim 1 , wherein the compiling further comprises determining to obfuscate the one or more measurable characteristics of the neural network.

3. 3. The method of claim 1, wherein causing the inference operation and the obfuscation operation to be performed on the hardware device comprises causing the hardware device to perform the inference operation and the obfuscation operation in parallel.

4. 4. The method of claim 1, wherein causing the inference operation and the obfuscation operation to be performed on the hardware device comprises causing the hardware device to sequentially perform the inference operation and the obfuscation operation.

5. 5. The method of claim 1, wherein the obfuscation operation is configured to obscure at least one of a number of network layers of the neural network, a number of nodes in a network layer of the neural network, node operations of nodes in a network layer of the neural network, or weight values ​​associated with nodes in a network layer of the neural network.

6. The method of any one of claims 1 to 5, wherein the one or more measurable characteristics of the neural network include at least one of a power profile, an electromagnetic profile, or a temporal profile.

7. 7. The method of claim 1, wherein determining a target layer in the sequence of network layers comprises determining the target layer based on at least one of a type of network layer, an update to a parameter of a network layer, or metadata associated with the network layer.

8. 8. The method of claim 1, wherein determining the obfuscated network structure associated with the target layer comprises adding an obfuscated network layer immediately before and / or after the target layer in the sequence of network layers.

9. 9. The method of claim 1, wherein determining the obfuscation network structure associated with the target layer comprises determining an obfuscation network layer with an obfuscation operation that is performed in parallel with the inference operation of the target layer.

10. 10. The method of claim 1, wherein determining the obfuscation network structure associated with the target layer comprises adding an obfuscation node to a set of nodes of the target layer, the obfuscation node comprising an obfuscation operation that is executed in parallel with the inference operation of the target layer.

11. The method of any one of claims 1 to 10, wherein the particular hardware device comprises an edge device.

12. Compiling the neural network to generate instructions includes:

12. The method of claim 1, further comprising determining a schedule in the instructions for the particular hardware device, the schedule specifying a sequence for performing the obfuscation operation and the inference operation on a set of respective computing units of the particular hardware device.

13. 13. A system comprising one or more computers and one or more storage devices storing instructions, the instructions, when executed by the one or more computers, causing the one or more computers to perform respective operations, the operations comprising a method according to any one of claims 1 to 12.

14. 13. One or more computer readable storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform respective operations, the respective operations comprising a method according to any one of claims 1 to 12.

Images