Modifying machine learning models to obfuscate their properties.

Abstract

Methods, systems, and apparatuses, including a computer program encoded on a computer storage medium, for obfuscating operations of a neural network. One method includes receiving data representing a neural network. The neural network includes parameters specifying a sequence of network layers and a plurality of nodes in each layer of the sequence of network layers. The neural network is compiled to generate instructions that, when executed, cause one or more computing units of a hardware device to perform obfuscation operations associated with an inference operation of the neural network. The obfuscation operations, when executed, obfuscate one or more measurable characteristics of the neural network. Compiling includes determining a target layer in the sequence of network layers, determining an obfuscation network structure to associate with the target layer, and compiling the neural network with the associated obfuscation network structure to generate instructions for performing the obfuscation operations specified by the obfuscation network structure.

Description

Technical Field

[0004] ,

[0001] This specification generally relates to machine learning. In particular, this specification describes techniques for modifying a machine learning model to include a structure that obfuscates the inference operations of the machine learning model when the machine learning model is executed.

Background Art

[0002] Artificial intelligence (AI) is the intelligence exhibited by machines and represents the ability of a computer program or machine to think and learn. Calculations can be performed using one or more computers to train a machine learning model for each task. Neural networks belong to a subfield of machine learning models.

[0003] A neural network can use one or more layers of nodes that represent multiple operations such as vector operations or matrix operations. One or more computers can be configured to perform the operations or calculations of a neural network to generate an output, such as the classification, prediction, or segmentation of received inputs. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from the received input according to the current values of each set of network parameters.

[0004] A specially designed hardware accelerator can execute specific functions and operations, including the operations or calculations specified within a neural network, faster and more efficiently compared to operations executed by a general-purpose central processing unit (CPU). The hardware accelerator can include a graphics processing unit (GPU), a tensor processing unit (TPU), a video processing unit (VPU), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). [Overview of the project]

[0005] A machine learning model (e.g., a neural network), after being properly trained, can be compiled and deployed on a hardware device configured to perform inference operations to process input data. These inference operations are defined by the neural network's parameters, which are updated during the training process. These parameters define (i) the node operations (e.g., linear and nonlinear operations) of the nodes within each network layer of the neural network, and (ii) the structure of the neural network. For example, parameters defining node operations include parameters defining the activation function for each network layer and parameters defining the node weights of the nodes within the network layer. As another example, parameters defining structure (also called hyperparameters) include at least one of the following: the number of nodes in a network layer, the number of network layers within a machine learning model (e.g., a neural network), or node connections across adjacent layers (e.g., fully connected layers, convolutional layers, or transposed convolutional layers). For simplicity, the following specification describes neural networks, but can be applied to other types of machine learning models.

[0006] It is extremely important to keep the parameters of trained neural networks confidential. Firstly, training a neural network, especially a deep neural network with sufficient accuracy to generate predictions, requires considerable computational cost and time. Furthermore, some neural networks have applications related to security sensitive authentication, such as face unlocking tasks where the neural network is configured to recognize faces to conveniently unlock devices. Therefore, it is extremely important to keep the structure and parameters of the neural network undecipherable, or at least difficult to decipher, to prevent malicious actors from learning the parameters and using them to unlock unauthenticated devices.

[0007] However, especially when a neural network is implemented on a hardware device accessible by a third party (e.g., an edge device such as a smartphone, smartwatch, or smart tablet, or other edge device), various techniques can be applied to "decode" the trained neural network. For example, one technique might measure the characteristics of a trained neural network when the hardware device performs inference operations on the trained neural network. More specifically, the technique might determine the parameters and structure of the neural network by collecting data, such as power consumption, electromagnetic waves, or time, when the hardware device performs inference operations, and analyzing the characteristic profile generated based on the collected data. This technique is also known as a side-channel attack.

[0008] The techniques described herein can enhance the security of neural networks implemented in hardware devices such as edge hardware devices. For example, the techniques described protect against side-channel attacks by determining, at compile time, one or more obfuscated network structures associated with one or more original network layers in the neural network, and, when executed by the hardware device, by generating instructions that cause the hardware device to perform obfuscated operations specified by the obfuscated network structures in parallel with and / or sequentially with the inference operations of one or more original network layers.

[0009] One or more original network layers generally refer to the “target layer” of a neural network. In some situations, a system may determine one or more critical layers of a neural network as the target layer for obfuscating the operations of the target layer. The target layer (e.g., the critical layer) typically requires considerable time and cost (e.g., the cost of computational resources) to train. Alternatively or additionally, the target layer (e.g., the critical layer) may generally include network layers that are substantially important to the network, e.g., layers that are important for improving the performance of the neural network. Performance may include hardware resource requirements, time requirements, power requirements, or other requirements for performing the inference operations of the neural network for various tasks. A system (or compiler) performing the techniques described may determine the target layer by determining whether a network layer is a critical layer based on one or more characteristics associated with the neural network. Exemplary characteristics may include the type of layer, the size of the layer, the inputs and / or outputs of the layer, or other appropriate characteristics. In some embodiments, the system may determine that a layer is a critical layer if one or more criteria, such as threshold memory bandwidth, threshold power consumption, or other criteria, are met by that layer. Thus, the techniques described can prevent side-channel attacks or at least raise the computational and / or time cost hurdle for decrypting a neural network deployed using a side-channel attack. While these techniques are primarily described in relation to target layers (e.g., critical layers), obfuscation layers can be added to or near other non-critical layers. Furthermore, it should be noted that, for simplicity, the term “critical layer” is used throughout this specification, but a “critical layer” may be equivalent to, or determined to be, a target layer if the operations in the target layer are obfuscated by introducing obfuscation operations performed on a hardware device.

[0010] As used throughout this specification, the term “obfuscation” generally refers to an operation that, when performed with machine learning operations (e.g., inference operations) on a neural network deployed by a hardware device, causes a change in one or more measurable properties of the neural network such that at least one parameter of the neural network is obfuscated, for example, the number of network layers in the neural network, the number of nodes in the network layers, the node operations of the nodes in the network layers, or the weights associated with the nodes in the network layers. Note that different types of machine learning models have different types of parameters that define the model. The techniques described in this document can obfuscate any type of parameter that affects the measurable properties of a machine learning model.

[0011] One or more measurable properties of a neural network generally refer to data that can be measured when a hardware device performs inference operations on the neural network. Measurable data may include data or profiles relating to power consumption, time, electromagnetic radiation, or other measurable data, as described above.

[0012] It should also be noted that obfuscation operations may be performed sequentially and / or in parallel with machine learning operations, depending on the determined obfuscation network structure. As used throughout this specification, the term “in parallel” generally refers to a common period during which both obfuscation and inference operations are performed by the hardware device. For example, the common period can be exactly the same period, substantially the same period (e.g., within each other’s threshold periods), or two different periods with overlapping regions. As used throughout this specification, the term “sequentially” generally refers to the obfuscation and inference operations being performed in a sequence over different periods. For example, an obfuscation operation may be performed before or after one or more inference operations have been performed. Different periods generally refer to periods without overlapping regions.

[0013] In situations where obfuscation operations are performed in parallel with inference operations, an obfuscated network structure may include, for example, one or more obfuscated nodes added to one or more critical layers. The obfuscation operations specified by the obfuscated nodes in the critical layers may be performed in parallel with the inference operations of the original nodes in the critical layers. As another example, an obfuscated network structure may be added to the instructions when the neural network is compiled. However, these obfuscated network structures do not modify the original structure or parameters of the neural network. Rather, the obfuscated operations specified by these obfuscated network structures are performed in parallel by the hardware device, along with the inference operations in one or more critical layers. An obfuscated network structure may mimic operations performed by critical layers with similar data flows and / or data operations.

[0014] In situations where obfuscation operations are performed sequentially with inference operations, the obfuscated network structure may include, for example, one or more obfuscated network layers added immediately before or after a critical layer. Even if the output from an obfuscated network layer is not used for a subsequent operation specified in a subsequent original layer, the compiler determines the sequence for performing obfuscation operations in the obfuscated layer and inference operations in the original layer as if the subsequent layer were waiting for output from the preceding obfuscated layer. In this way, it prevents, or at least raises the "cost" of, distinguishing between the critical layer and the obfuscated network layer using a side-channel attack.

[0015] Examples of obfuscated operations may include appropriate types of linear or nonlinear operations. In situations where obfuscated operations are performed concurrently with specific inference operations in critical layers, these obfuscated operations may mimic actual inference operations. For example, obfuscated operations performed concurrently with node-linear operations at a particular node may be linear operations such as addition, multiplication, and binary operations. Another example is obfuscated operations performed concurrently with node-nonlinear operations at a particular node, which may be nonlinear operations such as activation functions, e.g., ReLU, sigmoid, Tanh, or other appropriate nonlinear operations. Another example is that obfuscated operations may include tensor reduction operations that mimic action-weighted multiplication in network layers. In situations where obfuscated operations are performed sequentially with inference operations in critical layers, the obfuscated operations may be any appropriate node-operation or inter-layer operation, e.g., linear matrix operations, nonlinear node activation, pooling, or other appropriate operations. These operations can be independent of the inference operations performed by the associated critical layers. Additional examples of obfuscated operations are described below.

[0016] Generally, a host or a compiler contained within a host can compile a machine learning model (e.g., a neural network) and, when executed by a hardware device, generate instructions that cause the hardware device to perform at least the inference operations of the neural network. Compiling a neural network generally refers to translating the program code representing the neural network in a high-level programming language (e.g., C++, Python, Java®, or other programming languages) into a machine-readable low-level programming language (e.g., binary code). The compiled neural network can be deployed to one or more hardware devices to perform operations according to the corresponding instructions. During the compilation step, the techniques described can determine an obfuscated network structure associated with one or more critical layers of the neural network and, when executed by a hardware device, generate instructions that cause the hardware device to perform (i) the inference operations of the original inference operations of the neural network, and (ii) the obfuscated operations specified by the obfuscated network structure, in parallel and / or sequentially.

[0017] A hardware device may include one or more processing elements configured to perform operations assigned to them according to instructions. Each processing element may further include multiple computing units specifically positioned to perform the assigned operations. The assigned operations may include (i) machine learning operations in a manner that accelerates the performance of machine learning operations, and / or (ii) obfuscation operations specified by the associated obfuscation network structure. It should be noted that the techniques described are compatible with and independent of any type of hardware device suitable for performing machine learning operations (e.g., different types of accelerators such as GPUs, TPUs, VPUs, FPGAs, or ASICs). This is because the techniques described are executed during the compilation step, and the instructions for obfuscation of the parameters of the machine learning model may be determined differently depending on the different hardware device.

[0018] The techniques described can determine one or more properties of a neural network before compilation, and based on those properties, determine whether to associate an obfuscated network structure with the neural network, and if so, where and how to associate the obfuscated network structure within the neural network. More specifically, the techniques described determine whether to compile the neural network to run in "standard mode" (no obfuscation operations are added to the instructions) or "secure mode" (obfuscation operations are added). Compiling a neural network so that it runs in secure mode can be called performing a compilation in secure mode. Similarly, compiling a neural network so that it runs in standard mode can be called performing a compilation in standard mode.

[0019] Generally, a system may decide to compile a neural network in a way that prepares it to run in secure mode if the neural network is related to a security-sensitive task, such as a facial recognition task. Alternatively, if the neural network requires significant time and cost for training, the system may decide to compile it in secure mode. In some embodiments, the system may determine the properties of the neural network by metadata associated with it or by data contained in requests from one or more applications that utilize the neural network. In some embodiments, the system may decide whether to compile the neural network in secure mode based on user input. For example, a developer of a neural network may specify that the neural network be compiled and run in secure mode.

[0020] After deciding to compile the neural network in secure mode, the techniques described can determine one or more critical layers in the sequence of network layers within the neural network and the obfuscated network structures associated with one or more critical layers. A critical layer may be a network layer that requires significant time and cost (e.g., computational cost) to train. Alternatively, a critical layer may be a layer with operations that significantly impact the performance of the neural network, such as a particular type of network layer in the sequence of neural networks, a particular interlayer connection between a layer and an adjacent layer, or a layer with specially designed node operations.

[0021] The obfuscated network structure may include obfuscated nodes included in the original critical layer, obfuscated layers added immediately before or after the critical layer, and / or nodes or inter-layer operations that are separated from the neural network but run in parallel with the inference operations of the critical layer. These obfuscated nodes and layers, as well as the corresponding obfuscated operations, do not affect the original inference operations of the neural network, for example, because the outputs of these obfuscated operations are not used by the original inference operations.

[0022] The subject matter described herein may be implemented in specific embodiments to achieve one or more of the following advantages: By compiling a neural network in secure mode to obfuscate the parameters of the neural network deployed on a hardware device, it is possible to prevent the neural network from being deciphered. More specifically, the obfuscation operation can cause a change in one or more measurable properties of the neural network performed by the hardware device, making it difficult to decipher the corresponding neural network parameters based on the measurable properties, requiring more time and resources. Thus, the techniques described enhance data security by preventing the leakage of potentially sensitive machine learning data.

[0023] The subject matter described herein is even more advantageous from the standpoint of model compilation. For example, the techniques described are general and independent of the hardware device selected for deploying the compiled neural network. Instructions specifying inference and obfuscation operations are generated during hardware device compilation, and these operations are scheduled and assigned to different computing units and / or processing elements before the instructions are sent to the hardware device. Thus, the techniques described do not require specially designed hardware devices, making it possible to deploy neural networks on different types of hardware devices.

[0024] Furthermore, since the techniques described do not require modification of the neural network prior to compilation, the user or engineer does not need to modify the structure of the neural network in a high-level programming language to add obfuscated structures / operations. The obfuscated network structure and corresponding obfuscated operations are automatically determined and compiled at compile time. Thus, the techniques described significantly save research and development time for updating the neural network (e.g., determining obfuscated operations before compiling the neural network). This also prevents errors in the neural network that may occur in the development process to include the associated obfuscated operations.

[0025] Other embodiments of this and other aspects include corresponding systems, devices, and computer programs configured to perform the actions of the methods, encoded on a computer storage device. A system of one or more computers can be configured in such a manner by software, firmware, hardware, or any combination thereof, installed on the system, that in operation causes the system to perform the actions. One or more computer programs can be configured in such a manner by having instructions that, when executed by a data processing apparatus, cause the apparatus to perform the actions.

[0026] Details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

[0027] Like reference symbols and designations in the various drawings refer to like elements.

Brief Description of the Drawings

[0028] [Figure 1] FIG. is a block diagram of an exemplary system including an exemplary secure machine learning model compiler. [Figure 2]Illustrates an exemplary secure machine learning model compiler. [Figure 3] Illustrates an exemplary process of an obfuscation network structure associated with an input neural network. [Figure 4] Illustrates another exemplary process of an obfuscation network structure associated with an input neural network. [Figure 5] Illustrates another exemplary process of an obfuscation network structure associated with an input neural network. [Figure 6] An exemplary flowchart of a process for compiling an input neural network of an obfuscation network structure.

Best Mode for Carrying Out the Invention

[0029] The subject matter described herein relates to modifying machine learning models so that, when the machine learning model is run, an obfuscation operation is performed that obfuscates the measurable properties of the machine learning model. More specifically, the techniques described relate to determining an obfuscation network structure to be associated with one or more specific layers of an input neural network (also called the original neural network) before compiling the machine learning model; compiling the original neural network with the determined obfuscation network structure; and sending data containing the compiled neural network with the obfuscation network structure to one or more hardware devices for the execution of the neural network's operations. An obfuscation network structure represents a set of obfuscation operations that, when executed by the corresponding hardware device, can obfuscate one or more parameters of the original neural network. Obfuscation operations generally prevent a malicious entity from deciphering the network parameters based on the property profile associated with the observed original neural network, or at least increase the cost and / or time it takes. The obfuscation network structures added to the machine learning model may be predetermined or pre-configured by the user, or received as input to the system.

[0030] The term “hardware device” as used throughout this specification generally refers to a hardware processor, such as a hardware accelerator, deployed in an edge device such as a smartphone, smart tablet, smartwatch, or other suitable edge device. A hardware device may include one or more processing elements, each processing element may include one or more computing units. A hardware device may perform operations in different processing elements and / or different computing unit portions of each processing element, as scheduled by instructions received from a host. Each computing unit of a hardware computing system is autonomous and may independently perform at least a portion of the computations required for a given layer of a multilayer neural network, or at least a portion of the obfuscated operations specified by an obfuscated network structure.

[0031] One exemplary machine learning model may be a neural network trained to perform an inference task. A trained neural network includes several parameters that define the neural network and the inference operations within the neural network. The parameters of the neural network may include the number of network layers in the neural network, the number of nodes in the neural network, the node operations for each node, and / or the node weights for each node. The neural network computes inference for processing input by performing the neural network's inference operations. In particular, each layer of the neural network contains several nodes, each having its own node operations and weights. The node operations may include linear operations (e.g., multiplication and addition) or nonlinear operations (e.g., activation operations including ReLU activation functions, Tanh functions, and sigmoid functions). In some embodiments, the parameters further include data that determines the node connections between adjacent network layers, e.g., fully connected layers, convolutional layers, or transposed convolutional layers.

[0032] A hardware device may perform inference operations on a neural network and / or obfuscation operations on an obfuscated network structure associated with the neural network, based on instructions received from the host. When the compiler on the host compiles the neural network in "secure mode," it may schedule how and to which parts of the hardware device's computing units these operations are distributed or allocated. In some embodiments, one or more hardware components within the edge device may include a dynamic scheduling mechanism that can schedule operations to computing units at runtime.

[0033] An example of a computing unit is a tile, each tile having a computing unit or processing engine, and / or one or more caches and switches. An exemplary computing process performed on a neural network layer may involve multiplying an input tensor having an input activation function by a parameter tensor having weights. This computation involves multiplying the input activation function by the weights in one or more cycles and performing the accumulation of the product over many cycles. The computation results of the network layer can be written to an output bus and stored in memory. Similarly, obfuscation operations can mimic the inference operations described above, however, the results of obfuscation operations are generally not used by inference operations. In some embodiments, the results of obfuscation operations are discarded without being fetched in subsequent operations or stored in memory.

[0034] When an inference operation is performed on a deployed neural network, the hardware device generates one or more measurable properties of the neural network. Various techniques, such as side-channel attacks, can be used to reverse engineer the parameters of the trained neural network. In some situations where the hardware device is included in an edge device, such as a smartphone, smartwatch, smart tablet, or other suitable edge device, it may be possible to use the hardware device to repeatedly perform test operations and determine the parameters of the neural network.

[0035] For various reasons, it is desirable to maintain the confidentiality of neural network parameters. One example is security. A neural network may be configured to perform a human face recognition task to conveniently unlock a device. If a third party succeeds in reverse engineering the neural network and devising a way to circumvent the face unlock mechanism, then using the face unlock mechanism becomes insecure. For example, a malicious actor may use an adversarial attack to circumvent the neural network and unlock an unauthenticated device.

[0036] The described technique can address the aforementioned security concerns by obfuscating the inference operations of a neural network. More specifically, the described technique allows issuing instructions containing obfuscated operations, represented by an obfuscated network structure associated with the neural network, when compiling the neural network in secure mode. When executed on a hardware device, these instructions can cause the hardware device to perform both the inference operations of the neural network and obfuscated operations that disguise the original parameters of the neural network. These obfuscated operations may be performed in parallel with and / or sequentially with the inference operations, depending on the type of obfuscated network structure associated with the neural network. In this way, the hardware device can modify at least one of several measurable properties of the neural network to efficiently conceal or mask the measurable properties of the actual machine learning operations performed by the hardware device, making the decryption process impossible, unexecutable, or ultimately more difficult by analyzing the masked or modified measurable property. Details on performing the obfuscated operations are described below.

[0037] Figure 1 is a block diagram of an exemplary system 100, which includes an exemplary secure machine learning model compiler 105. As shown in Figure 1, the system 100 includes a host 108 and a hardware device 102 that is communicatively coupled to the host 108, for example, via one or more networks. Generally, the host 108 can be communicatively coupled to multiple hardware devices, but for simplicity, only one hardware device 102 is shown in Figure 1.

[0038] The host 108 is configured to receive a machine learning model, compile the received machine learning model using a secure machine learning model compiler 105, send instructions or data containing the compiled machine learning model to a hardware device 102, and receive output data from the hardware device 102. The secure machine learning model compiler 105 is configured to compile one or more trained machine learning models (e.g., neural networks) from a high-level programming language (e.g., C++, Python, Java®) into a machine-readable program (e.g., binary code). The binary code of a trained machine learning model generally includes at least all parameters that define the trained machine learning model. The binary code specifies, for example, the number of network layers in the compiled neural network, the type of each network layer, the number of nodes in each network layer, the node operations per node in each network layer, the node weights determined for each node in each network layer, and the inter-layer connectivity. The binary code may include any other parameters of the machine learning model.

[0039] Host 108 also generates instructions that, when executed by hardware device 102, cause hardware device 102 to perform an inference operation specified by binary code. In some situations, compiler 105 decides that binary code that, when executed by hardware device 102, causes one or more computing units or processing elements of hardware device 102 to perform an obfuscation operation includes an obfuscation operation. Generally, an obfuscation operation is an operation that can hide or mask the measurable characteristics of a neural network. In other words, an obfuscation operation may include an operation that causes hardware device 102 and / or edge devices containing hardware device 102 to generate or modify one or more measurable characteristics. An obfuscation operation may include any appropriate machine learning operation and / or other operation that generates or modifies measurable characteristics.

[0040] In some embodiments, the secure machine learning model compiler 105 is configured to determine, based on the characteristics of the machine learning model, whether to compile the machine learning model in standard mode (without obfuscation operations) or secure mode (with obfuscation operations). Generally, if the secure machine learning model compiler 105 decides to compile the machine learning model in standard mode, it does not include obfuscation operations in the instructions. On the other hand, if the secure machine learning model compiler 105 decides to compile the machine learning model in secure mode, it determines one or more obfuscated network structures associated with one or more specific network structures of the machine learning model (e.g., one or more critical layers of a neural network), and generates instructions that schedule computational components on the hardware device 102 to perform the respective inference operations specified by the original machine learning model and the obfuscation operations specified by the obfuscated network structures. The components and operations of the secure machine learning model compiler 105 are described in more detail with reference to Figure 2, and the details of determining the obfuscated network structures are described with reference to Figures 3 to 5.

[0041] To determine whether to compile a machine learning model in secure mode, the secure machine learning model compiler 105 may analyze the nature or characteristics of the machine learning model. For example, if the machine learning model is a large deep neural network that requires considerable time and cost (e.g., computational cost) to train, and the hardware device 102 is located within an edge device, the secure machine learning model compiler 105 may decide to perform the computation under secure mode. In this example, the secure machine learning model compiler 105 may select the secure mode based on size (e.g., by comparing the size of the model to a threshold) and / or training time (e.g., by comparing the time required to train the model to a threshold, and such information may be included in metadata sent to the secure machine learning model compiler 105, for example). As another example, the secure machine learning model compiler 105 may decide to perform the computation of the machine learning model under secure mode if the machine learning model is used for security sensitive applications, such as face unlocking, voice unlocking, signature verification, accessing or predicting personal information using the machine learning model, or other security sensitive applications. In this example, metadata associated with a machine learning model may include labels or metadata indicating whether the machine learning model is sensitive, or labels or metadata indicating which mode (e.g., secure or normal) is used to perform calculations on the machine learning model. Thus, a secure machine learning model compiler 105 can determine, based on the metadata, whether to compile the machine learning model under secure mode or standard mode.

[0042] The hardware device 102 may be a hardware processor, such as a hardware accelerator like a graphics processing unit (GPU), vision processing unit (VPU), or tensor processing unit (TPU), or other suitable hardware accelerator. To accelerate the execution of inference operations, the hardware device 102 includes one or more processing elements 104A-N, each processing element 104A-N including one or more compute units 106A-N, also referred to as compute units 106 for brevity. Each compute unit 106 is an autonomous unit for executing its assigned inference operation (e.g., linear or nonlinear operations on nodes within or across the network layer). The number of processing elements 104A-N and the number of compute units corresponding to each processing element may vary based on different computational requirements. For example, the hardware device 102 may include 4, 8, 16, or more processing elements, each having 4, 8, 16, or more compute units. Furthermore, different hardware devices 102 can have different arrangements and interconnections with respect to processing elements 104A to N.

[0043] After receiving instructions and / or data from host 108, hardware device 102 may store the instructions received from host 108. These instructions may include data representing the parameters of the assigned inference operations within the neural network in memory 110 (details of memory 110 will be described later). The neural network parameters may include the number of nodes and network layers assigned to hardware device 102, the node weights, and the corresponding input activation functions from previous network layers. If the neural network is compiled in secure mode, the instructions may further include parameters specifying the obfuscated network structure, the obfuscation operations associated with the obfuscated network structure, a method for associating the obfuscated network structure with the neural network, input data for performing the obfuscation operations, a memory location for storing the computation results obtained by performing the obfuscation operations, or other parameters. For example, the computation results from the obfuscation operations specified by the inference operations and / or obfuscated network structure may be stored in memory 110, and the results from the obfuscation operations are not used in subsequent operations.

[0044] In some embodiments, the hardware device 102 may include a data bus configured to connect multiple computing units of a processing element in a sequential manner for communication. The data bus may include different types of data buses for communicating instructions indicating different operations to be performed in different computing units (e.g., 106A-N in processing element 104A), input data used for operations in different computing units, and results for the input data generated in different computing units. For example, the data bus may include a ring bus starting from a controller (not shown) within the hardware device 102, providing communication that connects computing units 106-106N sequentially in a ring via a bus data path that returns to the controller. In some embodiments, the data bus may include a mesh bus, providing a communication path that connects or links each computing unit to its corresponding adjacent computing unit both laterally and vertically. The mesh bus can be used to transfer input activation amounts between one or more memory units in adjacent computing units.

[0045] Generally, instructions received by hardware device 102 are broadcast to corresponding processing elements 104A to N to process their respective operations. An instruction typically specifies a first computation unit portion for performing inference operations and a second computation unit portion for performing obfuscation operations. The instruction may further specify a memory unit for storing the results. For example, the instruction may indicate that the results generated by the second computation unit portion for performing obfuscation operations are to be discarded or stored (e.g., in a memory unit within computation unit 106) and not accessed for further computation. Obfuscation operations can be performed in parallel with or sequentially with inference operations.

[0046] In some embodiments, instructions issued from host 108 may specify a processing element or compute unit of a processing element to be added to hardware device 102, or to an edge device including hardware device 102 for dedicatedly performing obfuscation operations. The term “dedicated” generally refers to one or more processing elements or compute units that are incorporated additionally into a hardware device (for example, in addition to other elements or units in the hardware device for performing inference operations) and are configured to substantially perform only obfuscation operations and not to perform inference operations associated with the deployed machine learning model. A processing element or compute unit dedicated to performing obfuscation operations may include, for example, a processor, a multiplication unit, a multiplexer, a vector reduction unit, a logic gate, or other suitable processing element or compute unit.

[0047] Measurable characteristics include electromagnetic profiles for one or more inference operations, time profiles for performing one or more inference operations, or power consumption profiles of the computing unit performing the inference operations. In some embodiments, measurable characteristics may further include audio profiles and / or temperature profiles for performing one or more inference operations. An electromagnetic profile may represent, for example, a measure of electromagnetic radiation relative to capacitor charging on a hardware device when the hardware device is performing calculations of a machine learning model. In some embodiments, characteristic profiles may be represented by a graph having a horizontal axis representing time and a vertical axis representing a specific characteristic (e.g., electromagnetic radiation, power consumption, audio, or temperature).

[0048] The obfuscated network structures associated with a neural network may generally include obfuscation nodes in a critical network layer, one or more obfuscated network layers immediately preceding or following a critical network layer, or one or more parallel obfuscated network layers running in parallel with the critical network layer. The term "obfuscation of a network layer immediately preceding a critical layer" generally refers to a situation where the output from an obfuscated network layer is received as input to the critical layer, and the term "obfuscation of a network layer immediately following a critical layer" generally refers to a situation where the input to the obfuscated layer is the output from the critical layer. Obfuscation operations associated with obfuscation nodes and parallel obfuscated networks are scheduled by the secure machine learning model compiler 105 to run in parallel with the inference operations of the associated critical network layer, and obfuscation operations associated with one or more obfuscated network layers immediately preceding or following a critical network layer are scheduled to run sequentially with the inference operations specified by the critical network layer. For example, the obfuscation operations of an obfuscation layer preceding a critical layer are executed before the critical layer, as if the critical layer were receiving output from the preceding obfuscation layer. Similarly, the obfuscation operations of the obfuscation layer following the critical layer are performed after the critical layer, as if the output of the critical layer were received as input to the subsequent obfuscation layer.

[0049] In some embodiments, obfuscation operations can mimic corresponding inference operations. For example, the inference operations performed by the nodes of the critical layer and the obfuscation operations specified by the obfuscation nodes of the critical layer can be related to the activation function and executed in parallel. In another example, the obfuscation operations of the obfuscation layers before and after the critical layer may be similar to the operations specified by the critical layer. Therefore, the combined data profile (e.g., power consumption profile) is different from one that performs only inference operations, and the measurable characteristics deviate from the true measurable characteristics of the neural network. Consequently, any reverse-engineered neural network parameters will differ from the true neural network parameters. For example, a reverse-engineered neural network may include obfuscation nodes and obfuscated network layers that are not present in the original neural network.

[0050] In some embodiments, the obfuscation operation may be independent of (e.g., separate from) the corresponding inference operation, but by performing the obfuscation operation, any measurable data from the hardware device 102 can be rendered meaningless. For example, if the inference operation involves matrix reduction, the obfuscation operation may be addition or multiplication of a specific logical operation or scalar such that the true measurable profile is modified to lose its pattern or features, rendering it meaningless in determining the true parameters of the neural network. As another example, the obfuscated network structure may include a parallel obfuscation layer that includes an obfuscation operation performed in parallel with an inference operation specified by one or more critical layers, so that the measurable data from the hardware device 102 becomes meaningless.

[0051] The results of the inference operations are stored in memory 110 and provided to host 108 for other inference operations that depend on the results. However, the results of the obfuscation operations are not used in any inference operations. In some embodiments, the obfuscation results are discarded without being written to any memory. Alternatively, the obfuscation results are written to a memory unit that does not receive any data stored in the memory unit from other computational components of the inference operations. In other examples, the obfuscation results are written to the same memory as the inference operations but are not accessed by the inference operations.

[0052] Figure 2 shows an exemplary secure machine learning model compiler 200. The secure machine learning model compiler may be equivalent to, or used to implement, the secure machine learning model compiler 105 in Figure 1.

[0053] As shown in Figure 2, the secure machine learning model compiler 105 is configured to process input data and generate output data by processing the input data. As described above, the input data may include one or more machine learning models (e.g., neural networks) encoded in a high-level programming language. The output data may include compiled machine learning models encoded in a machine-readable low-level programming language (e.g., binary code).

[0054] The secure machine learning model compiler 200 may include a security engine 210 configured to determine, for example, whether to compile the machine learning model in the input data in secure mode or standard mode, based on the characteristics of the machine learning model. The characteristics of the machine learning model can represent the time and cost required to train the machine learning model. The time and cost required to train a machine learning model may generally relate to the size of the machine learning model, the complexity of designing the structure of the machine learning model, the size of the training examples used to train the machine learning model, the accuracy level set for training the machine learning model, the method by which the machine learning model is trained (separate training with additional machine learning models or end-to-end training), or other factors. For machine learning models that require high costs, the security engine 210 may decide to compile the machine learning model in secure mode. Otherwise, the machine learning model is compiled by the secure machine learning model compiler 200 in standard mode.

[0055] Alternatively or additionally, the characteristics of a machine learning model can indicate whether or not it is security sensitive. For example, as described above, a machine learning model trained for face unlocking is security sensitive. For security sensitive machine learning models, the security engine 210 may decide to compile those machine learning models under secure mode. Otherwise, the machine learning model is compiled under standard mode by the secure machine learning model compiler 200.

[0056] The characteristics of a machine learning model can be provided to the security engine 210 for analysis in different ways. For example, the characteristics of a machine learning model can be stored in metadata associated with the machine learning model included in the input data.

[0057] In some implementations, the secure machine learning model compiler 200 may receive instructions regarding whether to compile a machine learning model under secure mode. These instructions may be contained in instruction data stored in memory, embedded in or associated with a program representing the machine learning model, or provided by the user via a user interface. For example, the instructions may include a flag value embedded in the program representing the machine learning model. The flag value can be set interchangeably by the user, allowing the secure machine learning model compiler 200 to compile the corresponding machine learning model in standard mode or secure mode. The flag value can be a binary value, a boolean value, a string, a real number, or any other appropriate value. For example, a flag value of "0" allows the compiler 200 to compile the machine learning model in standard mode, and a flag value of "1" allows the compiler 200 to compile it in secure mode. The secure machine learning model compiler 200 further includes a modification engine 220 that determines the obfuscation structure associated with the machine learning model. For machine learning models that the security engine 210 decides to compile under standard mode, the secure machine learning model compiler 200 (or modification engine 220) does not determine the obfuscation network structure, and the machine learning model is compiled as is. For a machine learning model that the security engine 210 decides to compile under secure mode, the modification engine 220 determines one or more critical structures of the machine learning model (e.g., critical layers of a neural network) and corresponding obfuscation structures associated with one or more critical structures.

[0058] For example, if the machine learning model is a neural network, the modification engine 220 determines the critical layer in the sequence of network layers included in the neural network and determines a different obfuscation network structure associated with the critical layer. The obfuscation network structure specifies a series of obfuscation operations to modify the observed properties of the neural network, ultimately preventing the neural network parameters from being deciphered by an unauthorized third party. For simplicity, the following specification uses "neural network" as an illustrative machine learning model for illustrative purposes. It should be noted that the machine learning model can include any other suitable model other than a neural network, and the techniques described may be appropriately applied to different machine learning models.

[0059] The modification engine 220 can use various techniques to determine critical layers within a neural network. For example, the modification engine 220 can determine whether a layer is a critical layer based on one or more criteria. The criteria may include, for example, (i) whether a layer in the neural network contains one or more node weights that were updated beyond a threshold during training, (ii) whether the number of node weights updated in the network layer exceeds a threshold number, (iii) whether the number of node weights dropped out or determined to be zero in the network layer exceeds a threshold number, or other appropriate criteria. In response to determining that a layer satisfies one or more of the above criteria, the modification engine 220 can determine that such a layer is a critical layer. As a simple example, a trained machine learning model may be a pre-trained neural network after fine-tuning, and the modification engine 220 can determine that the layer with the largest change in node weights, or the layer where the change in node weights satisfies a threshold, is a critical layer. As another example, the modification engine 200 can determine whether a layer is a critical layer based on the layer's input data and / or output data. The correction engine 200 can compare one or more characteristics of the input and / or output data (e.g., data type, data size, or other characteristics) to their respective thresholds or a predetermined set of characteristics. In some situations, the correction engine 200 can further compare the data read / write levels associated with the input / output data of the network layer to the thresholds. For example, the correction engine 200 can determine a layer to be a critical layer if the data read / write levels associated with that layer meet the threshold.

[0060] After determining the critical layer, the modification engine 220 may determine one or more obfuscation network structures for the critical layer. The modification engine 220 may determine the obfuscation network structures based on different obfuscation tasks. For example, the obfuscation network structure may include obfuscation nodes having obfuscation node operations to be added to the critical layer. As another example, the obfuscation network structure may include obfuscation network layers immediately before and / or immediately after the critical layer. Alternatively or additionally, the obfuscation network structure may include one or more parallel obfuscation network layers specifying obfuscation operations scheduled to run in parallel with the inference operations of the critical layer. Details of generating the obfuscation network structures will be described with reference to Figures 3 to 5.

[0061] The scheduler 230 is configured to schedule the execution of both inference and obfuscation operations in the computing units of the hardware device. For example, the scheduler 230 may assign one or more inference operations to a first computing unit portion of the hardware device and one or more obfuscation operations to a second computing unit portion of the hardware device. The scheduler 230 may further determine when to perform the obfuscation and inference operations. For example, the scheduler 230 may allocate portions of the computing units to execute the inference and obfuscation operations in parallel or sequentially. In situations where the hardware device includes a dedicated computing unit and / or processing element for performing obfuscation operations, the scheduler 230 may assign the obfuscation operations to the dedicated computing unit and / or processing element. In this way, the hardware device can maximize the utilization of the processing element for inference operations.

[0062] Figure 3 shows an exemplary process 300 of an obfuscated network structure associated with an input neural network. The obfuscated network structure may be determined by a secure machine learning model compiler when the compiler compiles the input neural network. The secure machine learning model compiler may be equivalent to the secure machine learning model compiler 105 in Figure 1 and / or the secure machine learning model compiler 200 in Figure 2.

[0063] As shown in Figure 3, the secure machine learning model compiler included in the described system can determine the critical layer of the input neural network. The neural network may include multiple network layers arranged in sequence (e.g., network layers 312, 314, 316). The secure machine learning model compiler can determine whether to compile the neural network under secure mode, and in response to the decision to compile the neural network under secure mode, the secure machine learning model compiler can determine the critical layer from the sequence of network layers.

[0064] As described above, a secure machine learning model compiler can determine a critical layer based on one or more criteria, for example, by comparing the number of weights changed for a layer to a threshold number, or by comparing a measured value change in the node weights of a layer to a threshold. If a network layer satisfies one or more criteria, the secure machine learning model compiler determines that network layer to be a critical layer. For example, as shown in Figure 3, a secure machine learning model compiler can determine a critical layer 312 in an input neural network. The neural network further includes one or more preceding network layers 314 that precede the critical layer in the sequence, and one or more succeeding network layers 316 that follow the critical layer in the sequence. The critical layer 312 may include one or more nodes 306A to N, each having node weight values ​​and a corresponding node operation (e.g., the node activation function described above).

[0065] The obfuscated network structure in the exemplary process 300 includes one or more obfuscated nodes 370A-N associated with the critical layer 312 (e.g., by adding obfuscated nodes). The modified critical layer 352 represents both the original inference operation and the obfuscated operation specified by the added obfuscated nodes 370A-N.

[0066] A secure machine learning model compiler can incorporate one or more obfuscated nodes 370A-N into the critical layer in various ways when compiling the original neural network. For example, a secure machine learning model compiler can add an obfuscated node 370A between two adjacent nodes (e.g., 306A and 306B). Alternatively or additionally, a secure machine learning model compiler can add two or more obfuscated nodes (e.g., 370C, 370D, 370E, and 370F) between two adjacent nodes (e.g., 306C and 306D). Each obfuscated node 370A, 370B, ..., or 370N represents an obfuscated node weight and an obfuscated node operation, further specifying the corresponding obfuscated operation. For example, an obfuscated node may include an obfuscated node weight having a predetermined value, e.g., zero, one, or other values. The obfuscated node weight may be similar to, for example, one of the two adjacent original nodes. The node operations of obfuscated nodes 370A-N may include any suitable node operations such as Tanh, Sigmoid, or other node operations as described above. The obfuscated node operation can be, for example, similar to one of two adjacent nodes.

[0067] The obfuscation operation may include operations on values ​​associated with these obfuscated nodes 370A-N. For example, the obfuscation operation may include one or more obfuscated node weights, one or more obfuscated node operations, or appropriate operations on both (e.g., matrix multiplication, tensor reduction, pooling, or other appropriate operations). As described above, the input values ​​of the obfuscation operation may be predetermined and stored in one or more memory units in the hardware device. The output values ​​obtained by performing the obfuscation operation are typically not used in subsequent inference operations. Rather, these output values ​​are discarded or stored in one or more memory addresses that are not accessed for inference operations.

[0068] Furthermore, obfuscation operations are specified by obfuscation nodes 370A-N within the critical layer, and these obfuscation operations are scheduled by a secure machine learning model compiler and sent to the hardware device in instruction data format so that they run in parallel with the inference operations specified by the original node and the critical layer.

[0069] Figure 4 shows another exemplary process 400 of an obfuscated network structure associated with an input neural network. The obfuscated network structure may be determined by a secure machine learning model compiler when the compiler compiles the input neural network. The secure machine learning model compiler may be equivalent to the secure machine learning model compiler 105 in Figure 1 and / or the secure machine learning model compiler 200 in Figure 2.

[0070] The secure machine learning model compiler included in the described system can determine the critical layer 412 of the input neural network for compilation from a sequence of network layers (414, 412, and 416), as shown in Figure 4. One or more network layers 414 are preceding neural network layers that precede the critical layer 412 in the sequence, and one or more network layers 416 are succeeding network layers that follow the critical layer 412 in the sequence. The critical layer 412 may include one or more nodes 406A to N, each having node weight values ​​and a corresponding node operation (e.g., the node activation function described above).

[0071] The obfuscated network structure of the exemplary process 400 includes one or more obfuscated network layers 464 and / or 462. The secure machine learning model compiler is configured to modify the input neural network by adding one or more obfuscated network layers 464 and / or 462 before and / or after the critical layer 412 in sequence. For example, one or more obfuscated network layers 464 are added before the critical layer 412 and after the preceding network layer 414. Alternatively or additionally, other obfuscated network layers 462 are further added immediately after the critical layer 412 and before the subsequent network layer 416. Note that the obfuscated layers may include any appropriate number, e.g., 1, 2, 3, 5, 10, or any other appropriate number of obfuscated layers before and / or after the critical layer 412.

[0072] Each obfuscation network layer may include one or more obfuscation nodes. Each obfuscation node within an obfuscation network layer may include obfuscation node weight values ​​and obfuscation node operations. Each obfuscation network layer may include one or more obfuscation operations associated with the obfuscation layer. For example, an obfuscation operation may include one or more interlayer matrix operations (e.g., multiplying the layer output generated from the previous layer by the obfuscation node weights of the obfuscation layer). Another example is that an obfuscation operation may include one or more pooling operations on the layer output of the previous layer. In some embodiments, the layer and node operations within an obfuscation layer may be the same as those specified by the critical layer. Alternatively, the obfuscation operations specified by the obfuscation layer may be of a different type than those in the critical layer. For example, an obfuscation operation may include matrix multiplication while the critical layer includes pooling or softmax operations, or vice versa.

[0073] As described above, the input values ​​for the obfuscation operation specified by the obfuscation layer are predetermined and may be stored in one or more memory units within the hardware device. The output values ​​obtained by performing the obfuscation operation specified by the obfuscation layer are typically not used in subsequent inference operations. Rather, these output values ​​are either discarded or stored in one or more memory addresses that are not accessed for inference operations.

[0074] Furthermore, the obfuscation operations specified by the obfuscation layer are scheduled by a secure machine learning model compiler to be executed according to the sequence of network layers in the modified neural network. The output of the obfuscated network layer is not typically used for inference operations specified by subsequent network layers (e.g., critical layers), but the inference operations specified by the original network layers, and the obfuscation operations specified by the obfuscation layer, are still executed sequentially according to the layer sequence, as if the original network layers were receiving output from the obfuscation layer. This makes it more difficult for an unauthorized third party to determine which layers in the input neural network are obfuscated layers and which are actual layers based on data profiles observed from the aforementioned hardware device.

[0075] Figure 5 shows another exemplary process 500 of an obfuscated network structure associated with an input neural network. The obfuscated network structure may be determined by a secure machine learning model compiler when the compiler compiles the input neural network. The secure machine learning model compiler may be equivalent to the secure machine learning model compiler 105 in Figure 1 and / or the secure machine learning model compiler 200 in Figure 2.

[0076] As described above in relation to Figure 4, the obfuscated network structure of the exemplary process 500 also includes an obfuscated network layer associated with the critical layer. However, the secure machine learning model compiler does not use the obfuscated network layer to modify the structure (or parameters) of the input neural network. Instead, the secure machine learning model compiler determines one or more obfuscated layers that specify obfuscation operations to be performed in parallel with the inference operations specified by the critical layer. These obfuscated layers will also be referred to as parallel obfuscation layers in the following description.

[0077] More specifically, when a secure machine learning model compiler compiles an input neural network, it determines one or more parallel obfuscation layers 572 associated with the critical layer 512 in the sequence of network layers 512, 514, and 516 of the input neural network.

[0078] One or more parallel obfuscation network layers 572 each have one or more obfuscation nodes 574. For example, a first parallel obfuscation network layer may include multiple obfuscation nodes 580A-N, and a second parallel obfuscation network layer may include multiple obfuscation nodes 585A-N. As described above, each obfuscation node in an obfuscation network layer may include an obfuscation node weight value and an obfuscation node operation. Each parallel obfuscation network layer may include one or more obfuscation operations associated with the parallel obfuscation layer. For example, an obfuscation operation may include one or more inter-layer matrix operations (e.g., multiplying the layer output generated from the previous layer by the obfuscation node weight of the obfuscation layer). As another example, an obfuscation operation may include one or more pooling operations on the layer output of the previous layer. In some embodiments, the layer and node operations in a parallel obfuscation layer may be the same as those specified by the critical layer. Alternatively, the obfuscation operations specified by the parallel obfuscation layer may be of a different type than those in the critical layer. For example, the obfuscation operation may include pooling or softmax operations, while the critical layer may include matrix multiplication, or vice versa.

[0079] As described above, the input values ​​for the obfuscation operations specified by the concurrent obfuscation layer are predetermined and may be stored in one or more memory units within the hardware device. The output values ​​obtained by performing the obfuscation operations specified by the concurrent obfuscation layer are typically not used in subsequent inference operations. Rather, these output values ​​are either discarded or stored in one or more memory addresses that are not accessed for inference operations.

[0080] Furthermore, the obfuscation operations specified by the parallel obfuscation layer are scheduled by a secure machine learning model compiler to run in parallel with the inference operations specified by the corresponding critical network layer. In this way, the observed profile of one or more properties of the neural network is modified, thereby further preventing, or at least increasing the time and cost, of deciphering one or more parameters of the neural network (e.g., parameters specifying one or more critical layers of the neural network).

[0081] For simplicity, Figures 3-5 show only one critical layer, but it should be noted that a secure machine learning model compiler is configured to determine multiple critical layers within a neural network. Furthermore, although Figures 3-4 show only one type of obfuscated network structure, it should be understood that one or more obfuscated network structures can be combined in various ways for a critical network layer.

[0082] Figure 6 is an exemplary flowchart of process 600 for compiling an input neural network for an obfuscated network structure. For convenience, process 600 is described as being executed by a system of one or more computers located in one or more locations. For example, process 600 can be executed on or by a compiler located on a host, such as the secure machine learning model compiler 105 shown in Figure 1 or the secure machine learning model compiler 200 shown in Figure 2. The order of steps in process 600 is exemplary and can be executed in a different order. In some embodiments, process 600 may include additional or fewer steps, or some steps may be divided into multiple steps.

[0083] The system receives data representing a machine learning model (610). The machine learning model can include various types of models, such as a neural network. The data representing a neural network can specify multiple inference operations. The neural network can include parameters specifying a sequence of network layers, multiple nodes in each of the layers, node weights and operations per node, and other structures associated with the neural network. In some implementations, the data may further include metadata indicating additional information related to the machine learning model. For example, the metadata may indicate whether the machine learning model is security sensitive, and / or the time and / or cost to train the machine learning model. For simplicity, the following description is based on a neural network received by the system.

[0084] The system generally compiles the neural network to generate instructions that, when executed, cause one or more computing units of a hardware device to perform the neural network's inference operations, and, in some cases, also perform obfuscation operations associated with the neural network.

[0085] An obfuscation operation, when performed, can obfuscate at least one or more measurable properties of a neural network. More specifically, an obfuscation operation can alter at least one or more measurable properties of a neural network. For example, as described above, an obfuscation operation, when performed, is configured to obscure at least one of the following: the number of network layers in a neural network, the number of nodes in a network layer of a neural network, the node operations of the nodes in a network layer of a neural network, or the weight values ​​associated with the nodes in a network layer of a neural network.

[0086] Measurable characteristics may include at least one of the following: power profile, electromagnetic profile, or time profile. When measurable characteristics are changed, it becomes more difficult to determine the parameters of the neural network based on them. Obfuscation operations may include operations similar to inference operations, such as activation function operations, tensor multiplication, and subtraction. For example, an obfuscation operation may include an obfuscation node operation specified by an obfuscation node in the network layer, performed in parallel with other node operations in the network layer. An obfuscation node operation can be node addition or node multiplication. Alternatively, an obfuscation node operation of an obfuscation node may specify an activation function for a particular node that is different from the actual activation function of the actual node, which is performed in parallel with the obfuscation node operation in the same network layer. In some embodiments, obfuscation operations may include operations unrelated to and / or different from inference operations. Obfuscation operations may be performed in parallel with the corresponding inference operations, or sequentially in order. Further details regarding obfuscation operations are described above.

[0087] To compile a neural network, the system determines whether to obfuscate one or more measurable properties of the neural network (620). As described above, the system determines whether to obfuscate one or more measurable properties of the neural network based on the properties or nature of the neural network. Based on the input data associated with the neural network, the system determines whether the neural network is security sensitive and / or whether training to meet a certain threshold required a considerable amount of time and / or cost. In response to determining that the neural network is security sensitive and / or requires a considerable amount of resources, the system decides to compile the neural network under “secure mode” as described above. In “secure mode”, the system can obfuscate one or more measurable properties of the neural network by determining one or more obfuscation operations to be included in the instructions in addition to the inference operations specified by the neural network when compiling the neural network, and the one or more obfuscation operations, when executed together with the inference operations, can obfuscate one or more measurable properties of the neural network. In some embodiments, step 620 is optional. For example, a system or host may receive user instructions or input data that instructs the host to compile the input neural network under "secure mode" without having to make a decision.

[0088] The system determines the critical layers of a sequence of network layers (630). The critical layers are determined based on the characteristics of the neural network. For example, the system can determine the critical layers based on the type of layer (e.g., pooling layers, fully connected layers, softmax layers). As another example, the system can determine the critical layers based on updates to the parameters of the network layers during training. More specifically, the system can compare parameter updates (e.g., the number of node updates in a network layer, changes in node values ​​within a network layer, the number of dropout nodes in a network layer, or changes in inter-layer connectivity between a network layer and adjacent layers) to one or more criteria (e.g., a number or value of thresholds). Alternatively, the system can receive metadata associated with a neural network that indicates one or more critical layers. A detailed explanation of determining critical layers is provided above.

[0089] The system determines the obfuscated network structure associated with the critical layer (640). As described above, the system determines various obfuscated network structures associated with the critical layer. For example, an obfuscated network structure may include obfuscated nodes that specify their respective obfuscated node weights and obfuscated node operations. The system can modify a neural network by adding one or more obfuscated nodes to the critical layer. The obfuscated operations specified by the obfuscated nodes added to the critical layer are generally executed in parallel with the inference operations specified by the original nodes of the critical layer. As another example, an obfuscated network structure may include one or more obfuscated network layers. The system can modify a neural network by inserting one or more obfuscated network layers (immediately before) and / or (immediately after) the critical layer in a sequence of network layers. Each obfuscated network layer contains multiple obfuscated nodes as described above. The obfuscated operations specified by the obfuscated network layers are generally executed in order. For example, the order is determined based on the sequence of modified network layers in the modified network. Alternatively or additionally, an obfuscated network structure may include one or more concurrent obfuscated network layers. Unlike obfuscated network layers inserted into the original sequence of network layers in a neural network, the system does not modify the neural network using concurrent obfuscation layers. Instead, the system maintains the neural network structure unchanged and generates instructions to hardware devices to perform the obfuscation operations of the concurrent obfuscation layers in parallel with one or more corresponding critical layers.

[0090] The system compiles a neural network using the associated obfuscation network structure (650). During compilation, the system generates instructions for performing the obfuscation operations specified by the obfuscation network structure and the inference operations specified by the compiled neural network. In some embodiments, the obfuscation operations are scheduled to run in parallel with the inference operations, e.g., obfuscation operations specified by the obfuscation network structure, such as obfuscation nodes and parallel obfuscation network layers. Alternatively or additionally, the obfuscation operations and inference operations are scheduled to run sequentially in order, e.g., the obfuscation operations specified by the obfuscation network structure, such as obfuscation network layers.

[0091] In some embodiments, the system further determines a schedule in the instructions for one or more corresponding hardware devices. The schedule generally specifies a sequence for the hardware devices to perform obfuscation operations specified by the corresponding obfuscation network structure and inference operations specified by the neural network. For each of the one or more hardware devices, the schedule further indicates different portions of the set of computing units of the hardware device to perform the respective inference and / or obfuscation operations.

[0092] In some embodiments, the system may transmit data containing the above instructions to one or more hardware devices. Exemplary hardware devices may include edge devices such as smartphones, smartwatches, smart tablets, or laptops. In some embodiments, the system may further generate data to trigger instructions to be executed by one or more hardware devices, thereby causing inference operations specified by the neural network and obfuscation operations specified by the obfuscation network structure to be executed on the corresponding hardware devices (660).

[0093] This specification uses the term “configured” in relation to systems, devices, and components of computer programs. When one or more computer systems are configured to perform a particular operation or action, it means that software, firmware, hardware, or a combination thereof is installed on the system that causes the system to perform that operation or action during the operation. When one or more computer programs are configured to perform a particular operation or action, it means that one or more programs contain instructions that, when executed by a data processing device, cause the device to perform that operation or action. When a dedicated logic circuit is configured to perform a particular operation or action, it means that the circuit has electronic logic that performs that operation or action.

[0094] The term "data processing device" refers to data processing hardware and encompasses all types of devices, machines, and equipment for processing data, including, for example, programmable processors, computers, or multiple processors or computers. A device may also be, or further include, a special-purpose logic circuit (e.g., an FPGA (Field-Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit)). In addition to hardware, a device may optionally include code that creates an execution environment for computer programs, such as processor firmware, protocol stacks, database management systems, operating systems, or code that constitutes one or more of these.

[0095] Computer programs, which may be called or described as programs, software, software applications, modules, software modules, scripts, or code, can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. Computer programs may, but do not necessarily, correspond to files in a file system. A program may be stored in a single file dedicated to a program of interest, in part with other programs or data, for example, in a file containing one or more scripts stored in a markup language document, or in multiple collaborative files, for example, in a file containing one or more modules, subprograms, or parts of code. Computer programs can be deployed to run on one computer, or on multiple computers located in one location or distributed across multiple locations and interconnected by a communication network.

[0096] The processes and logic flows described herein can be performed by one or more programmable computers executing one or more computer programs to act on input data and produce outputs. Process and logic flows can also be performed by dedicated logic circuits, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and devices can also be implemented as those dedicated logic circuits.

[0097] A computer suitable for running computer programs includes, for example, a general-purpose or dedicated microprocessor, or both, or any other type of central processing unit. Generally, the central processing unit receives instructions and data from read-only memory, random-access memory, or both. The basic elements of a computer are the central processing unit for executing and running instructions, and one or more memory devices for storing instructions and data. Generally, a computer also includes one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or is arithmetically coupled for receiving data from them, transferring data to them, or both. However, a computer is not required to have such devices. Furthermore, to give some examples, a computer can be embedded in other devices, such as mobile phones, smartphones, personal digital assistants (PDAs), mobile audio or video players, game consoles, Global Positioning System (GPS) receivers, or portable storage devices, such as Universal Serial Bus (USB) flash drives.

[0098] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices including EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. Processors and memory may be complemented by or incorporated into dedicated logic circuits.

[0099] Embodiments of the subject matter described herein can be implemented in a computing system including, for example, a backend component as a data server, or a computing system including a middleware component, for example, an application server, or a frontend component, for example, a client computer having a graphical user interface or a web browser on which a user can interact with embodiments of the subject matter described herein, or in a computing system including one or more such backend, middleware, or frontend components in any combination. The components of the system can be interconnected by any form or medium of digital data communication, for example, a communication network. Examples of communication networks include local area networks (LANs), wide area networks (WANs), for example, the Internet.

[0100] A computing system can include clients and servers. Clients and servers are generally geographically distant from each other and typically communicate through a communication network. The client-server relationship arises from computer programs running on each computer that have a client-server relationship with each other. In some embodiments, the server sends data, such as a hypertext markup language (HTML) page, to a user device, for example, to display data to a user interacting with the user device acting as a client, and to receive user input from that user. Data generated on the user device (e.g., the results of user interactions) can be received by the server from the user device.

[0101] In addition to the embodiments described above, the following embodiments are also innovative. Embodiment 1 is a method comprising receiving data representing a neural network including an inference operation, wherein the neural network includes a sequence of network layers and parameters specifying a plurality of nodes in each layer of the sequence of network layers, the method further comprises compiling the neural network to generate instructions for causing one or more computing units of a hardware device to perform an obfuscation operation associated with the neural network and the inference operation of the neural network, wherein the obfuscation operation obfuscates one or more measurable properties of the neural network, the compilation comprising determining a critical layer in the sequence of network layers, determining an obfuscated network structure to associate with the critical layer, and compiling the neural network together with the associated obfuscated network structure to generate instructions for performing the obfuscation operation specified by the obfuscated network structure, the method further comprises causing the hardware device to perform the inference operation and the obfuscation operation.

[0102] Embodiment 2 is the method of Embodiment 1, further comprising determining that compiling obfuscates one or more measurable properties of the neural network.

[0103] Embodiment 3 is a method of Embodiment 1 or 2, wherein the hardware device is configured to perform the inference operation and the obfuscation operation in parallel.

[0104] Embodiment 4 is a method according to any one of Embodiments 1 to 3, wherein the hardware device is configured to perform the inference operation and the obfuscation operation sequentially.

[0105] Embodiment 5 is a method according to any one of Embodiments 1 to 4, wherein the obfuscation operation is configured to obscure at least one of the following: the number of network layers of the neural network, the number of nodes in the network layers of the neural network, the node operations of the nodes in the network layers of the neural network, or the weight values ​​associated with the nodes in the network layers of the neural network.

[0106] Embodiment 6 is the method according to any one embodiment of Embodiments 1 to 5, wherein the one or more measurable characteristics of the neural network include at least one of a power profile, an electromagnetic profile, or a time profile.

[0107] Embodiment 7 is a method according to any one embodiment of Embodiments 1 to 6, wherein determining the critical layer in the sequence of network layers includes determining the critical layer based on at least one of the network layer type, updates to network layer parameters, or metadata associated with the network layer.

[0108] Embodiment 8 is a method according to any one embodiment of Embodiments 1 to 7, wherein determining the obfuscated network structure associated with the critical layer includes adding an obfuscated network layer immediately before and / or immediately after the critical layer in the sequence of network layers.

[0109] Embodiment 9 is a method according to any one embodiment of Embodiments 1 to 8, wherein determining the obfuscated network structure associated with the critical layer includes determining an obfuscated network layer with obfuscation operations performed in parallel with the inference operations of the critical layer.

[0110] Embodiment 10 is a method according to any one embodiment of Embodiments 1 to 9, wherein determining the obfuscated network structure associated with the critical layer includes adding obfuscated nodes to the set of nodes in the critical layer, the obfuscated nodes include obfuscation operations performed in parallel with the inference operations of the critical layer.

[0111] Embodiment 11 is a method according to any one of Embodiments 1 to 10, wherein the specific hardware device comprises an edge device.

[0112] Embodiment 12 is a method according to any one embodiment of Embodiments 1 to 11, wherein compiling the neural network to generate instructions includes determining a schedule for the instructions on a particular hardware device, the schedule specifying a sequence for performing the obfuscation and inference operations on each set of computing units of the particular hardware device.

[0113] Embodiment 13 is a system comprising one or more computers and one or more storage devices for storing instructions, wherein when an instruction is executed by one or more computers, the system causes one or more computers to perform their respective calculations, and the calculations include the method described in any one of Embodiments 1 to 12.

[0114] Embodiment 14 is one or more computer-readable storage media for storing instructions, wherein when an instruction is executed by one or more computers, the one or more computers cause the one or more computers to perform their respective calculations, and each of the calculations includes the method described in any one of Embodiments 1 to 12.

[0115] While this specification provides details of many specific embodiments, these should not be interpreted as limitations on the scope of what can be claimed, but rather as descriptions of features that may be specific to a particular embodiment. Certain features described herein in the context of individual embodiments can also be implemented in combination in a single embodiment. Conversely, various features of the present invention described in the context of a single embodiment can also be implemented separately or in any preferred secondary combination in multiple embodiments. Furthermore, features may be described above as functioning in a particular combination, and even if initially claimed as such, one or more features from the claimed combination may be removed from the combination, and the claimed combination may cover secondary combinations or variations of secondary combinations.

[0116] Similarly, while calculations are shown in a specific order in the drawings, this should not be understood as requiring that such calculations be performed in a specific or sequential order shown, or that all shown calculations be performed, in order to obtain the desired result. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and the described program components and systems can generally be integrated into a single software product or packaged into multiple software products.

[0117] In each example where an HTML file is mentioned, other file types or formats may be substituted. For example, an HTML file could be replaced with XML, JSON, plain text, or other types of files. Furthermore, where a table or hash table is mentioned, other data structures (such as spreadsheets, relational databases, or structured files) may be used.

[0118] Specific embodiments of the present invention have been described. Other embodiments are within the scope of the following claims. For example, the desired results can still be obtained by performing the steps enumerated in the claims, the steps described herein, or the steps shown in the drawings in a different order. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. It is a method, The method includes receiving data representing a neural network including inference operations, wherein the neural network includes a sequence of network layers and parameters specifying a plurality of nodes in each layer of the sequence of network layers, and the method further includes: When executed, the process includes compiling the neural network to generate instructions that cause one or more computing units of a hardware device to perform obfuscation operations associated with the neural network and the inference operations of the neural network, wherein the obfuscation operations, when executed, obfuscate one or more measurable properties of the neural network, and the compilation process is Determining the target layer in the aforementioned network layer sequence, Determining an obfuscated network structure for associating with the aforementioned target layer, The method further includes compiling the neural network together with the associated obfuscation network structure to generate instructions for performing the obfuscation operation specified by the obfuscation network structure, and further includes: A method comprising causing the inference operation and the obfuscation operation to be performed on the hardware device.

2. The method according to claim 1, further comprising determining that the compilation obfuscates one or more measurable properties of the neural network.

3. The method according to claim 1, wherein enabling the hardware device to perform the inference operation and the obfuscation operation includes causing the hardware device to perform the inference operation and the obfuscation operation in parallel.

4. The method according to claim 1, wherein enabling the hardware device to perform the inference operation and the obfuscation operation includes causing the hardware device to sequentially perform the inference operation and the obfuscation operation.

5. The method according to claim 1, wherein the obfuscation operation is configured to obscure at least one of the following: the number of network layers of the neural network, the number of nodes in the network layers of the neural network, the node operations of the nodes in the network layers of the neural network, or the weight values ​​associated with the nodes in the network layers of the neural network.

6. The method according to claim 1, wherein the one or more measurable characteristics of the neural network include at least one of a power profile, an electromagnetic profile, or a time profile.

7. The method according to claim 1, wherein determining the target layer in the sequence of network layers includes determining the target layer based on at least one of the type of network layer, updates to the parameters of the network layer, or metadata associated with the network layer.

8. The method according to claim 1, wherein determining the obfuscated network structure associated with the target layer includes adding an obfuscated network layer immediately before and / or immediately after the target layer in the sequence of network layers.

9. The method according to claim 1, wherein determining the obfuscated network structure associated with the target layer includes determining an obfuscated network layer having an obfuscation operation performed in parallel with the inference operation of the target layer.

10. The method according to claim 1, wherein determining the obfuscated network structure associated with the target layer includes adding obfuscated nodes to a set of nodes in the target layer, the obfuscated nodes include obfuscation operations performed in parallel with the inference operations of the target layer.

11. The method according to claim 1, wherein the specific hardware device comprises an edge device.

12. Compiling the neural network to generate instructions is The method according to claim 1, comprising determining a schedule in the instructions for a specific hardware device, wherein the schedule specifies a sequence for performing the obfuscation operation and the inference operation for each set of computing units of the specific hardware device.

13. A system comprising one or more computers and one or more storage devices for storing instructions, wherein when an instruction is executed by one or more computers, it causes the one or more computers to perform the respective calculation, and the calculation includes the method according to any one of claims 1 to 12.

14. A program that causes one or more computers to perform each operation, wherein each operation includes the method according to any one of claims 1 to 12.

Images