Modifying machine learning model to obfuscate characteristics of machine learning model

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for obfuscating operations of neural networks. One of the methods includes receiving data representing a neural network. The neural network includes parameters specifying a sequence of network layers and a plurality of nodes in each layer of the sequence of network layers. The neural network is compiled to generate instructions that, when executed, cause one or more computing units of the hardware device to perform obfuscation operations associated with inference operations of the neural network. The obfuscation operation, when performed, obfuscates one or more measurable characteristics of the neural network. Compiling a target layer including determining a sequence of network layers; determining an obfuscated network structure for association with the target layer; and compiling the neural network and the associated obfuscation network structure together to generate an instruction for performing an obfuscation operation specified by the obfuscation network structure.

Description

Technical Field

[0001] The present specification generally relates to machine learning. In particular, the present specification describes techniques for modifying a machine learning model to include a structure that obfuscates the inference operations of the machine learning model when the machine learning model is executed. Background Art

[0002] Artificial intelligence (AI) is the intelligence exhibited by machines and represents the ability of a computer program or machine to think and learn. One or more computers can be used to perform calculations to train machine learning models for corresponding tasks. Neural networks belong to a subfield of machine learning models.

[0003] A neural network may employ one or more node layers representing a plurality of operations (e.g., vector or matrix operations). One or more computers may be configured to perform operations or calculations of the neural network to generate an output, such as a classification, prediction, or segmentation of a received input. In addition to an output layer, some neural networks include one or more hidden layers. The output of each hidden layer serves as the input to the next layer in the network, i.e., as the input to the next hidden layer or output layer. Each layer of the network generates an output from the received input according to the current values ​​of a corresponding set of network parameters.

[0004] Specially designed hardware accelerators can perform specific functions and operations faster and more efficiently than operations run by a general-purpose central processing unit (CPU), including operations or calculations specified in neural networks. Hardware accelerators can include graphics processing units (GPUs), tensor processing units (TPUs), video processing units (VPUs), field programmable gate arrays (FPGAs), or application-specific integrated circuits (ASICs). Summary of the invention

[0005] A machine learning model (e.g., a neural network) can be compiled and deployed on a hardware device after being properly trained, which is configured to perform inference operations for processing input data. The inference operations are defined by neural network parameters that are updated during the training process. These parameters define (i) node operations (e.g., linear and nonlinear operations) for nodes in each network layer of the neural network and (ii) the structure of the neural network. For example, the parameters defining the node operations include parameters defining the activation function for each network layer and parameters defining the node weights for nodes in the network layer. As another example, the parameters defining the structure (also called hyperparameters) include at least one of the number of nodes in the network layer, the number of network layers in the machine learning model (e.g., in the neural network), or node connections across adjacent layers (e.g., fully connected layers, convolutional layers, or transposed convolutional layers). For simplicity, the following description is described for neural networks, but can be applied to other types of machine learning models.

[0006] It is critical to keep the parameters of the trained neural network confidential. First, training a neural network, especially a deep neural network with satisfactory accuracy for generating predictions, requires considerable computational cost and time. In addition, some neural networks have applications related to security-sensitive authentication, such as face unlock tasks where the neural network is configured to recognize a face to conveniently unlock a device. Therefore, it is critical to keep the structure and parameters of the neural network undecipherable or at least difficult to decipher to avoid malicious actors learning these parameters and using them for unauthorized device unlocking.

[0007] However, different techniques can be applied to "decode" the trained neural network, especially when the neural network is implemented on a hardware device accessible to a third party (e.g., an edge device such as a smartphone, smart watch, smart tablet computer, or other edge device). For example, one technique can measure the characteristics of the trained neural network when the hardware device performs an inference operation of the trained neural network. More specifically, the technique can collect data such as power consumption, electromagnetic waves, or time when the hardware device performs an inference operation, and determine the neural network parameters and structure by analyzing a characteristic profile generated based on the collected data. This technique is also called a side channel attack.

[0008] The techniques described in this specification can enhance the security of a neural network implemented on a hardware device (e.g., on an edge hardware device). For example, the described techniques defend against side-channel attacks by determining one or more obfuscated network structures associated with one or more original network layers in a neural network during compile time, and generating instructions that, when executed by a hardware device, cause the hardware device to perform obfuscated operations specified by the obfuscated network structures concurrently and / or sequentially with reasoning operations of the one or more original network layers.

[0009] One or more original network layers are generally referred to as "target layers" of a neural network. In some cases, the system may determine one or more key layers of a neural network as target layers for obfuscation operations of the target layers. Target layers (e.g., key layers) generally require considerable time and cost (e.g., computing resource cost) to train. Alternatively or in addition, target layers (e.g., key layers) may generally include network layers that are quite important to the network, for example, layers that are critical to improving the performance of the neural network. The performance may include hardware resource requirements, time requirements, power requirements, or other requirements for performing reasoning operations for different tasks of the neural network. A system (or compiler) that performs the described techniques may determine the target layer by determining whether the network layer is a key layer based on one or more characteristics associated with the neural network. Example characteristics may include layer type, layer size, input and / or output of the layer, or other suitable characteristics. In some implementations, if once a layer meets one or more criteria, such as a threshold memory bandwidth, a threshold power consumption, or other criteria, the system may determine that the layer is a key layer. In this way, the described techniques may prevent side channel attacks, or at least raise the threshold for the computational cost and / or time cost of using side channel attacks to decipher the deployed neural network. Although these techniques are mainly described in terms of target layers (e.g., key layers), obfuscation layers may be added at or near other non-key layers. In addition, for simplicity, the term "key layer" is used in the following description, and it should be noted that the "key layer" may be equivalent, or determined or selected as a target layer, wherein operations in the target layer are obfuscated by introducing obfuscation operations executed on a hardware device.

[0010] As used throughout this specification, the term "obfuscation operation" generally refers to an operation that, when executed by a hardware device in conjunction with a machine learning operation (e.g., an inference operation) of a deployed neural network, causes one or more measurable characteristics of the neural network to change such that at least one parameter of the neural network (e.g., at least one of the number of network layers of the neural network, the number of nodes in a network layer, node operations for nodes in a network layer, or weights associated with nodes in a network layer) is obfuscated. It should be noted that different types of machine learning models include different types of parameters that define the model. The techniques described in this document can obfuscate any type of parameter that affects a measurable characteristic of a machine learning model.

[0011] The one or more measurable characteristics of a neural network generally refer to measurable data when a hardware device performs inference operations in the neural network. The measurable data may include data or profiles related to power consumption, time, electromagnetic radiation, or other measurable data, as described above.

[0012] It should also be noted that the obfuscation operation can be performed sequentially and / or concurrently with the machine learning operation, depending on the determined obfuscation network structure. The term "concurrently" as used throughout this specification generally refers to a common time period when both the obfuscation operation and the reasoning operation are performed by a hardware device. For example, the common time period can be exactly the same time period, substantially the same time period (e.g., within a threshold time period of each other), or two different time periods with overlapping areas. The term "sequentially" as used throughout this specification generally refers to performing obfuscation operations and reasoning operations according to a sequence in different time periods. For example, the obfuscation operation can be performed before or after performing one or more reasoning operations. Different time periods generally refer to time periods without overlapping areas.

[0013] In the case where the obfuscation operation is performed concurrently with the reasoning operation, the obfuscation network structure may, for example, include one or more obfuscation nodes to be added to one or more key layers. The obfuscation operation specified by the obfuscation node in the key layer can be performed concurrently with the reasoning operation of the original node in the key layer. As another example, when compiling the neural network, the obfuscation network structure can be additionally included in the instructions. However, these obfuscation network structures do not change the original structure or parameters of the neural network. On the contrary, the obfuscation operation specified by these obfuscation network structures will be performed by the hardware device concurrently with the reasoning operation in one or more key layers. The obfuscation network structure can mimic the operation performed by the key layer with similar data flow and / or data operation.

[0014] In the case where obfuscation operations and reasoning operations are performed in sequence, the obfuscation network structure may, for example, include one or more obfuscation network layers to be added immediately before or after the critical layer. Even if the output from the obfuscation network layer is not used for the downstream operation specified in the subsequent original layer, the compiler determines the sequence for performing the obfuscation operations in the obfuscation layer and the reasoning operations in the original layer as if the subsequent layer waits for the output from the previous obfuscation layer. In this way, it prevents the use of side channel attacks to distinguish between the critical layer and the obfuscation network layer or at least raises the "cost" threshold of the distinction.

[0015] Examples of obfuscation operations may include linear or nonlinear operations of suitable types. In the case where obfuscation operations are performed concurrently with specific reasoning operations of key layers, these obfuscation operations may sometimes mimic actual reasoning operations. For example, obfuscation operations performed concurrently with node linear operations of a particular node may also be linear operations such as addition, multiplication, and binary operations. As another example, obfuscation operations performed concurrently with node nonlinear operations of a particular node may also be nonlinear operations, such as activation functions, such as ReLU, Sigmoid, Tanh, or other suitable nonlinear operations. As another example, obfuscation operations may include tensor reduction operations that mimic action-weight multiplication of network layers. In the case where obfuscation operations are performed sequentially with reasoning operations of key layers, obfuscation operations may be any suitable node or inter-layer operations, such as linear matrix operations, nonlinear node activations, pooling, or other suitable operations. These operations may be independent of the reasoning operations performed by the associated key layers. Additional examples of obfuscation operations are described below.

[0016] In general, a host or a compiler included in the host can compile a machine learning model (e.g., a neural network) and generate instructions that, when executed by a hardware device, cause the hardware device to perform at least an inference operation of the neural network. Compiling a neural network generally refers to converting a program code in a high-level programming language (e.g., C++, Python, JAVA, or other programming language) representing the neural network into a machine-readable low-level programming language (e.g., binary code). The compiled neural network can be deployed on one or more hardware devices to perform operations according to corresponding instructions. During the compilation step, the described technology can determine an obfuscation network structure to be associated with one or more key layers of the neural network, and generate instructions that, when executed by the hardware device, cause the hardware device to concurrently and / or sequentially perform (i) an inference operation of the original inference operation of the neural network and (ii) an obfuscation operation specified by the obfuscation network structure.

[0017] The hardware device may include one or more processing elements configured to process respectively assigned operations according to instructions. Each of these processing elements may further include a plurality of computing units specifically arranged to perform the assigned operations. The assigned operations may include (i) machine learning calculations, for example, in a manner to accelerate the performance of the machine learning operations, and / or (ii) obfuscation operations specified by an associated obfuscation network structure. It should be noted that the described techniques are compatible with and independent of any type of hardware device suitable for performing machine learning operations (e.g., different types of accelerators, such as GPUs, TPUs, VPUs, FPGAs, or ASICs). This is because the described techniques are performed during a compilation step, and instructions for obfuscating parameters of a machine learning model may be determined in different ways according to different hardware devices.

[0018] The described techniques may determine one or more characteristics of a neural network prior to compilation, and based on the one or more characteristics, determine whether to associate an obfuscated network structure with the neural network, and if so, determine where and how to associate the obfuscated network structure with the neural network. More specifically, the described techniques determine whether to compile the neural network for execution in a "standard mode" (in which no obfuscation operations are added to instructions) or in a "secure mode" (in which obfuscation operations are added). Compiling the neural network so that the neural network executes in secure mode may be referred to as performing the compilation in secure mode. Similarly, compiling the neural network so that the neural network executes in standard mode may be referred to as performing the compilation in standard mode.

[0019] In general, if the neural network is associated with a security-sensitive task (e.g., a facial authentication task), the system can determine to perform compilation in a manner that prepares the neural network to run in a secure mode. Alternatively, if the neural network requires considerable time and cost to train, the system can determine to perform compilation in a secure mode. In some implementations, the system can determine the characteristics of the neural network through metadata associated with the neural network, or through data included in a request from one or more applications that will utilize the neural network. In some implementations, the system can determine whether to compile the neural network in a secure mode based on user input. For example, a developer of the neural network can specify that the neural network should be compiled and run in a secure mode.

[0020] After determining to compile the neural network in a safe mode, the described techniques can determine one or more key layers in a sequence of network layers in the neural network, and determine an obfuscated network structure to be associated with the one or more key layers. The key layers can be network layers that require considerable time and cost (e.g., computational cost) to train. Alternatively, the key layers can be layers with operations that significantly affect the performance of the neural network, such as a particular type of network layer in the sequence of the neural network, a particular inter-layer connection between a layer and an adjacent layer, or a layer with a specially designed node operation.

[0021] The obfuscated network structure may include obfuscated nodes to be included in the original key layer, obfuscated layers to be added immediately before or after the key layer, and / or node operations or layer operations that are separate from the neural network but executed concurrently with the reasoning operation of the key layer. These obfuscated nodes and layers and corresponding obfuscated operations do not affect the original reasoning operation of the neural network when executed, for example because the outputs of these obfuscated operations are not used by the original reasoning operation.

[0022] The subject matter described in this specification can be implemented in certain embodiments to achieve one or more of the following advantages. Compiling a neural network in a secure mode to obfuscate the parameters of the deployed neural network in a hardware device can prevent the neural network from being deciphered. More specifically, the obfuscation operation may cause one or more measurable characteristics of the neural network executed by the hardware device to change, making it difficult and more time-consuming and resource-intensive to decipher the corresponding neural network parameters based on the measurable characteristics. Therefore, the described techniques enhance data security by preventing the leakage of potentially sensitive machine learning data.

[0023] The subject matter described in this specification is more advantageous from a model compilation perspective. For example, the described techniques are generic and independent of the hardware device selected for deploying the compiled neural network. Instructions specifying inference operations and obfuscation operations are generated during compile time of the hardware device, and these operations are scheduled and assigned to different computational units and / or processing elements before the instructions are transmitted to the hardware device. Therefore, the described techniques do not require specially designed hardware devices, which enables the neural network to be deployed on different types of hardware devices.

[0024] Furthermore, the described techniques do not require modification of the neural network prior to compilation, and therefore, a user or technician does not need to modify the structure of the neural network in a high-level programming language to add obfuscation structures / operations. The obfuscated network structure and corresponding obfuscation operations are automatically determined at compile time and compiled. In this manner, the described techniques save considerable research and development time for updating the neural network (e.g., determining the obfuscation operations prior to compiling the neural network). This also prevents errors in the neural network that are introduced during development to include the associated obfuscation operations.

[0025] Other implementations of this aspect and other aspects include corresponding systems, devices, and computer programs that are configured to perform the actions encoded on computer storage devices of these methods. The system of one or more computers can be configured in this way by software, firmware, hardware, or a combination thereof installed on the system, which causes the system to perform these actions when running. One or more computer programs can be configured in this way by having instructions that, when executed by a data processing device, cause the device to perform these actions.

[0026] Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the following description. Other potential features, aspects, and advantages of the subject matter will become apparent from the specification, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a block diagram of an example system that includes an example secure machine learning model compiler.

[0028] Figure 2 An example secure machine learning model compiler is shown.

[0029] Figure 3 An example process of obfuscating a network structure associated with an input neural network is shown.

[0030] Figure 4 Another example process of obfuscating a network structure associated with an input neural network is shown.

[0031] Figure 5 Another example process of obfuscating a network structure associated with an input neural network is shown.

[0032] Figure 6 is an example flow chart of a process for compiling an input neural network with a confounding network structure.

[0033] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION

[0034] The subject matter described in this specification relates to modifying a machine learning model so that when the machine learning model is executed, an obfuscation operation that obfuscates measurable characteristics of the machine learning model is performed. More specifically, the described technology involves determining an obfuscated network structure associated with one or more specific layers of an input neural network (also referred to as an original neural network) before compiling the machine learning model; compiling the original neural network together with the determined obfuscated network structure; and sending data including the compiled neural network with the obfuscated network structure to one or more hardware devices to perform operations of the neural network. The obfuscated network structure represents a plurality of obfuscation operations that, when executed by corresponding hardware devices, can obfuscate one or more parameters of the original neural network. The obfuscation operation generally prevents malicious entities from deciphering network parameters based on an observed characteristic profile associated with the original neural network, or at least increases the cost and / or time of the deciphering. The obfuscated network structure to be added to the machine learning model can sometimes be predetermined or preset by a user, or received as an input to the system.

[0035] The term "hardware device" as used throughout the specification generally refers to a hardware processor, such as a hardware accelerator, deployed on an edge device (e.g., a smartphone, a smart tablet, a smart watch, or other suitable edge device). The hardware device may include one or more processing elements, and each processing element may include one or more computing units. The hardware device may perform operations on different processing elements and / or different parts of the computing units of each processing element scheduled by instructions received from the host. Each computing unit of the hardware computing system is self-contained and can independently perform at least a portion of the calculations required for a given layer of a multi-layer neural network or at least a portion of the obfuscation operations specified by the obfuscated network structure.

[0036] An example machine learning model can be a neural network trained to perform reasoning tasks. The trained neural network includes multiple parameters that define the neural network and the reasoning operations in the neural network. The parameters of the neural network may include the number of network layers in the neural network, the number of nodes in the neural network, the node operations for each node, and / or the node weights of each node. The neural network calculates the reasoning for processing the input by performing the reasoning operations of the neural network. In particular, the layers of the neural network each have multiple nodes including corresponding node operations and weights. The node operations may include linear operations (e.g., multiplication and addition) or nonlinear operations (e.g., activation operations, including ReLu activation functions, Tanh functions, and Sigmoid functions). In some implementations, the parameters further include data that determine the node connections between adjacent network layers (e.g., fully connected layers, convolutional layers, or transposed convolutional layers).

[0037] The hardware device may perform inference computations of the neural network and / or obfuscation operations of an obfuscated network structure associated with the neural network based on instructions received from the host. The compiler on the host may schedule how to distribute or assign these operations to which parts of the computing units of the hardware device when compiling the neural network in "safe mode". In some implementations, one or more hardware components in the edge device may include a dynamic scheduling mechanism so that operations can be scheduled for computing units at runtime.

[0038] An example of a computational unit is a block that each has a computational unit or processing engine and / or one or more caches and switches. An example computational process performed for a neural network layer may include multiplying an input tensor including input activations with a parameter tensor including weights. The computation includes multiplying the input activations with the weights over one or more cycles and performing accumulation of the products over many cycles. The computational results of the network layer may be written to an output bus and stored in a memory. Similarly, obfuscation operations may mimic the above-described inference computations. However, the results from the obfuscation operations are typically not used by the inference computations. In some implementations, the results from the obfuscation operations are discarded or stored in a memory and are never retrieved for downstream computations.

[0039] When performing inference operations of the deployed neural network, the hardware device generates one or more measurable characteristics of the neural network. Different techniques such as side channel attacks can be used to reverse engineer the parameters of the trained neural network. In some cases where the hardware device is included in an edge device such as a smartphone, smart watch, smart tablet computer or other suitable edge device, one can use the hardware device to repeatedly perform test operations and determine the parameters of the neural network.

[0040] There are different reasons why it is desirable to keep neural network parameters secret. One example is from a security perspective. A neural network can be configured to perform face recognition tasks to conveniently unlock a device. If a third party successfully reverse engineers the neural network and finds a way to trick the face unlock mechanism, it would be unsafe to use the face unlock mechanism. For example, a malicious actor could use an adversarial attack to trick the neural network into unlocking a device that the actor does not have authorization for.

[0041] The described technology can solve the above security issues by obfuscating the reasoning operation of the neural network. More specifically, the described technology can issue instructions that include obfuscation operations represented by an obfuscated network structure associated with the neural network when the neural network is compiled in a safe mode. These instructions, when executed on a hardware device, can cause the hardware device to perform both reasoning operations and obfuscation operations of the neural network to mask the original parameters of the neural network. These obfuscation operations can be performed concurrently and / or sequentially with the reasoning operations, depending on the type of obfuscated network structure associated with the neural network. In this way, the hardware device can change at least one of the multiple measurable characteristics of the neural network to effectively hide or mask the measurable characteristics of the actual machine learning operation being performed by the hardware device, and make the deciphering process impossible, impractical, or at least more difficult by analyzing the masked or changed measurable characteristics. The details of performing the obfuscation operation are described below.

[0042] Figure 1 is a block diagram of an example system 100 including an example secure machine learning model compiler 105. Figure 1 As shown, system 100 includes a host 108 and a hardware device 102 communicatively coupled to the host 108, for example, via one or more networks. In general, the host 108 can be communicatively coupled to more than one hardware device, but for simplicity, Figure 1 Only one hardware device 102 is shown.

[0043] The host 108 is configured to receive a machine learning model, compile the received machine learning model using the secure machine learning model compiler 105, transmit instructions or data including the compiled machine learning model to the hardware device 102, and receive data output from the hardware device 102. The secure machine learning model compiler 105 is configured to compile one or more trained machine learning models (e.g., neural networks) from a high-level programming language (e.g., C++, Python, Java) into a machine-readable program (e.g., binary code). The binary code for the machine learning model typically includes at least all parameters that define the trained machine learning model. The binary code specifies, for example, the number of network layers in the compiled neural network, the type of each network layer, the number of nodes in each network layer, the node operations for each node in each network layer, the node weights determined for each node in each network layer, and the inter-layer connectivity, etc. The binary code may include any other parameters of the machine learning model.

[0044] The host 108 also generates instructions that, when executed by the hardware device 102, cause the hardware device 102 to perform the inference operations specified by the binary code. In some cases, the compiler 105 determines to include obfuscation operations in the binary code that, when executed by the hardware device 102, cause one or more computing units or processing elements of the hardware device 102 to perform the obfuscation operations. In general, an obfuscation operation is an operation that is capable of hiding or masking a measurable characteristic of a neural network. In other words, the obfuscation operation may include an operation that causes the hardware device 102 and / or an edge device that includes the hardware device 102 to generate or adjust one or more measurable characteristics. The obfuscation operation may include any suitable machine learning operation and / or other operation that generates or adjusts a measurable characteristic.

[0045] In some implementations, the secure machine learning model compiler 105 is configured to determine whether to compile the machine learning model in standard mode (in which no obfuscation operations are added) or in secure mode (in which obfuscation operations are added) based on characteristics of the machine learning model. Generally speaking, if the secure machine learning model compiler 105 determines to compile the machine learning model in standard mode, the secure machine learning model compiler 105 does not include obfuscation operations in the instructions. On the other hand, if the secure machine learning model compiler 105 determines to compile the machine learning model in secure mode, the secure machine learning model compiler 105 determines one or more obfuscation network structures associated with one or more specific network structures of the machine learning model (e.g., one or more key layers of a neural network) and generates instructions that schedule computing components on the hardware device 102 to perform corresponding inference operations specified by the original machine learning model and obfuscation operations specified by the obfuscated network structures. In combination Figure 2 The components and operations of the secure machine learning model compiler 105 are described in more detail and combined with Figures 3 to 5 The details of determining the obfuscation network structure are described.

[0046] In order to determine whether to use the security mode to compile the machine learning model, the security machine learning model compiler 105 can analyze the properties or characteristics of the machine learning model. For example, if the machine learning model is a giant deep neural network that requires considerable time and cost (e.g., computational cost) to train and the hardware device 102 is located inside the edge device, the security machine learning model compiler 105 can determine to perform operations in a secure mode. In this example, the security machine learning model compiler 105 can select a secure mode based on size (e.g., by comparing the size of the model with a threshold) and / or based on training time (e.g., by comparing the time taken to train the model with a threshold), and such information can be included in metadata transmitted to the security machine learning model compiler 105, for example. As another example, if the machine learning model is used for security-sensitive applications, such as face unlocking, voice unlocking, signature verification, personal information access or prediction using a machine learning model, or other security-sensitive applications, the security machine learning model compiler 105 can determine to perform operations of the machine learning model in a secure mode. In this example, the metadata associated with the machine learning model may include a label or metadata indicating whether the machine learning model is sensitive or which mode (e.g., secure mode or normal mode) to use to perform operations of the machine learning model. Therefore, the secure machine learning model compiler 105 can determine whether to compile the machine learning model in secure mode or standard mode based on the metadata.

[0047] The hardware device 102 may be a hardware processor, such as a hardware accelerator, such as a graphics processing unit (GPU), a visual processing unit (VPU), a tensor processing unit (TPU), or other suitable hardware accelerator. In order to accelerate the execution of reasoning operations, the hardware device 102 includes one or more processing elements 104A to 104N, and each processing element 104A to 104N includes one or more computing units 106A to 106N, which are also referred to as computing units 106 for simplicity. The computing units 106 are each a self-contained unit for performing assigned reasoning operations (e.g., linear or nonlinear operations for nodes in a network layer or across network layers). The number of processing elements 104A to 104N and the corresponding computing units for each processing element may vary based on different computing requirements. For example, the hardware device 102 may include 4, 8, 16 or more processing elements, each having 4, 8, 16 or more computing units. Furthermore, different hardware devices 102 may have different arrangements and interconnections for processing elements 104A- 104N.

[0048] Upon receiving instructions and / or data from the host 108, the hardware device 102 may store the instructions received from the host 108. These instructions may include data representing parameters of the assigned reasoning operations in the neural network in the memory 110 (details of the memory 110 are described below). The parameters of the neural network may include the number of nodes and the number of network layers assigned to the hardware device 102, the node weights, and the corresponding input activations from the previous network layer. If the neural network is compiled in a safe mode, the instructions may further include: parameters specifying an obfuscated network structure, an obfuscated operation associated with the obfuscated network structure, and how the obfuscated network structure is associated with the neural network; input data for performing the obfuscated operation; a memory location for storing calculation results from performing the obfuscated operation; or other parameters. For example, the calculation results from the reasoning operation and / or the calculation results from the obfuscated operation specified by the obfuscated network structure can be stored in the memory 110, and the results from the obfuscated operation are not used for downstream operations.

[0049] In some implementations, the hardware device 102 may include a data bus configured to communicatively couple multiple computing units of the processing element according to a sequence. The data bus may include different types of data buses for transmitting corresponding instructions indicating different operations performed on different computing units (e.g., 106A to 106N in the processing element 104A), input data for performing operations on different computing units, and results for the input data generated on different computing units. For example, the data bus may include a ring bus starting from a controller (not shown) in the hardware device 102, and providing communication coupling by connecting the computing units 106-106N in sequence in the form of a ring back to the bus data path of the controller. In some implementations, the data bus may include a mesh bus that provides a communication path that couples or connects each computing unit to its corresponding adjacent computing unit in both the horizontal dimension and the vertical dimension. The mesh bus can be used to transfer input activations between one or more memory units in adjacent computing units.

[0050] In general, the received instructions on the hardware device 102 are broadcast to the corresponding processing elements 104A to 104N to process the corresponding operations. These instructions generally specify a first portion of the computing unit to perform the inference operation and specify a second portion of the computing unit to perform the obfuscation operation. These instructions may further specify a memory unit for storing the results. For example, these instructions may indicate that the results generated by the second portion of the computing unit for performing the obfuscation operation are discarded or stored in a memory unit (e.g., a memory unit in the computing unit 106) and are not accessed for further calculations. The obfuscation operation may be performed concurrently or sequentially with the inference operation.

[0051] In some implementations, the instructions issued from the host 108 may specify a processing element or a computing unit of a processing element added to the hardware device 102 or to an edge device including the hardware device 102 to specifically perform obfuscation operations. The term "specialized" generally refers to one or more processing elements or computing units that are additionally incorporated into the hardware device (e.g., in addition to other elements or units in the hardware device for performing inference operations) and are configured to substantially only perform obfuscation operations without performing inference operations associated with the deployed machine learning model. The processing elements or computing units that are specifically used to perform obfuscation operations may include, for example, processors, multiplication units, multiplexers, vector reduction units, logic gates, or other suitable processing elements or computing units.

[0052] The measurable characteristics include an electromagnetic profile for one or more reasoning operations, a time profile for performing one or more reasoning operations, or a power consumption profile of a computing unit for performing the reasoning operations. In some implementations, the measurable characteristics may further include a sound profile and / or a temperature profile for performing one or more reasoning operations. When the hardware device is performing the operation of the machine learning model, the electromagnetic profile may, for example, represent a measure of electromagnetic radiation on a capacitor charge on the hardware device. In some implementations, the characteristic profile may be represented by a graph having a horizontal axis representing time and a vertical axis representing a specific characteristic (e.g., electromagnetic radiation, power consumption, sound, or temperature).

[0053] The obfuscated network structure associated with the neural network may generally include obfuscated nodes in a critical network layer, one or more obfuscated network layers immediately before or after the critical network layer, or one or more concurrent obfuscated network layers executed concurrently with the critical network layer. The term "obfuscated network layer immediately before the critical layer" generally refers to the case where the output from the obfuscated network layer is received as input by the critical layer, and the term "obfuscated network layer immediately after the critical layer" generally refers to the case where the input of the obfuscated layer is the output from the critical layer. The obfuscated operations associated with the obfuscated nodes and the concurrent obfuscated network are scheduled by the secure machine learning model compiler 105 to be executed concurrently with the reasoning operations of the associated critical network layer, and the obfuscated operations associated with one or more obfuscated network layers immediately before or after the critical network layer are scheduled to be executed in sequence with the reasoning operations specified by the critical network layer. For example, the obfuscated operations of the obfuscated layer before the critical layer are executed before the critical layer, as if the critical layer receives the output from the previous obfuscated layer. Similarly, the obfuscated operations of the obfuscated layer after the critical layer are executed after the critical layer, as if the output of the critical layer is received as input by the subsequent obfuscated layer.

[0054] In some implementations, the obfuscation operation can mimic the corresponding reasoning operation. As an example, the reasoning operation performed by the nodes in the key layer and the obfuscation operation specified by the obfuscation node in the key layer can be related to the activation function and executed concurrently. As another example, the obfuscation operation of the obfuscation layer before and / or after the key layer can be similar to the obfuscation operation specified by the key layer. Therefore, the combined data profile (e.g., power consumption profile) is different from the data profile used only to perform the reasoning operation, and the measurable characteristics deviate from the true measurable characteristics of the neural network. Therefore, any reverse engineered neural network parameters will be different from the true neural network parameters. For example, the reverse engineered neural network may include obfuscated nodes and obfuscated network layers that are not included in the original neural network.

[0055] In some implementations, the obfuscation operation may be unrelated to (e.g., independent of) the corresponding reasoning operation, but the execution of the obfuscation operation may render any measurable data from the hardware device 102 meaningless. For example, when the reasoning operation is related to matrix reduction, the obfuscation operation may be a specific logical operation or scalar addition or multiplication such that the true measurable profile is changed to lose the pattern or feature, which becomes meaningless for determining the true parameters of the neural network. As another example, the obfuscated network structure may include concurrent obfuscation layers that include obfuscation operations to be executed concurrently with the reasoning operation specified by one or more key layers to render the measurable data from the hardware device 102 meaningless.

[0056] The calculation results of the reasoning operation are stored in the memory 110 and provided to the host 108 for other reasoning operations that rely on the calculation results. However, the calculation results of the obfuscated operation are not used by any of the reasoning operations. In some implementations, the obfuscated results are discarded and not written to any memory. Alternatively, the obfuscated results are written to the memory unit so that other computing components used for the reasoning operation do not receive any data stored in the memory unit. In another example, the obfuscated results are written to the same memory as the reasoning operation, but are not accessed by the reasoning operation.

[0057] Figure 2 An example secure machine learning model compiler 200 is shown. The secure machine learning model compiler may be equivalent to or used to implement Figure 1 Secure machine learning model compiler 105.

[0058] like Figure 2As shown, the secure machine learning model compiler 105 is configured to process input data and generate output data by processing the input data. As described above, the input data may include one or more machine learning models (e.g., neural networks) encoded in a high-level programming language. The output data may include a compiled machine learning model encoded in a machine-readable low-level programming language (e.g., binary code).

[0059] The secure machine learning model compiler 200 may include a security engine 210 configured to determine whether to compile the machine learning model in the input data in a secure mode or in a standard mode, for example based on the characteristics of the machine learning model. The characteristics of the machine learning model may represent the time and cost for training the machine learning model. The time and cost for training the machine learning model may generally be related to the size of the machine learning model, the complexity of the structure for designing the machine learning model, the size of the training examples used to train the machine learning model, the accuracy level set for training the machine learning model, how to train the machine learning (trained alone or end-to-end with additional machine learning models), or other factors. For machine learning models that require high costs, the security engine 210 may determine to compile the machine learning model under a secure model. Otherwise, the machine learning model is compiled by the secure machine learning model compiler 200 in standard mode.

[0060] Alternatively or in addition, the characteristics of the machine learning model may indicate whether the machine learning model is security sensitive. For example, the machine learning model trained for face unlocking is security sensitive, as described above. For security sensitive machine learning models, the security engine 210 may determine to compile these machine learning models in a secure mode. Otherwise, the machine learning model is compiled in a standard mode by the secure machine learning model compiler 200.

[0061] The machine learning model characteristics can be provided to the security engine 210 for analysis in different ways. For example, the machine learning model characteristics can be stored in metadata associated with the machine learning model included in the input data.

[0062] In some implementations, the secure machine learning model compiler 200 may receive instructions on whether to compile the machine learning model in a secure mode. These instructions may be included in instruction data stored in a memory, incorporated into or associated with a program representing the machine learning model, or provided by a user through a user interface. For example, the instructions may include a flag value embedded in a program representing the machine learning model. The flag value may be set interchangeably by the user and cause the secure machine learning model compiler 200 to compile the corresponding machine learning model in standard mode or in secure mode. The flag value may be a binary value, a logical value, a string, a real number, or any other suitable value. For example, a flag value of "0" may cause the compiler 200 to compile the machine learning model in standard mode, and a flag value of "1" may cause the compiler 200 to compile in secure mode. The secure machine learning model compiler 200 further includes a modification engine 220 for determining an obfuscation structure to be associated with the machine learning model. For a machine learning model that the security engine 210 determines to be compiled in standard mode, the secure machine learning model compiler 200 (or the modification engine 220) does not determine the obfuscation network structure and compiles the machine learning model as is. For a machine learning model that the security engine 210 determines to be compiled in a secure mode, the modification engine 220 determines one or more key structures of the machine learning model (e.g., key layers of a neural network) and corresponding obfuscation structures associated with the one or more key structures.

[0063] For example, if the machine learning model is a neural network, the modification engine 220 determines a key layer in the sequence of network layers included in the neural network, and determines a different obfuscation network structure associated with the key layer. The obfuscation network structure specifies multiple obfuscation operations to change the observed characteristics of the neural network, which ultimately protects the neural network parameters from being deciphered by unauthorized third parties. For simplicity, the following description uses "neural network" as an example machine learning model for illustration. It should be noted that in addition to neural networks, machine learning models can also include any other suitable models, and the described techniques can be appropriately applied to different machine learning models.

[0064] The modification engine 220 may use various techniques to determine the critical layers in the neural network. For example, the modification engine 220 may determine whether a layer is a critical layer based on one or more criteria. These criteria may include, for example, (i) whether the layer in the neural network includes one or more node weights that have been updated to exceed a threshold during training, (ii) whether the number of updated node weights in the network layer is higher than a threshold number, (iii) whether the number of node weights in the network layer that have been dropped or determined to be zero is higher than a threshold number, or other suitable criteria. In response to determining that the layer meets one or more of the above criteria, the modification engine 220 may determine such a layer as a critical layer. As a simple example, the trained machine learning model may be a pre-trained neural network after fine-tuning, and the modification engine 220 may determine the layer with the most node weight changes or the layer whose node weight changes meet the threshold as a critical layer. As another example, the modification engine 200 may determine whether the layer is a critical layer based on the input data and / or output data of the layer. The modification engine 200 may compare one or more properties (e.g., data type, data size, or other properties) of the input data and / or output data with corresponding thresholds or a predetermined set of properties. In some cases, modification engine 200 may further compare the data read / write level associated with the input data / output data for the network layer with a threshold value. For example, if the data read / write level associated with the layer satisfies the threshold value, modification engine 200 may determine the layer as a critical layer.

[0065] After determining the critical layer, the modification engine 220 may determine one or more obfuscation network structures for the critical layer. The modification engine 220 may determine the obfuscation network structure based on different obfuscation tasks. For example, the obfuscation network structure may include an obfuscation node having an obfuscation node operation to be added to the critical layer. As another example, the obfuscation network structure may include an obfuscation network layer immediately before and / or after the critical layer. Alternatively or in addition, the obfuscation network structure may include one or more concurrent obfuscation network layers that specify obfuscation operations that are scheduled to be executed concurrently with the reasoning operations of the critical layer. Figures 3 to 5 Describes the details of generating the obfuscated network structure.

[0066] The scheduler 230 is configured to schedule the execution of both reasoning operations and obfuscation operations at the computing units of the hardware device. For example, the scheduler 230 may assign one or more reasoning operations to a first group of computing units of the hardware device, and assign one or more obfuscation operations to a second portion of the computing units of the hardware device. The scheduler 230 may further determine when to perform obfuscation operations and reasoning operations. For example, the scheduler 230 may assign portions of the computing units to perform reasoning operations and obfuscation operations concurrently or sequentially. For the case where the hardware device includes computing units and / or processing elements specifically for performing obfuscation operations, the scheduler 230 may assign obfuscation operations to dedicated computing units and / or processing elements. In this way, the hardware device can maximize the utilization of processing elements for reasoning operations.

[0067] Figure 3 An example process 300 of obfuscating a network structure associated with an input neural network is shown. When a secure machine learning model compiler compiles the input neural network, the obfuscated network structure may be determined by the compiler. The secure machine learning model compiler may be equivalent to Figure 1 secure machine learning model compiler 105 and / or Figure 2 Secure machine learning model compiler 200.

[0068] like Figure 3 As shown, the secure machine learning model compiler included in the described system can determine the key layers of the input neural network. The neural network can include multiple network layers (e.g., network layers 312, 314, 316) arranged according to a sequence. The secure machine learning model compiler can determine whether to compile the neural network in a secure mode, and in response to determining to compile the neural network in a secure mode, the secure machine learning model compiler can determine the key layers from the sequence of the network layers.

[0069] As described above, the secure machine learning model compiler can determine a critical layer based on one or more criteria (e.g., comparing the number of weights changed for the layer to a threshold, or comparing a measure of the change in the value of the node weights for the layer to a threshold). If the network layer meets one or more criteria, the secure machine learning model compiler determines the network layer as a critical layer. For example and as Figure 3 As shown, the secure machine learning model compiler can determine a critical layer 312 in the input neural network. The neural network further includes one or more preceding network layers 314 in the sequence before the critical layer, and one or more subsequent network layers 316 in the sequence after the critical layer. The critical layer 312 can include one or more nodes 306A to 306N each having a node weight value and a corresponding node operation (e.g., a node activation function as described above).

[0070] The obfuscated network structure in the example process 300 includes one or more obfuscated nodes 370A-370N to be associated (eg, add obfuscated nodes) with the key layer 312. The modified key layer 352 represents both the original reasoning operation and the obfuscated operation specified by the added obfuscated nodes 370A-370N.

[0071] The secure machine learning model compiler may incorporate one or more obfuscation nodes 370A to 370N into the key layer in various ways when compiling the original neural network. For example, the secure machine learning model compiler may add an obfuscation node 370A between two adjacent nodes (e.g., 306A and 306B). Alternatively or in addition, the secure machine learning model compiler may add two or more obfuscation nodes (e.g., 370C, 370D, 370E, and 370F) between two adjacent nodes (e.g., 306C and 306D). Each obfuscation node 370A, 370B, ..., or 370N represents an obfuscation node weight and an obfuscation node operation, and further specifies the corresponding obfuscation operation. For example, the obfuscation node may include an obfuscation node weight having a predetermined value (e.g., zero, one, or other value). For example, the obfuscation node weight may be similar to one of two adjacent original nodes. The node operation of the obfuscation nodes 370A to 370N may include any suitable node operation, such as Tanh, Sigmoid, or other node operations, as described above. For example, the obfuscation node operation may be similar to one of two adjacent nodes.

[0072] The obfuscation operation may include an operation on values ​​associated with the obfuscation nodes 370A to 370N. For example, the obfuscation operation may include a suitable operation on one or more obfuscation node weights (e.g., matrix multiplication, tensor reduction, pooling, or other suitable operation), one or more obfuscation node operations, or both. As described above, the input values ​​for the obfuscation operation may be predetermined and stored in one or more memory units in the hardware device. The output values ​​obtained by performing the obfuscation operation are generally not used for downstream reasoning operations. Instead, these output values ​​are discarded or stored in one or more memory addresses that are not accessed for reasoning operations.

[0073] In addition, the obfuscation operations are specified by the obfuscation nodes 370A to 370N in the key layer, and therefore these obfuscation operations are scheduled by the secure machine learning model compiler and transmitted to the hardware device in the form of instruction data to be executed concurrently with the inference operations specified by the original nodes and the key layer.

[0074] Figure 4Another example process 400 of obfuscating a network structure associated with an input neural network is shown. When the secure machine learning model compiler compiles the input neural network, the obfuscated network structure may be determined by the compiler. The secure machine learning model compiler may be equivalent to Figure 1 secure machine learning model compiler 105 and / or Figure 2 Secure machine learning model compiler 200.

[0075] Similarly, the secure machine learning model compiler included in the described system can determine the key layer 412 of the input neural network for compilation from the sequence of network layers (414, 412, and 416), such as Figure 4 One or more network layers 414 are preceding neural network layers in the sequence before the key layer 412, and one or more network layers 416 are subsequent network layers 316 in the sequence after the key layer 412. The key layer 412 may include one or more nodes 406A to 406N each having a node weight value and a corresponding node operation (e.g., a node activation function as described above).

[0076] The obfuscated network structure of the example process 400 includes one or more obfuscated network layers 464 and / or 462. The secure machine learning model compiler is configured to modify the input neural network by adding one or more obfuscated network layers 464 and / or 462 before and / or after the key layer 412 according to a sequence. For example, one or more obfuscated network layers 464 are added before the key layer 412 and after the previous network layer 414. Alternatively or in addition, another obfuscated network layer 462 is further added immediately after the key layer 412 and before the subsequent network layer 416. It should be noted that the obfuscated layers may include any suitable number of obfuscated layers before and / or after the key layer 412, such as one, two, three, five, ten, or other suitable number.

[0077] Each obfuscated network layer may include one or more obfuscated nodes. Each obfuscated node in the obfuscated network layer may include a obfuscated node weight value and a obfuscated node operation. Each obfuscated network layer may include one or more obfuscated operations associated with the obfuscated layer. For example, the obfuscated operation may include one or more inter-layer matrix operations (e.g., the layer output generated from the previous layer multiplied by the obfuscated node weight of the obfuscated layer). As another example, the obfuscated operation may include one or more pooling operations of the layer output of the previous layer. In some implementations, the layer operations and node operations in the obfuscated layer may be similar to the layer operations and node operations specified by the key layer. Alternatively, the obfuscated operation specified by the obfuscated layer may be a different type from the obfuscated operation in the key layer. For example, the obfuscated operation may include a matrix multiplication, but the key layer includes a pooling or softmax operation, or vice versa.

[0078] As described above, the input values ​​for the obfuscation operations specified by the obfuscation layer can be predetermined and stored in one or more memory units in the hardware device. The output values ​​obtained by performing the obfuscation operations specified by the obfuscation layer are generally not used for downstream reasoning operations. Instead, these output values ​​are discarded or stored in one or more memory addresses that are not accessed for reasoning operations.

[0079] In addition, the obfuscation operations specified by the obfuscation layer are scheduled by the secure machine learning model compiler to be executed according to the sequence of network layers in the modified neural network. Although the output of the obfuscated network layer is generally not used for the reasoning operations specified by the subsequent network layer (e.g., the key layer), the reasoning operations specified by the original network layer and the obfuscation operations specified by the obfuscation layer are still executed in sequence according to the layer sequence, as if the original network layer after the obfuscation layer receives the output from the obfuscation layer. Therefore, this makes it more difficult for an unauthorized third party to determine which layer in the input neural network is an obfuscated layer or a real layer based on the above-mentioned data profile observed from the hardware device.

[0080] Figure 5 Another example process 500 for obfuscating a network structure associated with an input neural network is shown. When a secure machine learning model compiler compiles the input neural network, the obfuscated network structure may be determined by the compiler. The secure machine learning model compiler may be equivalent to Figure 1 secure machine learning model compiler 105 and / or Figure 2 Secure machine learning model compiler 200.

[0081] Similar to the above combination Figure 4 As described, the obfuscated network structure of the example process 500 also includes an obfuscated network layer associated with the key layer. However, the secure machine learning model compiler does not use the obfuscated network layer to change the structure (or parameters) of the input neural network. Instead, the secure machine learning model compiler determines one or more obfuscated layers that specify obfuscated operations to be performed concurrently with the reasoning operations specified by the key layer. In the following description, these obfuscated layers are also referred to as concurrent obfuscated layers.

[0082] More specifically, the secure machine learning model compiler determines one or more concurrent obfuscation layers 572 to be associated with the key layer 512 in the sequence of network layers 512, 514, and 516 of the input neural network when compiling the input neural network.

[0083] One or more concurrent obfuscation network layers 572 each have one or more obfuscation nodes 574. For example, a first concurrent obfuscation network layer may include a plurality of obfuscation nodes 580A to 580N, and a second concurrent obfuscation network layer may include a plurality of obfuscation nodes 585A to 585N. As described above, each obfuscation node in the obfuscation network layer may include an obfuscation node weight value and an obfuscation node operation. Each concurrent obfuscation network layer may include one or more obfuscation operations associated with the concurrent obfuscation layer. For example, the obfuscation operation may include one or more inter-layer matrix operations (e.g., a layer output generated from a previous layer multiplied by the obfuscation node weight of the obfuscation layer). As another example, the obfuscation operation may include one or more pooling operations of the layer output of the previous layer. In some implementations, the layer operations and node operations in the concurrent obfuscation layer may be similar to the layer operations and node operations specified by the key layer. Alternatively, the obfuscation operation specified by the concurrent obfuscation layer may be a different type from the obfuscation operation in the key layer. For example, the obfuscation operation may include pooling and / or softmax operations, but the key layer includes matrix multiplication, and vice versa.

[0084] As described above, the input values ​​for the obfuscation operations specified by the concurrent obfuscation layer can be predetermined and stored in one or more memory units in the hardware device. The output values ​​obtained by executing the obfuscation operations specified by the concurrent obfuscation layer are generally not used for downstream reasoning operations. Instead, these output values ​​are discarded or stored in one or more memory addresses that are not accessed for reasoning operations.

[0085] In addition, the obfuscation operations specified by the concurrent obfuscation layer are scheduled by the secure machine learning model compiler to be executed concurrently with the inference operations specified by the corresponding key network layer. In this way, the observed profile of one or more characteristics of the neural network can be changed, which further prevents the deciphering of one or more parameters of the neural network (e.g., parameters of one or more key layers of the neural network) or at least increases the time and cost of the deciphering.

[0086] Although for simplicity of explanation, Figures 3 to 5 Only one critical layer is shown in FIG, but it should be noted that the secure machine learning model compiler is configured to determine more than one critical layer in the neural network. Figure 3 to Figure 4 Each of the figures in FIG. 1 shows only one type of obfuscated network structure, but it should be understood that any one or more of the obfuscated network structures may be combined in various ways for key network layers.

[0087] Figure 6is an example flow chart of a process 600 for compiling an input neural network with a confusion network structure. For convenience, process 600 is described as being performed by a system of one or more computers located in one or more locations. For example, process 600 may be performed on a host or by a compiler located in a host (e.g., Figure 1 The secure machine learning model compiler 105 shown in Figure 2 The secure machine learning model compiler 200 shown in FIG. 6 is executed. The order of the steps in process 600 is illustrative only and may be performed in a different order. In some implementations, process 600 may include additional steps, fewer steps, or some of the steps may be divided into multiple steps.

[0088] The system receives data representing a machine learning model (610). The machine learning model may include various types of machine learning models, such as a neural network. The data representing the neural network may specify a plurality of inference operations. The neural network may include parameters specifying a sequence of a plurality of network layers, a plurality of nodes in each of the plurality of layers, node weights and node operations for each node, and other structures associated with the neural network. In some implementations, the data may further include metadata indicating additional information related to the machine learning model. For example, the metadata may indicate whether the machine learning model is security sensitive and / or the time and / or cost for training the machine learning model. For simplicity, the following description is based on a neural network received by the system.

[0089] The system typically compiles the neural network to generate instructions that, when executed, cause one or more computing units of the hardware device to perform inference operations of the neural network and, in some cases, also perform obfuscation operations associated with the neural network.

[0090] The obfuscation operation, when executed, may obfuscate one or more measurable characteristics of the neural network. More specifically, the obfuscation operation may at least change one or more measurable characteristics of the neural network. For example, the obfuscation operation, when executed, is configured to obfuscate at least one of the number of network layers of the neural network, the number of nodes in a network layer of the neural network, node operations for nodes in a network layer of the neural network, or weight values ​​associated with nodes in a network layer of the neural network, as described above.

[0091] The measurable characteristic may include at least one of a power profile, an electromagnetic profile, or a time profile. Once the measurable characteristic is changed, it becomes more difficult to determine the parameters of the neural network based on the measurable characteristic. The obfuscation operation may include operations similar to the reasoning operation, such as activation operations, tensor multiplications, and reductions. For example, the obfuscation operation may include an obfuscation node operation specified by an obfuscation node in a network layer that is executed concurrently with other node operations in the network layer. The obfuscation node operation may be a node addition or a node multiplication. Alternatively, the obfuscation node operation of the obfuscation node may specify an activation function for a specific node that is different from the actual activation function of the actual node in the same network layer that is executed concurrently with the obfuscation node operation. In some implementations, the obfuscation operation may include operations that are unrelated to and / or different from the reasoning operation. The obfuscation operation may be executed concurrently with the corresponding reasoning operation, or sequentially according to an order. More details related to the obfuscation operation are described above.

[0092] To compile the neural network, the system determines whether to obfuscate one or more measurable characteristics of the neural network (620). As described above, the system determines whether to obfuscate one or more measurable characteristics of the neural network based on characteristics or properties of the neural network. Based on input data associated with the neural network, the system determines whether the neural network is security sensitive and / or has required a significant amount of time and / or cost to train to meet a particular threshold. In response to determining that the neural network is security sensitive and / or has required a significant amount of resources, the system determines to compile the neural network in a "safe mode", as described above. In the "safe mode", the system can obfuscate the one or more measurable characteristics of the neural network by determining that the instructions include one or more obfuscation operations in addition to the reasoning operations specified by the neural network when compiling the neural network, and the one or more obfuscation operations can obfuscate the one or more measurable characteristics of the neural network when executed together with the reasoning operations. In some implementations, step 620 is optional. For example, the system or the host can receive a user instruction or input data instructing the host to compile the input neural network in a "safe mode" without making the determination.

[0093] The system determines critical layers of a sequence of network layers (630). The critical layers are determined based on characteristics of the neural network. For example, the system may determine the critical layers based on layer type (e.g., pooling layer, fully connected layer, softmax layer). As another example, the system may determine the critical layers based on updates to parameters of the network layers during training. More specifically, the system may compare parameter updates (e.g., the number of updates to nodes in the network layer, changes in values ​​of nodes in the network layer, the number of dropped nodes in the network layer, or changes in inter-layer connectivity between the network layer and adjacent layers) to one or more criteria (e.g., threshold numbers or values). Alternatively, the system may receive metadata associated with the neural network indicating one or more critical layers. A description of determining critical layers is described above in detail.

[0094] The system determines an obfuscated network structure to be associated with the key layer (640). As described above, the system determines various obfuscated network structures to be associated with the key layer. For example, the obfuscated network structure may include obfuscated nodes that specify corresponding obfuscated node weights and obfuscated node operations. The system may modify the neural network by adding one or more obfuscated nodes in the key layer. The obfuscated operations specified by the obfuscated nodes added to the key layer are generally executed concurrently with the reasoning operations specified by the original nodes in the key layer. As another example, the obfuscated network structure may include one or more obfuscated network layers. The system may modify the neural network by inserting one or more obfuscated network layers (immediately) before and / or after the key layer in a sequence of network layers. Each obfuscated network layer includes a plurality of obfuscated nodes similar to the obfuscated nodes described above. The obfuscated operations specified by the obfuscated network layers are generally executed according to a sequence. For example, the sequence is determined based on the sequence of modified network layers in the modified network. Alternatively or in addition, the obfuscated network structure includes one or more concurrent obfuscated network layers. Unlike the obfuscated network layers inserted in the original sequence of network layers of the neural network, the system does not use concurrent obfuscated layers to modify the neural network. Instead, the system keeps the neural network structure unchanged and generates instructions to cause the hardware device to perform the obfuscation operation of the concurrent obfuscation layer concurrently with one or more corresponding key network layers.

[0095] The system compiles the neural network with the associated obfuscation network structure (650). During compilation, the system generates instructions for performing obfuscation operations specified by the obfuscation network structure and reasoning operations specified by the compiled neural network. In some implementations, the obfuscation operations are scheduled to be performed concurrently with the reasoning operations, such as the obfuscation operations specified by the obfuscation network structure, such as obfuscation nodes and concurrent obfuscation network layers. Alternatively or in addition, the obfuscation operations and reasoning operations are scheduled to be performed sequentially according to an order, such as the obfuscation operations specified by the obfuscation network structure, such as the obfuscation network layers.

[0096] In some implementations, the system further determines a schedule in the instructions for one or more corresponding hardware devices. The schedule generally specifies a sequence of causing the hardware device to perform obfuscation operations specified by the corresponding obfuscation network structure and reasoning operations specified by the neural network. For each of the one or more hardware devices, the schedule further instructs different portions of a set of computing units of the hardware device to perform corresponding reasoning operations and / or obfuscation operations.

[0097] In some implementations, the system may transmit data including the above instructions to one or more hardware devices. Example hardware devices may be edge devices such as smartphones, smart watches, smart tablets, or laptops, to name a few. In some implementations, the system may further generate data to trigger instructions to be executed by the one or more hardware devices, which causes the inference operations specified by the neural network and the obfuscation operations specified by the obfuscation network structure to be performed at the corresponding hardware devices (660).

[0098] This specification uses the term "configured to" in conjunction with systems, devices, and computer program components. A system of one or more computers is configured to perform specific operations or actions means that the system has installed thereon software, firmware, hardware, or a combination thereof that, when operated, causes the system to perform those operations or actions. One or more computer programs are configured to perform specific operations or actions means that the one or more programs include instructions that, when executed by a data processing device, cause the device to perform those operations or actions. A dedicated logic circuit system is configured to perform specific operations or actions means that the circuit system has electronic logic that performs those operations or actions.

[0099] The term "data processing equipment" refers to data processing hardware, and includes all kinds of equipment, devices and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. The equipment may also be or further include a dedicated logic circuit system, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the equipment may also optionally include code that creates an execution environment for a computer program, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0100] A computer program (which may also be referred to or described as a program, software, software application, module, software module, script or code) may be written in any form of programming language including compiled or interpreted languages ​​or declarative or procedural languages, and it may be deployed in any form, including as a standalone program or as a module, component, subroutine or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines or code portions). A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0101] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and the device can also be implemented as, a special purpose logic circuit system, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0102] Computers suitable for executing computer programs include, for example, general or special purpose microprocessors or both, or any other kind of central processing unit. Typically, the central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more large-capacity storage devices for storing data, such as a disk, a magneto-optical disk, or an optical disk, or be operatively coupled to receive data from it or transfer data to it or both. However, a computer does not necessarily have such a device. In addition, a computer can be embedded in another device, such as a mobile phone, a smart phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name a few.

[0103] Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0104] Embodiments of the subject matter described in this specification may be implemented in a computing system that includes a back-end component, such as a data server, or includes a middleware component, such as an application server, or includes a front-end component, such as a client computer with a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0105] The computing system may include a client and a server. The client and the server are usually remote from each other and typically interact through a communication network. The relationship between the client and the server is generated by a computer program that runs on a corresponding computer and has a client-server relationship with each other. In some embodiments, the server transmits data (e.g., a hypertext markup language (HTML) page) to a user device, for example, to display data to a user interacting with a user device acting as a client and to receive user input therefrom. Data generated at the user device, such as the result of a user interaction, can be received at the server from the user device.

[0106] In addition to the above embodiments, the following embodiments are also innovative:

[0107] Embodiment 1 is a method comprising: receiving data representing a neural network including an inference operation, wherein the neural network includes parameters specifying a sequence of network layers and a plurality of nodes in each layer of the sequence of network layers; compiling the neural network to generate instructions, which when executed cause one or more computing units of a hardware device to perform an obfuscation operation associated with the neural network and the inference operation of the neural network, wherein the obfuscation operation obfuscates one or more measurable characteristics of the neural network when executed, the compiling comprising: determining a key layer of the sequence of network layers; determining an obfuscation network structure for associating with the key layer; and compiling the neural network together with the associated obfuscation network structure to generate instructions for performing the obfuscation operation specified by the obfuscation network structure; and causing the inference operation and the obfuscation operation to be performed at the hardware device.

[0108] Embodiment 2 is the method of embodiment 1, wherein the compiling further comprises determining the one or more measurable characteristics that confuse the neural network.

[0109] Embodiment 3 is a method as described in embodiment 1 or 2, wherein causing the inference operation and the obfuscation operation to be performed at the hardware device includes causing the hardware device to perform the inference operation and the obfuscation operation concurrently.

[0110] Embodiment 4 is the method of any one of embodiments 1 to 3, wherein causing the inference operation and the obfuscation operation to be performed at the hardware device comprises causing the hardware device to perform the inference operation and the obfuscation operation sequentially.

[0111] Embodiment 5 is a method as described in any one of embodiments 1 to 4, wherein the obfuscation operation is configured to obfuscate at least one of the following: the number of network layers of the neural network, the number of nodes in the network layers of the neural network, the node operations used for the nodes in the network layers of the neural network, or the weight values ​​associated with the nodes in the network layers of the neural network.

[0112] Embodiment 6 is the method of any one of embodiments 1 to 5, wherein the one or more measurable characteristics of the neural network include at least one of a power profile, an electromagnetic profile, or a time profile.

[0113] Embodiment 7 is a method as described in any one of Embodiments 1 to 6, wherein determining the key layer of the sequence of network layers includes determining the key layer based on at least one of the type of the network layer, an update to a parameter of the network layer, or metadata associated with the network layer.

[0114] Embodiment 8 is a method as described in any one of embodiments 1 to 7, wherein determining the obfuscated network structure to be associated with the key layer includes adding an obfuscated network layer immediately before and / or after the key layer in the sequence of network layers.

[0115] Embodiment 9 is a method as described in any one of embodiments 1 to 8, wherein determining the obfuscation network structure to be associated with the key layer includes determining an obfuscation network layer having an obfuscation operation to be executed concurrently with the reasoning operation of the key layer.

[0116] Embodiment 10 is a method as described in any one of embodiments 1 to 9, wherein determining the obfuscated network structure to be associated with the key layer includes adding an obfuscated node to a set of nodes in the key layer, wherein the obfuscated node includes an obfuscated operation to be performed concurrently with the reasoning operation of the key layer.

[0117] Embodiment 11 is the method as described in any one of embodiments 1 to 10, wherein the specific hardware device comprises an edge device.

[0118] Embodiment 12 is a method as described in any one of embodiments 1 to 11, wherein compiling the neural network to generate instructions includes: determining a timing for the specific hardware device in the instructions, wherein the timing specifies a sequence for performing the obfuscation operation and the reasoning operation on a corresponding group of computing units of the specific hardware device.

[0119] Embodiment 13 is a system comprising one or more computers and one or more storage devices storing instructions, which, when executed by one or more computers, cause the one or more computers to perform corresponding operations, wherein the operations include the method described in any one of Embodiments 1 to 12.

[0120] Embodiment 14 is one or more computer-readable storage media storing instructions, which, when executed by one or more computers, cause the one or more computers to perform corresponding operations, wherein the corresponding operations include the method as described in any one of embodiments 1 to 12.

[0121] Although this specification contains many specific implementation details, these details should not be interpreted as limitations on the scope of the content that can be claimed, but should be interpreted as descriptions of features that may be peculiar to a particular embodiment. Certain features described in the context of a separate embodiment in this specification may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. In addition, although features may be described above as acting in certain combinations and even initially claimed as such, in some cases, one or more features from the claimed combination may be deleted from the combination, and the claimed combination may involve a sub-combination or a variation of the sub-combination.

[0122] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in a continuous order, or that all of the operations shown be performed, to achieve the desired result. In some contexts, multitasking and parallel processing can be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0123] In each instance where an HTML file is mentioned, other file types or formats may be substituted. For example, an HTML file may be replaced by XML, JSON, plain text, or other types of files. In addition, where a table or hash table is mentioned, other data structures (such as a spreadsheet, a relational database, or a structured file) may also be used.

[0124] Specific embodiments of the present invention have been described. Other embodiments are within the scope of the appended claims. For example, the steps set forth in the claims, described in the specification, or depicted in the drawings can be performed in a different order and still achieve the desired results. In some cases, multitasking and parallel processing can be advantageous.

Claims

1. A method comprising: receiving data representing a neural network including an inference operation, wherein the neural network includes parameters specifying a sequence of network layers and a plurality of nodes in each layer of the sequence of network layers; compiling the neural network to generate instructions that, when executed, cause one or more computing units of a hardware device to perform an obfuscation operation associated with the neural network and the inference operation of the neural network, wherein the obfuscation operation, when executed, obfuscates one or more measurable characteristics of the neural network, the compiling comprising: determining a target layer for said sequence of network layers; determining an obfuscation network structure for associating with the target layer; and compiling the neural network together with the associated obfuscation network structure to generate instructions for performing the obfuscation operation specified by the obfuscation network structure; and The inference operation and the obfuscation operation are caused to be performed at a hardware device.

2. The method of claim 1, wherein: The compiling further includes determining the one or more measurable characteristics that confound the neural network.

3. The method according to claim 1 or 2, wherein: Causing execution of the inference operation and the obfuscation operation at the hardware device includes causing the hardware device to concurrently execute the inference operation and the obfuscation operation.

4. The method according to any one of claims 1 to 3, wherein: Causing execution of the inference operation and the obfuscation operation at the hardware device includes causing the hardware device to execute the inference operation and the obfuscation operation sequentially.

5. The method according to any one of claims 1 to 4, wherein: The obfuscation operation is configured to obfuscate at least one of: the number of network layers of the neural network, the number of nodes in a network layer of the neural network, node operations for nodes in a network layer of the neural network, or weight values ​​associated with nodes in a network layer of the neural network.

6. The method according to any one of claims 1 to 5, wherein: The one or more measurable characteristics of the neural network include at least one of a power profile, an electromagnetic profile, or a time profile.

7. The method according to any one of claims 1 to 6, wherein: Determining a target layer for the sequence of network layers includes determining the target layer based on at least one of a type of network layer, an update to a parameter of a network layer, or metadata associated with the network layer.

8. The method according to any one of claims 1 to 7, wherein: Determining the obfuscation network structure to be associated with the target layer includes adding an obfuscation network layer immediately before and / or after the target layer in the sequence of network layers.

9. The method according to any one of claims 1 to 8, wherein: Determining the obfuscation network structure to be associated with the target layer includes determining an obfuscation network layer having an obfuscation operation to be performed concurrently with the inference operation of the target layer.

10. The method according to any one of claims 1 to 9, wherein: Determining the obfuscation network structure to be associated with the target layer includes adding an obfuscation node to a set of nodes in the target layer, wherein the obfuscation node includes an obfuscation operation to be performed concurrently with the inference operation of the target layer.

11. The method according to any one of claims 1 to 10, wherein: The specific hardware device comprises an edge device.

12. The method according to any one of claims 1 to 11, wherein: Compiling the neural network to generate instructions includes: A timing in the instruction for the specific hardware device is determined, wherein the timing specifies a sequence for performing the obfuscation operation and the inference operation on a corresponding set of computational units of the specific hardware device.

13. A system comprising one or more computers and one or more storage devices storing instructions, wherein the instructions, when executed by the one or more computers, cause the one or more computers to perform corresponding operations, the operations comprising the method according to any one of claims 1 to 12.

14. One or more computer-readable storage media storing instructions which, when executed by one or more computers, cause the one or more computers to perform corresponding operations, the corresponding operations comprising the method of any one of claims 1 to 12.

Images