On-chip integrity verification of execution pipeline for machine-learning models

Abstract

Methods and systems, including computer-readable media, are described for on-chip integrity verification of an execution pipeline for a machine-learning (ML) model. An integrated circuit stores a golden hash in a protected register of the circuit in response to receiving the golden hash from a source. The circuit performs operations to compute an inference from an input sample by executing a model binary for the ML model at the integrated circuit. The circuit also generates a runtime hash based on an instruction sequence derived from execution of the model binary at the circuit. The circuit determines whether the runtime hash is consistent with the golden hash based on a comparison of respective values of the runtime hash and the golden hash. The circuit generates an output of a verification check on the execution pipeline. The output is generated based on determinations about consistency between the runtime hash and the golden hash.

Description

ON-CHIP INTEGRITY VERIFICATION OF EXECUTION PIPELINE FOR MACHINE-LEARNING MODELSBACKGROUND

[0001] This specification generally relates to machine-learning models.

[0002] In the context of machine-learning, neural networks can generate an output from a received input in accordance with current values of a respective set of parameters. Some neural networks can be convolutional neural networks (CNNs) configured for image processing or recurrent neural networks (RNNs) configured for speech and language processing. Other neural networks can have a transformer network architecture that is based on attention mechanisms, which could augment recurrent and convolutional networks.

[0003] Different types of neural network architectures can be used to perform a variety of tasks related to classification or pattern recognition, predictions that involve data modeling, and information clustering. A machine-learning (ML) model that is based on a trained neural network can be stored and implemented locally to compute inferences on an edge, mobile, or client computing device. Implementing ML models locally on an edge device presents various security concerns and challenges. For example, the model binary and sensitive data processed by the model can be leaked and accessed by unauthorized users.

[0004] Inference and self-learning capabilities of local ML models can be abused by adversaries that gain unauthorized access to the mobile / edge device storing the models. The abuse can be exacerbated when compensating controls and other safety provisions are bypassed, allowing an attacker to exploit entity-specific proprietary data models. For example, when security controls are compromised or entirely absent, attackers can exploit a model’s inference capabilities / functions to perpetuate illegal and harmful activities.SUMMARY

[0005] This specification describes techniques for a secure verification framework that reliably limits or precludes a user’s ability to leverage a machine-learning (“ML”) model to generate abusive or malicious content. More specifically, the disclosed techniques are used to assess and / or verify the overall integrity of a target ML model’s runtime execution pipeline based on a measure of consistency between a respective set of hashes. For example, the set of hashes includes one or more golden security hashes that are calculated during a compile time using a compiler that generates a model binary for the target ML model. The set of hashes also includes one or more runtime security hashes that are generated during runtimewhen the model binary is executed on a special-purpose integrated circuit, such as a neural network processor or hardware accelerator.

[0006] One aspect of the subject matter described in this specification can be embodied in a computer-implemented method for secure execution of the target ML model on an integrated circuit. The method includes storing a golden security hash in a protected register of the integrated circuit in response to receiving the golden security hash from a source and performing, using the target ML model, operations to compute an inference from a received input sample in response to executing a model binary for the target ML model at the integrated circuit.

[0007] Additionally, the method includes, concurrent with performing the operations, generating a runtime security hash based on an instruction sequence derived from execution of the model binary at the integrated circuit, determining whether the runtime security hash is consistent with the golden security hash based on a comparison of respective values of the runtime security hash and the golden security hash, and generating an output of a verification check on an execution pipeline comprising the operations to compute the inference. The output is generated based on the determination about consistency between the run time security hash and the golden security hash.

[0008] In some implementations, the method includes determining that the runtime security hash and the golden security hash are inconsistent, generating an output of the verification check that indicates a verification failure based on the inconsistency, and terminating operations to compute the inference from the received input sample based on the verification failure.

[0009] In some implementations, the method includes determining that the runtime security hash and the golden security hash are consistent, computing the inference from the received input sampled based on the determination, and generating an output representing the computed inference.

[0010] In some implementations, the execution pipeline includes one or more stages, and each stage corresponds to one or more of the operations for computing the inference and for generating an output representing the computed inference. In particular, for each stage of the one or more stages, the method further includes generating a runtime security hash for the stage, performing a verification check on the stage using the runtime security hash for the stage, and generating a corresponding output of the verification check on the stage.

[0011] In some examples, generating the runtime security hash can include generating the runtime security hash based on a hash calculation performed during runtime using aninstruction sequence that corresponds to one or more nodes in a compute graph for the target ML model. In some examples, the integrated circuit includes multiple instruction streams, and generating the runtime security hash is based on an instruction sequence derived from one or more of the multiple instruction streams.

[0012] In some implementations, the runtime security hash is generated based on an instruction stream of an instruction vector engine of the integrated circuit. In this case, the runtime security hash can be calculated by the instruction vector engine.

[0013] In some implementations, the integrated circuit is a tensor processing unit (“TPU”) that includes multiple compute tiles, and generating a runtime security hash includes generating a runtime security hash based on instructions derived from an instruction set architecture (ISA) of the TPU and for the multiple compute tiles.

[0014] In some implementations, the instructions derived from the ISA are staged in a software-managed queue, and generating a runtime security hash includes reading, from the software-managed queue, the instructions derived from the ISA, and calculating a rolling runtime security hash using each ISA instruction read from the software-managed queue. In this case, the rolling runtime hash can be calculated by an instruction vector engine of the TPU during runtime. In some implementations, receiving the golden security hash includes receiving the golden security hash from a source that is external to the TPU and that communicates data to a processor of the SoC.

[0015] In some implementations, the source is a cloud-based compiler that is external to the integrated circuit. The method can further include receiving, at the integrated circuit, a model file from the source, where the model file is for the target ML model and includes the model binary and metadata including type information that indicates a level of assurance verification assigned to the target ML model. In some implementations, the integrated circuit is a TPU, and receiving the golden security hash includes receiving the golden security hash from a source that is external to the TPU. In this case, receiving the golden security hash includes receiving the golden security hash from a processor of the SoC via a secure channel of the SoC.

[0016] In some implementations, the method further includes generating the golden security hash based on a first hash calculation performed during compile time when source code for the target ML model is compiled to generate the model binary for the target ML model. In some cases, generating the golden security hash can include generating a first golden security hash that represents an input filter that is used as a first verification check to verify integrity of an input sample to be processed by the target ML model. In some cases,generating the second golden security hash can include generating the second golden security hash that represents an output filter that is used as a second verification check to verify integrity of an inference output generated by the target ML model. In some cases, the first hash calculation performed over compile time is performed over a compute graph denved from a cloud-based compiler.

[0017] The system for secure execution of the target ML model on an integrated circuit includes an integrated circuit, a processing device, and a non-transitory machine-readable storage device storing instructions that are executable by the processing device to cause performance of operations including storing a golden security hash in a protected register of the integrated circuit in response to receiving the golden security hash from a source and performing, using the target ML model, operations to compute an inference from a received input sample in response to executing a model binary' for the target ML model at the integrated circuit.

[0018] Additionally, the method includes, concurrent with performing the operations, generating a runtime security hash based on an instruction sequence derived from execution of the model binary at the integrated circuit, determining whether the runtime security hash is consistent with the golden security hash based on a comparison of respective values of the runtime security hash and the golden security hash, and generating an output of a verification check on an execution pipeline comprising the operations to compute the inference. The output is generated based on the determination about consistency between the run time security hash and the golden security hash.

[0019] The non-transitory machine-readable storage device storing instructions for secure execution of the target ML model implemented on an integrated circuit, the instructions being executable by a processing device to cause performance of operations including storing a golden security hash in a protected register of the integrated circuit in response to receiving the golden security hash from a source and performing, using the target ML model, operations to compute an inference from a received input sample in response to executing a model binary for the target ML model at the integrated circuit.

[0020] Additionally, the method includes, concurrent with performing the operations, generating a runtime security hash based on an instruction sequence derived from execution of the model binary at the integrated circuit, determining whether the runtime security hash is consistent with the golden security hash based on a comparison of respective values of the runtime security hash and the golden security hash, and generating an output of a verification check on an execution pipeline comprising the operations to compute the inference. Theoutput is generated based on the determination about consistency between the run time security hash and the golden security hash.

[0021] Other implementations of this and other aspects include corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices. A system of one or more computers can be so configured by virtue of software, firmware, hardware, or a combination of them installed on the system that in operation causes the system to perform the actions. One or more computer programs can be so configured by virtue of having instructions that, when executed by a data processing apparatus, cause the apparatus to perform the actions.

[0022] The subject matter described in this specification can be implemented in particular embodiments to realize one or more of the following advantages.

[0023] Current systems may implement safety filters as part of execution of a ML model to control the inputs to, and outputs generated by, the model. The mechanisms used by current systems are ineffective at safeguarding against misuse of ML models to generate abusive content. For example, current systems can implement watermarking or privacypreserving filters as mechanisms to limit a model’s ability to generate malicious outputs. However, these mechanisms do not adequately prevent malicious content from being generated by a model during runtime. In particular, current systems cannot preserve the integrity and security of an ML model during runtime.

[0024] To address the above challenges, techniques are described for a secure venfication framework that reliably restricts a ML model’s ability to generate abusive or malicious content. The disclosed techniques use a respective set of compile time and runtime hashes to assess and enforce secure safeguards of a target ML model. The techniques leverage a protected set of hardware registers to store the respective set of hashes. The system implements efficient comparisons of secure hash values to enforce input and output safety filters. The safety filters can function as integrity mechanisms that control the passing of inputs to, and outputs from, an ML model.

[0025] For example, an input safety filter can be an ML model that permits the passing of input to an ML model only when safety criteria of the input safety filter are satisfied. In some cases, the criteria are satisfied based on a determined consistency between certain hash values in the respective set of compile time and runtime hashes. To enforce security requirements, the system can limit or preclude a model’s ability' to generate abusive content by, for example, terminating operations of an ML workload based on a determined inconsistency between a golden security hash and a runtime security hash.

[0026] The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Fig. 1 is a block diagram of an example computing system for performing integrity verification of a pipeline for executing a ML model.

[0028] Fig. 2 shows an example of a system-level runtime operation for implementing integrity verification of an execution pipeline.

[0029] Fig. 3 shows an example compiler for generating a model file and corresponding hashes for implementing pipeline integrity verification.

[0030] Fig. 4 shows an example compiler operation for inserting one or more hash operations in an instruction stream.

[0031] Fig. 5 is an example process for integrity verification of an execution pipeline for ML models.

[0032] Like reference numbers and designations in the various drawings indicate like elements.DETAILED DESCRIPTION

[0033] Fig. 1 is a block diagram of an example computing system 100 for performing integrity verification of a pipeline either prior to, or concurrent with, executing operations relating to ML models, including neural network models. The system 100 includes a system- on-chip 102 (“SoC 102”) and a compiler 120.

[0001] The SoC 102 includes a central processing unit 104 (“CPU 104”), a shared memory 106 (“memory 106”), a security resource manager 108, and an IP / circuit block 110. In some implementations, system 100 can include multiple SoCs and descriptions for the SoC 102 will apply equally to each of the multiple SoCs that may be included at system 100. The CPU 104 can be a general -purpose CPU (e.g., a single or multi-core CPU).

[0002] The CPU 104 generates one or more indicators, such as an app-launch indicator or a function call that is triggered in response to executing or launching an application at a user device. For example, the application can be a camera application that uses an imaging sensor to generate image data or a gaming application that requires substantial memory and graphics processing resources to render graphical content of the game. The CPU 104 also generatesone or more application values, such as pixel values or frame rate. The application values may be associated with a function call, may be descriptive of an event that occurs during execution of the application, or both.

[0034] The memory 106 is a system memory, shared memory, or both. In the example of Fig. 1, memory 106 is depicted external to circuit block 110. However, memory 106 can include portions of memory that are: i) specific to circuit block 110, ii) external to circuit block 110, or iii) both. The memory 106 can be random access memory of the SoC 102, such as static random-access memory (SRAM), dynamic random-access memory (DRAM), a synchronous DRAM (SDRAM), or double data rate (DDR) SDRAM. In some implementations, aspects of memory 106 are configured as a shared scratchpad memory that supports parallel access of its memory resources by two or more processors of the circuit 110. Memory 106 can also include various other types of memory, such as high bandwidth memory (HBM), narrow memory (e.g., for storing 8-bit values), wide memory (e.g., for storing 16-bit or 32-bit values), etc.

[0035] The security resource manager 108 can be implemented in hardware, software, or both. Aspects of the security resource manager 108 can be also implemented as firmware of the SoC 102 or firmware of a device of the SoC 102, such as an ML or neural network (NN) processor, host (or tensor) processing unit, or safety filter model. The security resource manager 108 is used to implement a high-integrity ML execution framework and includes control logic implemented in hardware, software, or both. For example, the security resource manager 108 can include resources such as flip-flops, registers, buffers, etc. that are implemented in hardware and control logic (e.g., programmed code) that is implemented in software.

[0036] The IP / circuit block 110 generally includes individual IP devices such as processors, processor cores, or special-purpose processing devices. For example, the circuit block 110 can include an image signal processor (ISP) 112, a host processing unit (HPU) 114, a safety filter model (SFM) 116, and a graphics processing unit (GPU) 118. The circuit block 110 is referred to alternatively as an IP block 110, where the IP block can include one or more proprietary hardware elements. For example, each of the ISP 112, HPU 114, SFM 116, and GPU 118 can be a respective proprietary IP block (or IP device) of a particular entity or device manufacturer.

[0037] The HPU 114 can be a special-purpose processor, such as ML hardware accelerator, neural processor unit, neural network processor, or an application-specific processor. In some implementations, the HPU 114 is a neural network tensor processor thatincludes an integrated circuit architecture that is optimized for power-efficient execution of ML computations using tensor constructs such as multi-dimensional matrices. The HPU 114 is configured to access the safety filter model 116. In some implementations, one or more safety filter models 116 are included among the ML / neural network models that are implemented and / or executed on the HPU 114.

[0038] The inputs, outputs, and predictive / inference analytics of the safety filter models 116 can vary based on the watermarking and / or safety filter technology selected for establishing secure, high-integrity execution of inference operations along the pipeline. In some implementations, a safety filter model 116 can be configured as an abusive content filter, profanity filter, or the like. An input safety' filter model can be configured to run a standard (or more rigorous) censorship filter on inputs to the target model, whereas an output safety filter model can run corresponding filter processes on an output generated by the target model. These types of safety filters can be run on the HPU 114 concurrent with a target model or on a separate processor of the SoC 102.

[0039] The security resource manager 108 and / or CPU 104 can generate control signaling 125 to configure, execute, and manage model safety and verification pipeline activities of system 100. In some implementations, the control signaling 125 is used to distribute execution pipeline and ML model verification operations to at least the safety filter models 115 and / or the processors / devices of the IP block 110. The control signaling 125 is routed at system 100 using an example bus of the SoC 102. The control signaling 125 can include control / configuration commands, requests, data, instructions, or combination of these.

[0040] The system 100 is configured to implement a secure verification framework that reliably limits or precludes a user’s ability to generate abusive or malicious content as an output of a target model implemented on the HPU 114. In some implementations, the secure verification framework provides or generates an assurance indicator that confirms a particular output step, such as an output filter operation, will happen or has happened. For example, the framework can be used to generate an assurance indicator signal that confirms a prior secure watermarking action (e.g., based on a hash history check) was performed on an output computed by a target model or on a prior intermediate version of the output.

[0041] In some stances, the HPU 114 or SoC 102 can compute, determine, or otherwise obtain a future expectation that a watermarking operation will occur. Based on this required occurrence, the HPU 114 (or SoC 102) can securely buffer an unwatermarked version of a corresponding output of the target model while it determines whether the required watermarking operation was actually run. If it is determined that the required watermarkingoperation was run, then the buffered output can be securely passed as an output of the target model. If it is determined that the required watermarking operation was not run, then the buffered output may be discarded.

[0042] These aspects of system 100 provide a secure ML model execution environment and is defined as a secure pipeline configuration that includes the target model. The target model is a target ML model that computes inferences and generates corresponding ML outputs from a received input, such as a text input, audio input, images, or a combination of these. In some implementations, the ML model is a large language model (“LLM”) or a large computer vision model, for a generative artificial intelligence (“GenAI”) application.

[0043] The source / compiler 120 is configured to generate executable code based on compiler operations performed on input data (described below). The system 100 is configured to generate a set of golden security hashes based on hash calculations that are performed during compile time operations of the compiler 120. For example, the system 100 can generate a hash key by processing certain data values during the compiler operations and provide reference hashes in the form of golden security hashes for the target model. An example process for generating the golden security hashes is described in detail below with reference to Fig. 3.

[0044] During runtime, the system 100 compares the golden security hashes to runtime hashes to implement one or more security mechanisms corresponding to the safety filter models. The runtime hashes are newly obtained hashes that are computed at runtime when instructions derived based on the model binary / file 135 are executed to implement the target model. In some implementations, the runtime security hashes are computed and compared / evaluated against corresponding golden security hashes via an accumulation operation. For example, as system 100 executes a target model, the secure verification framework is used to evaluate or assess runtime security hashes against golden security hashes. Each comparison or evaluation can occur at certain checkpoints or stages along a verification pipeline.

[0045] In some implementations, at least one checkpoint of the accumulation operation determines whether a newly computed runtime security hash matches a corresponding golden security hash, and initiates execution in a high assurance mode based on a determined match. For example, the system 100 can execute a safety filter associated with a SFM 116, which generates an accumulated runtime hash. The system 100 can also execute an inference task or workload using a target model and determine that a golden security hash corresponding to the inference task (or model) matches a high assurance signal value.

[0046] In some implementations, to obtain a high assurance signal as early as possible in the execution pipeline, the system 100 generates accumulated runtime hash values and concurrently measures / compares those hashes to corresponding golden hashes alongside execution of a target model. For example, the system 100 can generate a runtime hash of the SFM 116 (or target model) across a first 10MB of instructions for the model and / or a set of weights for at least one neural network layer of the model. If the runtime hash matches a corresponding golden hash, then the system 100 can enter a high integrity mode or generate some other type of safety output, such as a watermarker.

[0047] As discussed above, the high assurance signal can be an indicator signal that causes the target model to enter a secure (or high assurance) execution mode. While in this secure execution mode, control logic (e.g., firmware) of a host processor that executes the model identifies and / or verifies the accumulated runtime hash of the input safety filter. The accumulated runtime hash can be verified as a reliable indicator of prior execution of a required safety filter based on a determined consistency between the accumulated runtime hash and a corresponding golden security hash. This consistency verification of the accumulated runtime hash of the input safety filter provides a provenance check that ensures execution of a watermarker associated with the high security mode. These and other runtime operations are described in more detail below with reference to FIG. 2.

[0048] As used in this document, a watermarker can be defined as an ML model that generates a type of digital breadcrumb (e.g., a watermark) or secure marker that can be used to identify the model and to enforce a security requirement of system 100. The digital breadcrumb or secure marker is generated using, for example, a corresponding set of weights for a neural network layer of the ML model. There are multiple uses for such a watermark. In some implementations, the watermarks are used to preserve or assess overall health of training data processed by a target model. For example, performance of certain models that employ unsupervised learning (e.g., generative models) can degrade over time if such models are trained on outputs of other generative models. Accordingly, the watermarking operations can be used to identify and / or mark certain ty pes of generative outputs (and inputs) to prevent or minimize the extent to which generative media enters a given training set.

[0049] As indicated above, the compiler / source 120 can be a cloud-based compiler asset (e.g., a server) that is external to the SoC 102 and HPU 114. The source 120 can generate, store, and / or transmit a golden security hash 132, metadata 134, and a machine learning (ML) model binary 135. In some implementations, the source 120 can generate a model file / binary 135 that encodes data that represents one or more golden security hashes and correspondingmetadata that specifies type information about a target model implemented based on the binary 135. The golden security hash 132 can include one or more sets of golden hashes, where each set of golden hashes includes one or more discreet hash values that are calculated during compile time based on an example hash function implemented using the compiler.

[0050] As discussed above, the set of hashes includes one or more runtime security hashes that are generated during runtime when the model binary 135 is executed on a specialpurpose integrated circuit, such as a neural network processor or hardware accelerator. By using the hashes to assess different stages of a target model’s execution pipeline, the system 100 can determine whether the model is being run / executed outside of the safety filters and / or watermarking tools, such that the model can be used to generate abusive content. Based on the outcome of that determination, the system 100 can restrict or prevent the target model from generating the abusive content, for example, by terminating the relevant ML operation being executed by the target model.

[0051] The compiler 120 is configured to generate the ML model binary 135 and the metadata 134 during compile time and send the ML model binary 135 and the metadata 134 as part of a model data file 124. In some implementations, the compiler 120 is a secure on- device ML compiler. For example, the compiler 120 and the SoC 102 are collocated in a single electronic / computing device (e.g., a computer, server, or mobile / client device) such that the model data file 124 is transmitted or passed locally at the electronic device. In some other implementations, compiler 120 is remote relative to the SoC 102 or a computing device that includes the SoC 102. In general, the model data file 124, as well as specific items of the file 124, can be transmitted within system 100 via a wired or wireless connection.

[0052] The ML model binary 135 is compiled from source code of the target model (described below). The metadata 134 includes type information that indicates a level of assurance verification assigned to the target model. In some implementations, the metadata includes an encoded type (e.g., a binary ) value that indicates whether a target model will participate in a high assurance verification process and indicates the types of high assurance verification actions that are required to be executed. In some examples, the system 100 can generate the golden security' hashes during compile time, as described in further detail below with reference to FIG. 2.

[0053] In the example of Fig. 1, system 100 and the SoC 102 is an integrated circuit of an example user / client device 130, consumer electronic device, or mobile device, where each of these devices can include items such as a smartphone 130a, tablet 130b, laptop 130c, smartwatch or wearable device 130d. The devices 130 may also include other items such asan eNotebook, Netbook, smart speaker, or mobile computer. In some implementations, the system 100 and the SoC 102 are integrated circuits of a desktop computer, network server, or related cloud-based asset.

[0054] Fig. 2 shows an example architecture for a verification pipeline 200. In some implementations, the pipeline 200 is a secure verification pipeline 200 for implementing safety and integrity verification of target ML models 208 during system-level runtime operations. The system 100 uses verification pipeline 200 to implement techniques for a secure verification framework that limits or prevents target models 208 from generating abusive or malicious content.

[0055] The security mechanisms of the verification pipeline 200 are implemented at system 100 at least by determining consistency between one or more golden security hashes 132 and one or more runtime security hashes during a runtime operation of a target model 208. The consistency is determined by comparing a respective set of compile time and runtime security hashes or comparing a single compile time security hash with a single corresponding runtime security hash.

[0056] The consistency is determined based on a comparison of a hash value generated during a runtime operation (e.g., a runtime security hash 204) and a corresponding hash value generated during a compile time operation (e.g., a golden security hash 132. In some implementations, the system 100 determines whether to terminate inference operations being performed by the target model 208 based on a result of the determined consistency. For example, the security resource manager 108 or HPU 114 can generate an inference termination flag to proactively terminate an on-going inference operation in response to determining that a runtime security hash 204 is inconsistent with a corresponding golden security hash 132.

[0057] The system 100 can configure the generation of runtime security hashes through different instruction streams. In particular, the system 100 can select one or more instruction streams, perform a hashing operation / calculation using the selected streams, and to generate corresponding runtime security hashes 204 based on the hashing calculation. The runtime security hashes 204 can represent, or correspond to, different safety filters 210, 212.

[0058] The verification pipeline 200 includes a secure channel 206, at least one input safety filter 210, and at least one output safety filter 212. The verification pipeline 200 can be implemented in hardware, software, or both. In some implementations, the verification pipeline 200 is implemented through firmware control state 207 of a processing device (e.g., HPU 114) of system 100.

[0059] The verification pipeline 200 includes one or more verification stages that implement different aspects of an example secure verification framework. In the example of Fig. 2, the verification pipeline 200 includes five verification stages, however more or fewer stages can be included. The system 100 can configure and implement a sequence of verification stages during or prior to executing an ML model for a runtime operation. In some implementations, at least one verification stage includes an operation in which hash calculations are performed to generate corresponding runtime security hashes 204.

[0060] One or more aspects of the verification pipeline 200 can be configured through the golden security hashes 132. The verification pipeline 200 can include any number of verification stages. One or more of the verification stages can correspond to each of the golden security hashes 132. In some implementations, the system 100 includes a verification stage for each safety filter 210, 212. In the example of Fig. 2, verification stages 3 and 4 correspond to input safety filter 210- A and 210-B, respectively, whereas verification stage 5 corresponds output safety filter 212.

[0061] Each of these verification stages (e.g., 3-5) can include a hash consistency operation executed using the corresponding safety filter 210, 212 for that verification stage. The system 100 is configured to store a hash history for a given chain of hash history across the verification pipeline 200 based on the operations of the corresponding safety filter 210 at each verification stage. In examples, the operations of safety filter 210-A / B include the hash consistency determinations described herein.

[0062] For example, each verification stage of a subset of verification stages can include a determination whether a golden security hash is consistent with a runtime security hash 204. The consistency determination can be implemented based on a comparative operation that assesses or compares a respective value of a runtime security hash 204 to a respective value of a corresponding golden security hash 132. In some implementations, the consistency is determined based on a match between the runtime security and golden security hash 132, 204. In some other implementations, the consistency is determined based on an approximate or substantial match between the runtime security and golden security hash 132, 204.

[0063] The system 100 can store runtime security hashes 204 generated by the model 208 in the protected register 202. For security purposes, the secure / protected register 202 is a register configured to securely store the golden security hashes 132 as well as runtime security hashes 204 generated during runtime execution of ML workload via target model 208. In some implementations, protected register 202 is configured to securely receive andstore different sets of security hashes based on certain encryption and cryptographic processes.

[0064] For example, the system 100 can use certain symmetric cryptographic ciphers to derive shared keys for encrypting / decrypting hash values. This security mechanism can be provided by asymmetric authentication through a secure boot feature of the host processor. In some implementations, the security mechanism uses an asymmetric key to establish a secure channel for iteratively obtaining or receiving new sets of golden security hashes from a source, such as a remote server or secure local resource. The asymmetric key is leveraged to provide a communication channel that is protected based on end-to-end encryption established via the asymmetric key.

[0065] The system 100 can implement the protected register 202 with access control (e.g., limited access for particular users). In some examples, the system 100 stores the runtime security hashes and the golden secunty hashes in the SRAM 106. In some other examples, the system 100 can store the runtime security hashes and the golden security hashes in a local memory of the HPU 114, an internal / external DRAM device, or both. Irrespective of the storage location / resource, the security hashes are stored using a secured / protected resource or register of the particular memory resource used to store the security hashes.

[0066] As discussed above, the stages of the verification pipeline 200 implemented during runtime can include the input safety filters 210-A / B, the output safety filter 212, or both. Each of the safety filters 210-A / B is used to verify the safety and / or integrity of an inference input, whereas the safety filter 212 is used to verify the safety and / or integrity of an inference output of the target model 208. The safety and / or integrity venfication of an inference input or output is established by performing hash calculations and consistency determinations corresponding to the appropriate input or output safety filters 2102, 212.

[0067] During compile time, the system 100 can compile source code for the target model 208 to generate the ML model binary 135. In some implementations, the firmware control state 207 is used to execute the binary 135 to implement the target ML model 208.Additionally, the system 100 can generate the golden security hashes 132 by performing a hash calculation based on a particular set of instruction branches that are generated from compiling the source code. Each golden security hash 132 can be mapped to an input safety filter 210 or an output safety filter 212 based on aspects of the verification pipeline 200 that are known to the compiler 120 during the compile time operations. The system 100 performs one or more hash calculations over a compute graph derived from, for example, a cloud-based compiler. This is described in more detail below at least with reference to FIGs. 3 and 4.

[0068] During runtime, the system 100 can execute the verification pipeline 200 based on a request to perform an inference task using the target model 208. In particular, as shown in FIG. 2, at verification stage 1, the system 100 receives the model file 124, reads and / or extracts the golden security hashes 132 from the model file, and routes the golden security hashes 132 through the secure channel 206 to be stored in the protected register 202.

[0069] The secure channel 206 can be implemented using one or more on-device security mechanisms. For example, the secure channel 206 can be authenticated and provided via a secure boot image. In other examples, the secure channel 206 can be a secure channel from a back-end service with end-to-end encryption between the back-end service and the HPU 114. In some implementations, the end-to-end encryption of the secure channel can be established based on mutual authentication and a certificate provisioned in the HPU 114.

[0070] In some examples, the system 100 can receive the golden security hashes 132 from a processor of the SoC 102 via a secure channel of the SoC 102, such as secure channel 206 or other secure means of communication. In some implementations, the golden security hashes 132 can be generated locally at the SoC 102 via a secure hashing mechanism of system 100. The system 100 can also perform implicit authentication of the golden security hashes 132 by receiving the golden hashes 132 at the HPU 114 from an on-device trusted entity that executes the secure hashing mechanism.

[0071] The system 100 can then access the metadata 134 to implement the verification pipeline 200. In particular, the system 100 can read a type field of one or more data values encoded as metadata 134 to determine whether a particular target model 208 is identified as a high-integrity ML model. For example, a particular type of target model 208 may be identified as a GenAI model that processes certain types of sensitive information, such as classified data, intelligence data, medical information, or personally identifiable information. The model may also be capable of generating certain types of sensitive outputs.

[0072] Different metadata type fields can be mapped to these and other model attributes. The type fields can also indicate either a low-, medium-, or high-degree of assurance is required over the safety and integrity of the model’s execution. In some implementations, the firmware (or software) control state 207 performs a read metadata operation 211, reads a metadata type field of the metadata based on the operation, and generates a control signal 214 to trigger an input safety filter action 215, 217 based on the value of the type field.

[0073] The control state 207 can also generate a control signal 229 to trigger an output safety filter action 240 based on a value of the metadata type field. The output safety filter action 240 includes performing venfication check 242. An output signal of the verification check 242 can be used to control whether an output of the target model 208 is ultimately generated and / or passed as output of an inference task or workload. The result action 244 of this verification check 242 can be based on the comparative operation that determines a consistency between RTHash2 (runtime security hash) and GRHash2 (golden security hash). In some implementations, a final encrypted result 246 of this check is stored in the protected register 202.

[0074] The type field can cause the control state 207 to enter a particular mode (e.g., a high integrity mode) of the verification pipeline 200. For example, the type field can indicate “input safety filter,” “high integrity,” “output filter,” or a combination thereof as part of the verification pipeline 200, as described in more detail below. In some implementations, access to the type field or information associated with the “type” indicator of the model is restricted. For example, the type field can be encrypted or otherwise conveyed to, and stored at, the integrated circuit of the HPU 114 in a manner that precludes tampering with or changing of the type field.

[0075] For example, as shown in FIG. 2, at verification stage 1, the system 100 can read the model file 124 and determine whether to implement (e.g., activate) one or more verification stages. As discussed above, the stages can include input safety filters 210, a high integrity model mode, an output safety filter 212, or a combination thereof based on the model type. The system 100 can activate the safety filters 210, 212 and the high integrity model mode 230 of the verification pipeline 200.

[0076] The control state 207 can implement a hash function for performing the hash calculations to generate or compute the runtime security hashes 204. In some implementations, the hash function receives input values that are based on an instruction issued or managed by the control state 207. For example, the hash function can receive opcodes or data values corresponding to a sequence of instruction streams that are passed to individual compute units / tiles of a special-purpose integrated circuit that implements the target model 208. For each model type, the system 100 can store (e.g., in a register) a hash history 218 for each of the generated runtime security hashes 204.

[0077] As an ML model representing a particular safety filter 210, 212 is executed, the system 100 can concurrently calculate a hash based on the instructions for executing the model and / or values of weights for corresponding layers of a neural network on which themodel is based. For example, the system 100 may calculate one or more hash values from an example hash function that receives a data value or an opcode of an instruction as inputs to the hashing function. These calculated hash values can be runtime secunty hashes 204 associated with safety filter operations and are measured or compared against corresponding golden security hashes 132. In some implementations, these hash values can also correspond to a watermarking operation used to impart a hash-based fingerprint onto security operations of the execution pipeline.

[0078] As one or more of these safety-filter-based runtime security hashes 204 are calculated and compared against a corresponding golden security hash, a respective result of each comparison can be accumulated and stored in a set of secure, tamper-resistant registers, such as protected register(s) 202). In some implementations, the secure, tamper-resistant registers are used to securely store hash history 218 that provides information about input and / or output ML-based safety filter operations that are performed along the execution pipeline. A host processor of the system 100 can use this information to verify and / or validate whether a particular safety filter operation(s) was performed for a specific target model 208, such as a target model that is required to be run or executed in a high integrity mode.

[0079] The system 100 can trigger security operations of the verification pipeline 200 in response to a host processor of the SoC 102 detecting or receiving a request to compute an inference. In this example, each verification stage corresponds to one or more operations for computing the inference and generating a corresponding output (or result) of the inference computation. The system 100 can generate the runtime security hash based on the instruction set architecture (ISA) of the HPU 114 or another processor of the IP block 110, such as the GPU 118. In this case, the instructions of the ISA are staged in a software-managed queue, and the control state 207 can generate the runtime security hash 204 by reading each instruction and calculating a rolling runtime security hash 204 based on each instruction being scheduled for one or more compute tiles of the HPU 114.

[0080] Based on the model type, the system 100 can implement one or more verification stages to perform one or more hash calculations as part of an input safety filter 210, an output safety filter 212, or both (e.g., as part of the high integrity verification pipeline). For example, if the type field of a metadata value indicates the target model 208 is of the “high integrity” type, the system 100 can receive the golden security hashes 132 from the source 120 via the secure channel 206 and store the golden security hashes 132 in the protected register 202.

[0081] As shown in FIG. 2, at verification stage 2, the system can load (232) the received golden security hashes from the protected register 202 to compare against the runtimesecurity hashes 204 as part of the pipeline verification check 234. For example, the golden security hashes 132 can be loaded to a general-purpose register of the HPU 114 and later accessed for a given comparative operation against the runtime security hashes 204. To implement its pipeline verification check 234, the system 100 can iteratively check 236 for additional (“More”) golden security hashes 132 for use in determining consistency (or inconsistency) with a related set of runtime security hashes 204.

[0082] At verification stages 3 and 4, the system 100 performs hash calculations at the input safety filter 210-A and the input safety filter 210-B to generate RTHashO and RTHashl, respectively. The system can then compare RTHashO and RTHashl (220) with corresponding golden security hashes 132 (e.g., GRHashO and GRHashl). Based on the comparison, the system 100 can output a verification result 222 for each input safety filter 210. The system 100 is configured to generate a verification failure when a verification result 222 indicates: i) RTHashO is inconsistent with GRHashO and / or ii) RTHashO is inconsistent with GRHashl. If verification result 222 indicates inconsistency between a runtime security hash 204 and a golden security hash 132, then the system can terminate the operations of the verification pipeline 200. If the system generates a verification success, the system can continue the operations of the verification pipeline 200.

[0083] At verification stage 5, the system 100 can perform the inference on the input sample (e.g., image, text, or both), and can generate RTHash 2. The system can then compare RTHash2 with corresponding golden security' hashl32 (e.g., GRHash2). Based on the comparison, the system 100 can output a verification result for each input safety filter 210. If the verification result is a verification success, the system can determine whether there are any remaining golden security hashes. If there are remaining golden secunty hashes, the system can store the result of the inference as an encrypted result. In some cases, the system can determine to decrypt the result.

[0084] At verification stage 6, the system 100 performs hash calculations at the output safety filter 212 to generate RTHash3. The system can then compare RTHash3 with corresponding GRHash3. Based on the comparison, the system can output a verification result 250 for the output safety filter 212. If the system 100 generates a verification success, the system can decrypt the encrypted result generated by the target model 208. In some cases, based on the verification success, the system can schedule the execution of another task or ML workload at the target model 208.

[0085] In another example, if the model is of the “input safety filter” type, the system may only perform steps 1, 3, and 4 . In particular, the system may only generate runtimesecurity hashes 204 by performing hash calculations at corresponding input safety filters. In another example, if the model is of the “output safety filter type,” the system may only perform steps 1, 5, and 6. In particular, the system 100 may only generate runtime security hashes 204 by performing hash calculations at inference and at the output filters.

[0086] Fig. 3 shows an example compiler system 300 for generating a model binary / file 135 and corresponding golden security hashes 132 for implementing pipeline integrity verification in a runtime environment.

[0087] System 300 includes compiler 120, which is configured to generate executable code based on compiler operations performed on input data. The input data includes program / source code 304 (“source code 304”) written using a high-level programming language. The compiler 120 generates the executable code in response to compiling the source code 304 based in part on a set of compiler operations. In some implementations, the compiler operations are a set of core compiler operations generally (or normally) performed by any, some, or most example compilers.

[0088] System 300 includes an example computing device 312 that is used to initiate and / or perform compiler operations of compiler 120. The compiler 120 can be integrated in, or accessible by, computing device 312, such as a consumer electronic device or mobile / client device. Computing device 312 can be a tablet, laptop, Chromebook, eNotebook, Netbook, or another related mobile computing device. In some implementations, computing device 312 is a desktop computer, network server, or related cloud-based asset. The executable code generated by the compiler 120 is ultimately passed to an example hardware integrated circuit 310.

[0089] In the example of Fig. 1, the integrated circuit 310 is a host processor 114, as described above. The HPU 114 can have multiple execution contexts. In particular, the HPU 114 can have a control core execution context that receives requests from applications and reserves resources for anew workload by executing instructions of an ISA. The ISA can be based on advanced reduced instruction set computer (RISC) instructions. Additionally, the HPU 114 can have a processing core execution context that serves information to a tensor core execution context through execution of the RISC instructions. The processing core can be an instruction and vector engine configured to generate instructions for vector operations in support of ML workloads. The tensor core includes multiple compute tiles that perform computations by executing instructions that are generated using the ISA.

[0090] In some examples, the integrated circuit 310 can be a graphics processing unit (GPU) adapted for ML or other computations related to graphics processing. In some otherexamples, the integrated circuit 310 can be a special-purpose integrated hardware circuit such as a hardware accelerator configured to accelerate computations for ML workloads. The integrated circuit 310 can also be a neural network processor configured to implement one or more neural networks for various ML models.

[0091] In some implementations, system 300 is a subsystem / block of system 100 that implements the high-integrity verification pipeline based on the golden security hashes 132. The compiler 120 includes hash generation logic 324 that generates the golden security hashes 132 by performing hash calculations over a compute graph based on a hash flag 322 inserted during the compile time operations. In particular, compiler 120 can perform for in- situ hash management.

[0092] The hash generation logic 324 is configured to extract control independent instructions to generate the golden security hashes 132. In some implementations, an inline hash generation (IHG) operation 325 is performed to insert hash calculations into the instruction stream, whereas an inline verification check (IVC) operation 328 is performed to insert inline verification checks along the verification pipeline 200. The IHG operation 325 includes inserting compounding hash calculations throughout the instruction stream, which includes information about the instruction preceding each hash calculation (e.g., pc, opcode, operands).

[0093] Thus, during compile time, the system 100, 300 can compile source code 304 for the target model 208 to generate the ML model binary 135. The compiler 120 of system 300 can generate the golden security hashes 132 by performing hash calculations over the compute graph during its compile time operations. The compute graph can be derived from a cloud-based compiler 120. The compute graph is described by the instruction sequence for execution of the target model 208. The compute graph can include model data such as weights, but the size of these values was considered to be prohibitively large for use in hash calculations.

[0094] Fig. 4 shows an example compiler operation 400 for inserting one or more hash operations in an instruction stream.

[0095] As discussed above, the compiler 120 can generate a set of golden security hashes 132 by performing hash calculations over a compute graph based on a hash flag 322 inserted during the compile time operations. The compute graph includes nodes that represent operations to be executed by a processor core of a host processor of the SoC 102. The compute graph can also include different target parameters. Referring briefly to runtime operations, the system 100 can generate the runtime security hashes 204 based on a hashcalculation performed during runtime using an instruction sequence that corresponds to one or more nodes in a compute graph for the target ML model.

[0096] The compiler operation 400 includes an IHG-PASS 402 performed to insert hash calculations into the instruction stream for inline hash generation. The instruction stream can be deterministic such that hashes are consistent across arbitrary inputs to the target model(s) 208. The IHG-PASS 402 calculates rolling runtime security hashes 410 using each ISA instruction read from the queue 405.

[0097] In some implementations, precursor passes associated with CIE-1, CIE-2 can also be included to extract a post-dominance tree, which represents a set of control independent instructions. The post-dominance tree can be extracted prior to applying the hash insertion pass. The compiler operation 400 includes an IVC-PASS 404 performed to insert inline verification checks along a verification pipeline. For the IVC-PASS 404, hashes 412 can be generated and checked individually through execution of a target model 208 or GenAI workload for the model.

[0098] In some implementations, the compute graph includes nodes that correspond to an instruction or an instruction sequence of one or more instruction streams and one or more inline verification checks corresponding to stages of the verification pipeline 200. The instruction sequence is derived from executing the ML model binary 135. The IVC can include the input safety filters 210 or the output safety filter 212 at which the system 100 performs corresponding hash calculations during runtime. In some examples, the system 100 can partition (e.g., chunk) the hash calculations across the instructions (e.g. , the static binary) to reduce latency in the system during runtime.

[0099] Fig. 5 is an example process 500 for secure execution of a target ML model on an integrated circuit. Process 500 is also implemented or executed at system 100 using at least the compiler 120 and computing device 112 described above with reference to Fig. 1. Hence, descriptions of process 500 will reference the above-mentioned computing resources of system 100. In some examples, the steps or actions of process 500 are enabled by programmed software instructions, firmware instructions, or both. Each type of instruction may be stored in a non-transitory machine-readable storage device and is executable by one or more of the processors or other resources described in this specification.

[0100] Referring again to process 500, the system 100 can receive a golden security hash from a source at an integrated circuit of the system (502). For example, the integrated circuit can be an SoC that includes a TPU. The integrated circuit can receive the golden security hash from the source, where the source is external to the TPU and communicates data to aprocessor of the SoC. The source can be a cloud-based compiler external to the integrated circuit. In some examples, the source is external to the TPU, and the system can receive the golden security hash from a processor of the SoC via a secure channel of the SoC.

[0101] In some examples, the system 100 can receive a model fde from the source at the integrated circuit. The model file is for a target ML model that includes the model binary and metadata including type information that indicates a level of assurance verification assigned to the target ML model. In some examples, the source can generate the golden security hash based on a first hash calculation performed during compile time (e.g., when the target ML model is compiled to generate the model binary). The system performs the first hash calculation during compile time over a compute graph derived from a cloud-based compiler.

[0102] In particular, the system can generate a first golden security hash that represents an input filter. The system uses the input filter as a first verification check to verify integrity of an input sample, where the input sample is processed by the target ML model. In some examples, the system generates a second golden security hash that represents an output filter. The system uses the output filter as a second verification check to verify integrity of an inference output generated by the target ML model.

[0103] The system 100 can store the golden security hash in a protected register of the integrated circuit (504) and execute a model binary for a target ML model (506).

[0104] The system 100 can perform operations to compute an inference from a received input sample using the target ML model (508) and, concurrently, the system 100 can generate a runtime security hash based on an instruction sequence (510). The instruction sequence is derived from execution of the model binary at the integrated circuit. The system 100 generates the runtime secunty hash 204 based on a hash calculation performed during runtime based on the instruction sequence. The instruction sequence corresponds to one or more nodes in a compute graph for the target ML model.

[0105] In some examples, the integrated circuit includes multiple instruction streams, and the system 100 can generate the runtime security hash based on an instruction sequence derived from one or more of the instruction streams. In this case, the system 100 generates the runtime security hash 204 based on an instruction stream of an instruction vector engine, where the instruction vector engine calculates the runtime security hash.

[0106] For example, the integrated circuit can be a TPU that includes multiple compute tiles. The system can generate the runtime security hash based on instructions derived from an ISA of the TPU for the multiple compute tiles. In this case, the instructions from the ISA are staged in a software-managed queue, and the system can generate the runtime securityhash by reading the instructions from the software-managed queue and calculating a rolling runtime security hash using each ISA instruction read from the queue. The system can calculate the rolling runtime security hash using an instruction vector engine during runtime.

[0107] System 100 can determine whether the runtime security hash is consistent with the golden security hash based on a comparison of the runtime security hash and the golden security hash (512). In particular, the system compares each of the respective values of the runtime security hash and the golden security hash.

[0108] If the system 100 determines that the runtime security hash and the golden security hash are inconsistent, the system can generate an output of the verification check that indicates a verification failure, and the system can terminate operations to compute the inference from the received input sampled based on the verification failure. If the system determines that the runtime security hash and the golden security hash are consistent, the system computes the inference from the received input sample, and the system generates an output representing the computed inference.

[0109] The system 100 can generate an output of a verification check on an execution pipeline including the operations to compute the inference ( 14). The output is generated based on determining whether the runtime security hash is consistent with the golden security hash. The execution pipeline includes one or more stages, where each stage corresponds to one or more operations for computing the inference and for generating an output representing the computed inference.

[0110] For each stage of the one or more stages, the system generates a runtime hash and performs a verification check on the stage using the runtime security hash corresponding to the stage. The system then generates a corresponding output representing the computed inference.

[0111] In some implementations, the respective steps of process 500 are performed at a hardware integrated circuit as part of a larger compute operation to generate a ML output, including an output for a neural network layer of a neural network that implements one or more ML models. For example, the output can be a portion of a computation for a ML task or inference workload to generate an image processing, speech processing, or image recognition output. As indicated above, a portion of the integrated circuit can include a special-purpose neural network processor or hardware ML accelerator configured to accelerate computations for generating different types of data processing outputs.

[0112] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodiedcomputer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non transitory program carrier for execution by, or to control the operation of, data processing apparatus.

[0113] Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0114] The term “computing system” encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0115] A computer program (which may also be referred to or described as a program, software, a software application, a module, a software module, a script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0116] A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0117] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by. and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array), an ASIC (application specific integrated circuit), or a GPGPU (General purpose graphics processing unit).

[0118] Computers suitable for the execution of a computer program include, by way of example, can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory or a random-access memory or both. Some elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0119] Computer readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0120] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that isused by the user; for example, by sending web pages to a web browser on a user’s client device in response to requests received from the web browser.

[0121] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network.Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g., the Internet.

[0122] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0123] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0124] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components andsystems can generally be integrated together in a single software product or packaged into multiple software products.

[0125] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

Claims

What is claimed is:

1. A method for secure execution of a target machine-learning (“ML”) model on an integrated circuit, the method comprising: storing a golden security hash in a protected register of the integrated circuit in response to receiving the golden security hash from a source; performing, using the target ML model, operations to compute an inference from a received input sample in response to executing a model binary for the target ML model at the integrated circuit; concurrent with performing the operations, generating a runtime security hash based on an instruction sequence derived from execution of the model binary' at the integrated circuit; determining whether the runtime security hash is consistent with the golden security hash based on a comparison of respective values of the runtime security hash and the golden security hash; and generating an output of a verification check on an execution pipeline comprising the operations to compute the inference, wherein the output is generated based on the determination about consistency between the runtime security hash and the golden security hash.

2. The method of claim 1, further comprising: determining that the runtime security hash and the golden security hash are inconsistent; generating an output of the verification check that indicates a verification failure based on the inconsistency between the runtime and golden security hashes; and terminating operations to compute the inference from the received input sample based on the verification failure.

3. The method of claim 1, further comprising: determining that the runtime security hash and the golden security hash are consistent; computing the inference from the received input sample based on the determination that the runtime security hash and the golden security hash are consistent; and generating an output representing the computed inference.

4. The method of claim 1, wherein the execution pipeline comprises one or more stages, and each stage corresponds to one or more of the operations for computing the inference and for generating an output representing the computed inference.

5. The method of claim 4, further comprising, for each stage of the one or more stages of the execution pipeline: generating a runtime security hash for the stage; performing a verification check on the stage using the runtime security hash for the stage; and generating a corresponding output of the verification check on the stage.

6. The method of claim 1, wherein generating a runtime security hash comprises: generating the runtime security hash based on a hash calculation performed during runtime using an instruction sequence that corresponds to one or more nodes in a compute graph for the target ML model.

7. The method of claim 6, wherein the integrated circuit comprises a plurality of instruction streams and generating a runtime security hash comprises: generating a runtime security hash based on an instruction sequence derived from one or more of the plurality of instruction streams.

8. The method of claim 7, wherein the runtime security hash is generated based on an instruction stream of an instruction vector engine of the integrated circuit.

9. The method of claim 8, wherein the runtime security hash is calculated by the instruction vector engine.

10. The method of claim 6, wherein the integrated circuit is a tensor processing unit (“TPU”) comprising a plurality of compute tiles and generating a runtime security hash comprises: generating a runtime security hash based on instructions derived from an instruction set architecture (ISA) of the TPU and for the plurality of compute tiles.

11. The method of claim 10, wherein the instructions derived from the ISA are staged in a software-managed queue and generating a runtime secun ty hash comprises: reading, from the software-managed queue, the instructions derived from the ISA; and calculating a rolling runtime security hash using each ISA instruction read from the software-managed queue.

12. The method of claim 11, wherein the rolling runtime security hash is calculated by an instruction vector engine of the TPU during runtime.

13. The method of claim 1, wherein the integrated circuit is a system-on-chip (“SoC”) that includes a tensor processing unit (“TPU”), and receiving the golden security hash comprises: receiving the golden security hash from a source that is external to the TPU and that communicates data to a processor of the SoC.

14. The method of claim 13, wherein the source is a cloud-based compiler that is external to the integrated circuit.

15. The method of claim 1, further comprising: receiving, at the integrated circuit, a model file from the source, wherein the model file is for the target MU model and includes the model binary and metadata comprising type information that indicates a level of assurance verification assigned to the target ML model.

16. The method of claim 1, wherein the integrated circuit is a TPU, and receiving the golden security hash comprises: receiving the golden security hash from a source that is external to the TPU.

17. The method of claim 16, wherein the TPU is included on an SoC, and receiving the golden security hash comprises: receiving the golden security hash from a processor of the SoC via a secure channel of the SoC.

18. The method of claim 1, further comprising:generating the golden security hash based on a first hash calculation performed during compile time when source code for the target ML model is compiled to generate the model binary for the target ML model.

19. The method of claim 18, wherein generating the golden security hash comprises: generating a first golden security hash that represents an input filter that is used as a first verification check to verify integrity of an input sample to be processed by the target ML model.

20. The method of claim 18, wherein generating the golden security hash comprises: generating a second golden security hash that represents an output filter that is used as a second verification check to verify integrity' of an inference output generated by the target ML model.

21. The method of claim 18, wherein the first hash calculation performed during compile time is performed over a compute graph derived from a cloud-based compiler.

22. A system for secure execution of a machine-learning (“ML”) model implemented on an integrated circuit, the system comprising: an integrated circuit; a processing device; and a non-transitory machine-readable storage device storing instructions that are executable by the processing device to cause performance of operations comprising: storing a golden security hash in a protected register of the integrated circuit in response to receiving the golden security hash from a source; performing, using the target ML model, operations to compute an inference from a received input sample in response to executing a model binary for the target ML model at the integrated circuit; concurrent with performing the operations, generating a runtime security hash based on an instruction sequence derived from execution of the model binary at the integrated circuit; determining whether the runtime security hash is consistent with the golden security hash based on a comparison of respective values of the runtime security hash and the golden security hash; andgenerating an output of a verification check on an execution pipeline comprising the operations to compute the inference, wherein the output is generated based on the determination about consistency between the runtime security hash and the golden security hash.

23. A non-transitory machine-readable storage device storing instructions for secure execution of a machine-learning (“ML”) model implemented on an integrated circuit, the instructions being executable by a processing device to cause performance of operations comprising: storing a golden security hash in a protected register of the integrated circuit in response to receiving the golden security hash from a source; performing, using the target ML model, operations to compute an inference from a received input sample in response to executing a model binary for the target ML model at the integrated circuit; concurrent with performing the operations, generating a runtime security hash based on an instruction sequence derived from execution of the model binary at the integrated circuit; determining whether the runtime security hash is consistent with the golden security hash based on a comparison of respective values of the runtime security hash and the golden security hash; and generating an output of a verification check on an execution pipeline comprising the operations to compute the inference, wherein the output is generated based on the determination about consistency between the runtime security hash and the golden security hash.

Images