System and method for processing data via a data processing accelerator

Abstract

Embodiments of the present disclosure relate to methods and systems for processing data through a data processing (DP) accelerator. In one embodiment, a system receives an inference request from an application. The system extracts a watermark from an AI model with a watermark. The system verifies the extracted watermark based on a policy. The system applies the AI ​​model with the watermark to an inference input set to generate an inference result. The system sends a verification proof and the inference result to the application.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to machine learning model training and inference. More specifically, embodiments of the present disclosure relate to systems and methods for processing data through a data processing (DP) accelerator. Background Art

[0002] Machine learning models have been widely utilized recently as artificial intelligence (AI) technology has been deployed in various fields such as image classification or autonomous driving. Similar to an executable image or binary image of a software application, a machine learning model (also called an artificial intelligence (AI) model) can perform inference based on a feature set to classify features after training. As a result, machine learning models can be "portable" and can be utilized without authentication. There is currently a lack of effective digital copyright protection for machine learning models. In addition, for processing tasks performed using AI models delegated to auxiliary processing systems such as processing (DP) accelerators or remote systems, there is also a lack of evidence that the results produced by the DP accelerator system are protected by a "trust root" system. Summary of the invention

[0003] Embodiments of the present disclosure relate to methods and systems for processing data through a data processing (DP) accelerator.

[0004] A first aspect of an embodiment of the present disclosure provides an embodiment of the present disclosure relating to a method for processing data through a data processing (DP) accelerator, comprising: receiving an inference request from an application executed by a host; extracting a watermark from an artificial intelligence (AI) model with a watermark through the data processing DP accelerator; verifying the extracted watermark based on a policy; applying the AI ​​model with the watermark to an inference input set to generate an inference result; and sending the verification proof and the inference result to the application.

[0005] A second aspect of an embodiment of the present disclosure provides an embodiment of the present disclosure relating to an apparatus for processing data through a data processing (DP) accelerator, comprising: a receiving module for receiving an inference request from an application executed by a host; an extraction module for extracting a watermark from an artificial intelligence (AI) model with a watermark through the data processing DP accelerator; a verification module for verifying the extracted watermark based on a policy; an application module for applying the AI ​​model with the watermark to an inference input set to generate an inference result; and a sending module for sending the verification proof and the inference result to the application.

[0006] A third aspect of an embodiment of the present disclosure provides an embodiment of the present disclosure relating to a method for processing data through a data processing (DP) accelerator, comprising: sending an inference request to the data processing DP accelerator through an application, requesting to generate an inference result by applying an artificial intelligence (AI) model with a watermark to an inference input, wherein the request includes a request for performing the following operations: extracting a watermark from the AI ​​model with a watermark through the DP accelerator, and verifying the extracted watermark based on a policy; and receiving a verification certificate and an inference result through the application.

[0007] A fourth aspect of an embodiment of the present disclosure provides an embodiment of the present disclosure relating to an apparatus for processing data through a data processing (DP) accelerator, comprising: a sending module for sending an inference request to the data processing DP accelerator through an application, requesting to generate an inference result by applying an artificial intelligence (AI) model with a watermark to an inference input, wherein the request includes a request for performing the following operations: extracting a watermark from the AI ​​model with a watermark through the DP accelerator, and verifying the extracted watermark based on a policy; and a receiving module for receiving a verification certificate and an inference result through an application.

[0008] In a fifth aspect of an embodiment of the present disclosure, a data processing DP accelerator is provided: a processor; and a memory coupled to the processor to store instructions, which, when executed by the processor, causes the processor to execute the method described in the first aspect or the third aspect of the embodiment of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Embodiments of the present disclosure are illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references indicate similar elements.

[0010] Figure 1 is a block diagram illustrating an example of a system configuration for communication between a host and a data processing (DP) accelerator according to one embodiment.

[0011] Figure 2 is a block diagram illustrating an example of a host server in communication with a data processing (DP) accelerator according to one embodiment.

[0012] Figure 3 is a block diagram illustrating a host in communication with a data processing accelerator according to one embodiment.

[0013] Figure 4 is a flow chart illustrating a process of signing a watermarked AI model according to one embodiment.

[0014] Figure 5 is a flow chart illustrating a method performed by a DP accelerator according to one embodiment.

[0015] Figure 6 is a flow chart illustrating a method performed by an application according to one embodiment.

[0016] Figure 7 is a block diagram illustrating a DP accelerator with a configurable watermark unit according to one embodiment.

[0017] Figure 8 is a flow chart illustrating a process of configuring a watermark unit using a watermark algorithm according to one embodiment.

[0018] Fig. 9 is a flow chart illustrating a method performed by a DP accelerator according to one embodiment.

[0019] Fig.10 is a block diagram illustrating a DP accelerator with a watermark algorithm trainer according to one embodiment.

[0020] Fig.11 is a flow chart illustrating a method performed by a DP accelerator according to one embodiment.

[0021] Fig.12 is a flow chart illustrating a method performed by a DP accelerator according to one embodiment.

[0022] Fig.13 is a block diagram illustrating a DP accelerator with a watermark verification module according to one embodiment.

[0023] Fig.14 is a flow chart illustrating a process of verifying a watermark according to one embodiment.

[0024] Fig.15 is a flow chart illustrating a method performed by a DP accelerator according to one embodiment.

[0025] Fig.16 is a flow chart illustrating a method performed by an application of a host according to one embodiment.

[0026] Fig.17 is a block diagram illustrating a data processing system according to one embodiment. DETAILED DESCRIPTION

[0027] Various embodiments and aspects of the present disclosure will be described with reference to the details discussed below, and the accompanying drawings will illustrate various embodiments. The following description and the accompanying drawings are illustrative of the present disclosure and should not be construed as limiting the present disclosure. Many specific details are described to provide a thorough understanding of various embodiments of the present disclosure. However, in some cases, well-known or conventional details are not described in order to provide a brief discussion of embodiments of the present disclosure.

[0028] References in the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present disclosure. The phrase "in one embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment.

[0029] The following embodiments involve using a data processing (DP) accelerator to increase the processing throughput of certain types of operations that can be offloaded (or delegated) from a host device to the DP accelerator. The DP accelerator may be a graphics processing unit (GPU), an artificial intelligence (AI) accelerator, a math coprocessor, a digital signal processor (DSP), or other type of processor. The DP accelerator may be a proprietary design such as AI accelerators, GPU, etc. Although the embodiments are shown and described with respect to a host device securely coupled to one or more DP accelerators, the concepts described herein may be more generally implemented as a distributed processing system.

[0030] The host device and the DP accelerator may be interconnected via a high-speed bus such as a peripheral component express (PCIe) or other high-speed bus. Prior to performing the operations of the various aspects of the present invention described below, the host device and the DP accelerator may exchange keys and initiate a secure channel via the PCIe bus. Some of the operations include the DP accelerator using an artificial intelligence (AI) model (or a machine learning model) to perform inferences using data provided by the host device. Before the host device trusts the AI ​​model inference, the host device may have the DP accelerator perform one or more verifications, as described below, including determining a watermark of the AI ​​model. In some embodiments and operations, the DP accelerator is unaware that the host device is verifying the watermark of the AI ​​model at the DP accelerator.

[0031] The watermark of the AI ​​model is an identifier that identifies the source / manufacturer of the AI ​​model, embedded within the AI ​​model, or in the output of the AI ​​model, or a combination thereof. Embedding or implanting refers to modifying the AI ​​model or the training of the AI ​​model, such as modifying the layers or weights of the AI ​​model, or the training epoch (or other training parameters) of the AI ​​model. In some embodiments, the watermark may be a subset of the weights within the AI ​​model that includes the watermark when extracted from the AI ​​model. In an embodiment, the host device may send specific inputs to the DP accelerator, and when the DP accelerator executes the AI ​​model using the specific inputs, the DP accelerator extracts the watermark from the AI ​​model. The host device may verify / validate the watermark to obtain authorization before using the DP accelerator and / or AI model.

[0032] According to a first aspect, in response to a request received by a data processing accelerator, sent by an application (which may be hosted by a host machine) to embed digital copyright protection into an artificial intelligence (AI) model, a system (e.g., a DP accelerator) generates a watermark for the AI ​​model based on a watermark algorithm. The system embeds the watermark onto the AI ​​model. The system signs the AI ​​model with the embedded watermark to generate a signature. The system returns the signature and the AI ​​model with the embedded watermark to the application, wherein the signature is used to authenticate the watermark and / or the AI ​​model.

[0033] In one embodiment, a system (e.g., a host) sends a request to a data processing accelerator through an artificial intelligence (AI) application, requesting to generate a watermark for an AI model based on a watermark algorithm, embed the watermark into the AI ​​model, and generate a signature for the AI ​​model with the watermark based on a security key pair. The system receives the signature, wherein the signature is used to authenticate the watermark and / or the AI ​​model.

[0034] According to a second aspect, in response to a request received by a data processing (DP) accelerator, a request sent by an application to apply a watermark algorithm to an artificial intelligence (AI) model by the DP accelerator, a system (e.g., the DP accelerator) determines that the watermark algorithm is not available at a watermark unit of the DP accelerator. The system sends a request for the watermark algorithm. The system receives the watermark algorithm by the DP accelerator. The system configures the watermark unit with the watermark algorithm at runtime for causing the watermark algorithm to be used by the DP accelerator.

[0035] According to a third aspect, a system (eg, a DP accelerator) trains a watermark algorithm based on a predetermined set of criteria, wherein the watermark algorithm is trained to generate a variant of the watermark algorithm. The system configures a watermark unit using the variant of the watermark algorithm at runtime for the watermark algorithm to be used by the DP accelerator.

[0036] According to a fourth aspect, a system (e.g., a DP accelerator) receives an inference request from an application. The system extracts a watermark from an artificial intelligence (AI) model having a watermark. The system verifies the extracted watermark based on a policy. The system applies the AI ​​model having the watermark to an inference input set to generate an inference result. The system sends a verification proof and the inference result to the application.

[0037] In one embodiment, a system (e.g., a host application) sends an inference request to a data processing (DP) accelerator, requesting to generate an inference result by applying an artificial intelligence (AI) model with a watermark to an inference input, wherein the request includes extracting a watermark from the AI ​​model with a watermark by the DP accelerator, and verifying the extracted watermark based on a policy. The system receives a verification certificate and the inference result to the application.

[0038] Any of the above functions may be programmed as executable instructions on one or more non-transitory computer-readable media. When the executable instructions are executed by a processing system having at least one hardware processor, the processing system causes the functions to be implemented.

[0039] Any of the above functions may be implemented by a processing system having at least one hardware processor coupled to a memory programmed with executable instructions that, when executed, cause the processing system to implement the functions.

[0040] With respect to any of the above aspects, in one embodiment, a watermark may be embedded in one or more nodes of one or more layers of an AI or machine learning model. For example, a watermark may be implanted in one or more weight variables or bias variables. Alternatively, one or more nodes (e.g., fake nodes that are not used or unlikely to be used by the machine learning model) or node layers may be created to implant or store watermarks. The host processor may be a central processing unit (CPU), and the DP accelerator may be a general processing unit (GPU) coupled to the CPU via a bus or interconnect. The DP accelerator may be implemented in the form of an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) device, or other form of integrated circuit (IC). Alternatively, the host processor may be part of a primary data processing system, and the DP accelerator may be one of many distributed systems as an auxiliary system, and the main system may remotely offload its data processing tasks via a link or network, such as a cloud computing system, such as a software as a service or SaaS system, or a platform as a service or Paas system. The link between the host processor and the DP accelerator may be a peripheral component express interconnect (PCIe) link or a network connection (such as an Ethernet connection).

[0041] Figure 1 is a block diagram showing an example of a system configuration for communication between a host and a data processing (DP) accelerator according to one embodiment. Figure 1, the system configuration 100 includes, but is not limited to, one or more client devices 101-102 communicatively coupled to a data processing server 104 via a network 103. The data processing server 104 may be coupled to a cloud provider 108 via the network 103. The client devices 101-102 may be any type of client device, such as a personal computer (e.g., a desktop computer, a laptop computer, and a tablet computer), a "thin" client, a personal digital assistant (PDA), an appliance with web capabilities, a smart watch, or a mobile phone (smart phone), etc. Alternatively, the client devices 101-102 may be other servers. The network 103 may be any type of network, such as a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination thereof, wired or wireless. Communications between the client devices 101-102, the data processing server 104, and the cloud provider 108 through the network 103 may be protected, for example, via TLS / SSL.

[0042] The data processing server (e.g., host) 104 can be any kind of server or server cluster, such as a web or cloud server, an application server, a back-end server, or a combination thereof. The server 104 can include an interface that allows a client, such as a client device 101-102, to access resources or services provided by the server 104. For example, the server 104 can be a cloud server or a server in a data center that provides various cloud services to clients, such as, for example, cloud storage, cloud computing services, big data services, modeling services, machine learning training services, data mining services, etc. The server 104 can be configured as part of a software as a service (SaaS) or platform as a service (PaaS) system on a cloud, which can be a private cloud, a public cloud, or a hybrid cloud. The interface can include a web interface, an application programming interface (API), and / or a command line interface (CLI).

[0043] For example, a client, in this example, a user application of client device 101 (e.g., a web browser, an application), can send or transmit instructions for execution (e.g., artificial intelligence (AI) training, inference instructions, etc.) to server 104, and server 104 receives the instructions via an interface through network 103. In response to the instruction, server 104 communicates with DP accelerators 105-107 to complete the execution of the instruction. In another embodiment, the client can remotely run an application from server 104 (via a web browser or web application), and the client can remotely request the application to execute the instruction. In some embodiments, the instruction is a machine learning type instruction, where a DP accelerator as a dedicated machine or processor can execute the instruction many times faster than the execution of server 104. Server 104 can therefore control / manage the execution of one or more DP accelerators in a distributed manner. Server 104 then returns the execution results to client devices 101-102. A DP accelerator or AI accelerator may include one or more dedicated processors, such as from Available Artificial Intelligence (AI) chipset, or alternatively, the DP accelerator can be Or some other AI chipset provider's AI chipset.

[0044] According to one embodiment, each application that accesses any of the DP accelerators 105-107 and is hosted by the DP server 104 (also referred to as a host) can be verified to be provided by a trusted source or vendor. Each application can be launched and executed within a trusted execution environment (TEE) that is specifically configured and executed by the central processing unit (CPU) of the host 104. When an application is configured to access any of the DP accelerators 105-107, a secure connection will be established between the host 104 and a corresponding one of the DP accelerators 105-107, so that data exchanged between the host 104 and each of the DP accelerators 105-107 is protected from attacks from malware.

[0045] Cloud provider 108 may provide cloud operation services to data processing server 104. Such services include management, provisioning, and configuration of a cloud network of which server 104 may be a part. Although illustrated as a single server, in some embodiments, cloud provider 108 may represent many servers or may be integrated with data processing server 104.

[0046] Figure 22 is a block diagram illustrating an example of a host communicating with a data processing (DP) accelerator according to one embodiment. In one embodiment, system 200 provides a protection scheme for secure communication between a host accelerator and a DP accelerator with or without hardware modifications to the DP accelerator. Figure 2 , the host machine or server 104 can be depicted as a system with one or more layers to be protected from intrusion, such as user applications 203, runtime libraries 205, drivers 209, operating system 211, and hardware 213 (e.g., security module (Trusted Platform Module (TPM) / Central Processing Unit (CPU))). The host machine 104 is typically a CPU system that can control and manage execution jobs on the host system or DP accelerators 105-107. In order to protect the communication channel between the DP accelerator and the host, different components may be required to protect different layers of the host system that are vulnerable to data intrusion or attack. For example, a trusted execution environment (TEE) can protect the user application layer and the runtime library layer from data intrusion.

[0047] refer to Figure 2 In one embodiment, the host system 104 will include hardware with one or more CPUs 213, which are equipped with a security module (such as a trusted platform module (TPM)) within the host machine 104. The TPM is a dedicated chip on an endpoint device that stores cryptographic keys (e.g., RSA cryptographic keys) specific to the host system for hardware authentication. Each TPM chip can contain one or more RSA key pairs (e.g., public and private key pairs) called endorsement keys (EK) or endorsement certificates (EC), i.e., root keys. The key pairs are maintained inside the TPM chip and cannot be accessed by software. Critical parts of the firmware and software can then be hashed by the EK or EC before they are executed to protect the system from unauthorized firmware and software modifications. Therefore, the TPM chip on the host machine can be used as a root of trust for secure boot.

[0048] The TPM chip also protects the driver 209 and the operating system (OS) 211 in the working kernel space to communicate with the DP accelerator. Here, the driver 209 is provided by the DP accelerator vendor and can be used as one or more drivers of the user application to control the communication channel between the host and the DP accelerator. Because the TPM chip and secure boot protect the OS and the driver in its kernel space, the TPM also effectively protects the driver 209 and the operating system 211.

[0049] Since the communication channels for the DP accelerators 105 - 107 may be exclusively occupied by the OS and drivers, the communication channels are also protected by the TPM chip.

[0050] In one embodiment, the host 104 includes a trusted execution environment (TEE) 201, which is forced to be protected by a TPM / CPU 213. TEE is a secure environment. TEE can ensure that the code and data loaded inside the TEE are protected with respect to confidentiality and integrity. An example of TEE can be Intel Software Guard Extensions (SGX) or AMD Secure Encrypted Virtualization (SEV). Intel SGX and / or AMD SEV may include a central processing unit (CPU) instruction code set that allows user-level code to allocate a private area of ​​the CPU's memory, and the private area is protected from the impact of processes running at a higher privilege level. Here, TEE 201 can protect user applications 203 and runtime libraries 205, wherein user applications 203 and runtime libraries 205 can be provided by end users and DP accelerator vendors, respectively. Here, runtime libraries 205 can convert API calls into commands for the execution, configuration and / or control of DP accelerators. In one embodiment, runtime libraries 205 are used to be executed by user applications by providing a predetermined (e.g., predefined) kernel set.

[0051] In another embodiment, the host machine 104 includes a memory security application 207 implemented using a memory security language such as Rust and GoLang. These memory security applications running on a memory security Linux distribution such as MesaLock Linux can also protect the system 200 from data confidentiality and integrity attacks. However, the operating system can be any Linux distribution, UNIX, Windows OS, or Mac OS.

[0052] In one embodiment, the system can be set up as follows: Install the Storage Secure Linux distribution to a system equipped with TPM secure boot (such as Figure 2 The host system 104 can be installed offline during the manufacturing or preparation phase. The installation can also ensure that the user space applications of the host system are programmed using a memory-safe programming language. Ensuring that other applications running on the host system 104 are memory-safe applications can also mitigate potential confidentiality and integrity attacks on the host system 104.

[0053] After installation, the system can then be started through a TPM-based secure boot. TPM secure boot ensures that only signed / certified operating systems and accelerator drivers are turned on in the kernel space that provides accelerator services. In one embodiment, the operating system can be loaded through a hypervisor. Note that a hypervisor or virtual machine manager is computer software, firmware, or hardware that creates and runs virtual machines. Note that kernel space is a declarative area or range where the kernel (i.e., a predetermined (e.g., predefined) set of functions for execution) is identified as providing functions and services to user applications. If the integrity of the system is compromised, TPM secure boot may not be able to start, but instead shut down the system.

[0054] After the secure boot, the runtime library 205 runs and creates the TEE 201, which places the runtime library 205 in the trusted storage space associated with the CPU 213. Next, the user application 203 is opened in the TEE 201. In one embodiment, the user application 203 and the runtime library 205 are statically linked and opened together. In another embodiment, the runtime library 205 is first opened in the TEE, and then the user application 203 is dynamically loaded in the TEE 201. In another embodiment, the user application 203 is first opened in the TEE, and then the runtime library 205 is dynamically loaded in the TEE 201. Note that the static link library is a library linked to the application at compile time. Dynamic loading can be performed by a dynamic linker. The dynamic linker loads and links a shared library for running the user application at runtime. Here, the user application 203 and the runtime library 205 in the TEE 201 are visible to each other at runtime, for example, all process data are visible to each other. However, external access to the TEE is denied.

[0055] In another embodiment, the user application can call the kernel only from the set of kernels predetermined by the runtime library 205. In another embodiment, the user application 203 and the runtime library 205 are hardened using a side-channel-free algorithm to defend against side-channel attacks, such as cache-based side-channel attacks. A side-channel attack is any attack based on information obtained from the implementation of a computer system, rather than based on weaknesses in the implemented algorithm itself (e.g., cryptanalysis and software errors). Examples of side-channel attacks include cache attacks, which are attacks based on the ability of an attacker to monitor the cache of a shared physical system in a virtualized environment or cloud environment. Hardening can include shielding the cache, outputs generated by the algorithm to be placed on the cache. Next, when the user application completes execution, the user application terminates its execution and exits from the TEE.

[0056] In summary, system 200 provides multiple layers of protection for the DP accelerator (such as data communications such as machine learning models, training data, and inference outputs) from loss of data confidentiality and integrity. System 200 may include a TPM-based secure boot protection layer, a TEE protection layer, and a kernel authentication / verification layer. In addition, system 200 may provide a memory-safe user space by ensuring that other applications on the host machine are implemented in a memory-safe programming language, which may further eliminate attacks by eliminating potential memory corruption / vulnerabilities. In addition, system 200 may include applications using side-channel-free algorithms to defend against side-channel attacks such as cache-based side-channel attacks.

[0057] Figure 3 is a block diagram illustrating a host computer in communication with a data processing accelerator according to one embodiment. System 300 may be Figure 2 A detailed view of a portion of system 200 is provided. Figure 3 In one embodiment, the host system 104 includes a runtime library 205, which includes a host channel manager (HCM) 259. In one embodiment, the HCM 259 includes an authentication module 251, a termination module 252, a key manager 253, (one or more) key stores 254, and a cryptographic engine 255. The authentication module 251 can authenticate user applications running on the host server 104 to obtain permission to access or use resources of the DP accelerator. The termination module 252 can terminate the connection (e.g., terminate the channel associated with the connection). The key manager 253 can manage (e.g., create or destroy) asymmetric key pairs or symmetric keys for encrypting / decrypting one or more data packets of different secure data exchange channels. Here, each user application (as Figure 2 The user application 203 of the embodiment of the present invention may correspond or be mapped to different secure data exchange channels in a one-to-many relationship, and each data exchange channel may correspond to a DP accelerator. The key storage 254 may store encrypted asymmetric key pairs or symmetric keys. The cryptographic engine 255 may encrypt or decrypt data packets for data exchanged through any of the secure channels.

[0058] refer to Figure 3In one embodiment, the DP accelerator 105 includes an ACM 270 (similarly, the DP accelerator 107 includes 290) and a security unit (SU) 275. The security unit 275 may include a key manager 271, (one or more) key storage 272, a true random number generator 273, and a cryptographic engine 274. The key manager 271 may manage (e.g., generate, securely maintain, and / or destroy) asymmetric key pairs or symmetric keys. (One or more) key storage repositories 272 may store cryptographic asymmetric key pairs or symmetric keys. The true random number generator 273 may generate true random numbers for cryptographic purposes. The cryptographic engine 274 may encrypt or decrypt key information or data packets for data exchange. The security unit 275 may contain one or more RSA key pairs (e.g., public and private key pairs), i.e., root keys, referred to as endorsement keys (EKs) or endorsement certificates (ECs). The key pairs are maintained inside the security unit 275 and cannot be accessed by software. The security unit can then use the EK or EC to encrypt data or generate other security keys (which in turn can be used to encrypt data). Essentially, the root certificate (EK / EC) is hardened onto the security unit 275 for cryptographic purposes. In some embodiments, the ACM 270 and SU 275 are integrated modules.

[0059] In one embodiment, the DP accelerator 105 includes a watermark unit 276. The watermark unit 276 can be a hardware-based module that can identify or extract watermarks from an artificial intelligence (AI) model (also known as a "machine learning model"). In an embodiment, the watermark unit 276 includes a generation / embedding module 260. The generation / embedding module 260 can generate a watermark and embed / implant the watermark into the artificial intelligence (AI) model. In an embodiment, the watermark can be implanted into the AI ​​model before, after, or during the training of the AI ​​model. The watermark unit 276 can also implant (or embed) the watermark into the AI ​​model during the inference of the AI ​​model. In an embodiment, the functionality of the watermark unit 276 can be implemented alternatively using a kernel of one or more codes received from the host 104.

[0060] DP Accelerator 105 may include memory / storage 280. Memory / storage 280 is a non-permanent or permanent storage medium for storing data. The stored data may include AI model 277, watermark algorithm 278, watermark 279, signature 281, or any data used by DP Accelerator 105.

[0061] Figure 4 is a flow chart illustrating a process of signing an AI model embedded with a watermark according to one embodiment. Operation 400 illustrates that embedding a watermark into an AI model after training the AI ​​model may be performed by Figure 3 The host server 104 and / or the DP accelerator 105 executes. Figure 4 In operation 401, the host server 104 sends a training data set for AI model training to the DP accelerator 105 for processing by the DP accelerator. In operation 402, the DP accelerator 105 trains the AI ​​model based on the training data set. In operation 403, when the training is completed, the DP accelerator 105 sends a training completion notification to the host 104. In some embodiments, the host server 104 may send an AI model to be retrained based on the training data set. In some embodiments, the training data set in the plurality of training data sets is in a storage / memory of the DP accelerator 105 that is retrievable by the DP accelerator 105, wherein the host 104 sends a selection of the training data set instead of the base training data set.

[0062] In operation 404, the host 104 selects a watermark algorithm supported by the AI ​​model. In operation 405, the DP accelerator 105 generates a watermark based on the watermark algorithm and embeds the watermark onto the AI ​​model. In operation 406, the DP accelerator 105 selects a security key pair and generates a signature for the AI ​​model / watermark pair. The security key pair can be generated based on the root certificate of the DP accelerator 105. The security key pair can therefore be verified as coming from a trusted source (e.g., the DP accelerator 105). In one embodiment, the security key pair includes, for example, a public key and a private key based on an asymmetric key algorithm such as an RSA encryption algorithm. The signature includes generating a hash of (one or more) AI model / watermark binaries and encrypting the hash based on the private key of the security key pair to generate a signature. In one embodiment, the hash is a variant of an MD5 hash or a secure hash algorithm (SHA) hash or any other hash function. In operation 407, the encrypted hash (e.g., signature), the public key of the security key pair, and / or (one or more) watermark / AI model files are returned to the host 104, where the host 104 or any other third party can use the signature and public key to verify the (one or more) watermark / AI model files for the first verification and extract the watermark for the second verification to perform two-factor authentication.

[0063] Although operation 400 describes a watermark embedded in a trained AI model, in some embodiments, the watermark algorithm may be pre-selected as a default selection prior to training and applied to the AI ​​model before, during, or after training. In another embodiment, a trained AI model is provided by the host 104, and watermarking and signing of the AI ​​model is performed by the DP accelerator 105 during inference. In this embodiment, before the DP accelerator 105 applies the watermark and / or generates a signature for the AI ​​model / watermark, the DP accelerator 105 attempts to extract the watermark from the AI ​​model / watermark to determine whether the AI ​​model already includes a watermark. If the extraction is unsuccessful, the DP accelerator 105 continues to apply the watermark and / or generate a signature for the AI ​​model / watermark. In some embodiments, the AI ​​model / watermark pair includes metadata to indicate the version and / or type of the watermark algorithm applied to generate the watermark so that the watermark can be extracted and / or verified based on the version.

[0064] Figure 5 5 is a flow chart illustrating a method performed by a DP accelerator according to one embodiment. Process 500 may be performed by processing logic, which may include software, hardware, or a combination thereof. For example, process 500 may be performed by a DP accelerator such as Figure 3 DP accelerator 105) is executed. Figure 5 At box 501, in response to a request received by a data processing accelerator, a request sent by an application (which may be hosted by a host 104) to apply a watermark to an artificial intelligence (AI) model, processing logic generates a watermark for the AI ​​model based on a watermark algorithm. At box 502, processing logic embeds the watermark onto the AI ​​model. At box 503, processing logic signs the AI ​​model with the embedded watermark to generate a signature. At box 504, processing logic returns the signature and the AI ​​model with the embedded watermark to the application, where the watermark and / or AI model is verified using the signature.

[0065] In one embodiment, the processing logic also generates an AI model by training the AI ​​model based on a training data set. In one embodiment, the processing logic also receives a pre-trained AI model and retrains the AI ​​model by training the AI ​​model based on the training data set.

[0066] In one embodiment, signing the AI ​​model with the embedded watermark includes: generating a security key pair by a security unit of the DP accelerator; generating a hash for the AI ​​model with the embedded watermark; and encrypting the hash using a private key of the security key pair to sign the AI ​​model with the embedded watermark. In one embodiment, the hash is decrypted using a public key of the security key pair to verify the signature for the watermark and / or the AI ​​model.

[0067] In another embodiment, the processing logic further extracts the watermark from the AI ​​model and verifies both the extracted watermark and the signature for two-factor authentication. In another embodiment, the security unit includes a root certificate hardened onto the security unit, wherein the security key pair is a plurality of key pairs generated based on the root certificate of the security unit, and the security key pair is selected by the watermark algorithm to sign the AI ​​model with the embedded watermark.

[0068] Figure 6 is a flow chart illustrating a method performed by an application according to one embodiment. Process 600 may be performed by processing logic, which may include software, hardware, or a combination thereof. For example, process 600 may be performed by a processor such as Figure 3 The host server of the host 104 or any other third party (eg, end user, client, etc.) performs the reference Figure 6 , at box 601, processing logic sends a request to a data processing accelerator, the request generating a watermark for an artificial intelligence (AI) model based on a watermark algorithm, embedding the watermark onto the AI ​​model, and generating a signature for the AI ​​model with the watermark based on a secure key pair. At box 602, processing logic receives a signature and / or (one or more) watermark / AI model files, where the signature is used to verify the watermark and / or AI model. For example, processing logic decrypts the signature using a public key of the secure key pair to generate a first hash. Processing logic applies a hash algorithm to (one or more) watermark / AI model files to generate a second hash. The first hash and the second hash are compared for a match. If there is a match, the signature is successfully verified.

[0069] In one embodiment, processing logic sends a training data set (e.g., via host 104) to a data processing accelerator to train an AI model based on the training data set. In another embodiment, processing logic sends a pre-trained AI model to a data processing accelerator, where the AI ​​model is retrained based on the training data set. In one embodiment, the AI ​​model with an embedded watermark is signed by a security unit of the DP accelerator. The security unit may generate a security key pair based on the EK or EC. The security unit may then generate a hash for the AI ​​model with the embedded watermark and encrypt the hash using the private key of the security key pair to sign the AI ​​model with the embedded watermark. In another embodiment, the public key of the security key pair is used to decrypt the hash to verify the signature for the watermark and / or AI model.

[0070] In another embodiment, the verification includes extracting the watermark from the AI ​​model and further verifying the extracted watermark for two-factor authentication. In another embodiment, the security unit includes a root certificate hardened to the security unit, wherein the security key pair is a plurality of key pairs generated based on the root certificate of the security unit, and the security key pair is selected by the watermark algorithm to sign the AI ​​model with the embedded watermark.

[0071] At runtime, a user may select a watermark algorithm via an application on the host 104 to apply a watermark to the AI ​​model. The selected watermark algorithm may be selected based on a set of algorithms that are available or unavailable at a particular DP accelerator. If the watermark algorithm is available at the DP accelerator, the DP accelerator may apply the algorithm to the AI ​​model. If the watermark algorithm is not available at the DP accelerator, the DP accelerator may send a request for the watermark algorithm to the application via the host 104. The application may then send the requested watermark algorithm to the DP accelerator. In addition, expired or unused watermark algorithms (after an expiration time threshold) may be automatically scheduled to be removed from the storage of the DP accelerator.

[0072] Figure 7 is a block diagram illustrating a DP accelerator with a configurable watermark unit according to one embodiment. Figure 7 In one embodiment, the watermark unit 276 includes a generation / embedding module 260, a watermark algorithm receiver module 701, a watermark algorithm manager 702, and a configuration channel module 703. The generation / embedding module 260 can generate a watermark based on a watermark algorithm and embed the watermark into the AI ​​model. The watermark algorithm receiver module 701 can receive a watermark algorithm from a third-party source or from an application on the host 104. The watermark algorithm manager 702 can manage one or more watermark algorithms of the unit 276. The configuration channel module 703 can configure a channel via a cloud provider at the virtual machine level for the virtual machine to send the watermark algorithm to the unit 276.

[0073] As a first matter, a watermark for an AI model is an identifier that identifies the source for the AI ​​model. A watermark may include a pattern of redundant or non-redundant weights, nodes, and / or node layers for an AI model. A watermark may also include a modification of the weights during training to generate an output having a watermark identifier with a specific inferred input. Examples of watermark identifiers in the output of an AI model for outputting an image include images that present machine and / or human-readable identifiers (e.g., logos of companies, organizations, or teams) as the source of the AI ​​model. Types of watermarking algorithms include algorithms that add virtual nodes, layers, and / or change the weights, biases, and / or activation functions of an AI model. Other watermarking algorithms may change one or more parameters of a training subroutine, or include a hidden training data set for training, where the hidden training data set is not retrievable by the user / operator / trainer of the AI ​​model. Thus, a trained AI model may infer a watermark output based on inputs to a hidden training data set or inputs derived from a hidden training data set.

[0074] Figure 8 800 is a flow chart illustrating a process of configuring a watermark unit using a watermark algorithm according to one embodiment. Process 800 may be performed by processing logic, which may include software, hardware, or a combination thereof. For example, process 800 may be performed by Figure 1 Host 104, DP accelerator 105 or cloud provider 108 or Figure 7 The DP accelerator 105 performs the reference Figure 8At box 801, the application (via the host 104) sends a request to the DP accelerator 105 requesting that a specific watermark algorithm be applied to the AI ​​model. At box 802, the DP accelerator 105 determines that the specific watermark algorithm is not available at the DP accelerator 105. At box 803, the DP accelerator 105 notifies the cloud provider 108 to enable the cloud provider 108 to apply the configuration settings of the cloud provider 108. In one embodiment, the cloud provider 108 is notified to configure a dedicated channel for the DP accelerator 105 (via the host 104) to receive the specific watermark algorithm from the application. In another embodiment, the cloud provider 108 is notified, and so the cloud provider 108 can subsequently accept the specific requested changes in the configuration settings. In one embodiment, the dedicated channel configuration is at the virtual machine level. At box 804, the DP accelerator 105 sends a request for the watermark algorithm to the host 104. At block 805, if the cloud provider 108 has a watermark algorithm, the cloud provider 108 returns the requested watermark algorithm to the DP accelerator 105, or the cloud provider 108 configures a dedicated channel. At block 806, the host 104 sends the watermark algorithm over the dedicated channel. At block 807, the DP accelerator 105 configures the watermark unit with the watermark algorithm to make the watermark algorithm available at the DP accelerator 105 for any request. In some cases, the watermark algorithm may be configured to expire if the algorithm is not used after a predetermined time threshold. Expired watermark algorithms may be scheduled for removal to be replaced with an improved version of the algorithm or to make storage space available for other algorithms.

[0075] Fig. 9 900 is a flow chart illustrating a method performed by a DP accelerator according to one embodiment. Process 900 may be performed by processing logic, which may include software, hardware, or a combination thereof. For example, process 900 may be performed by Figure 7 The DP accelerator 105 performs the reference Fig. 9 At block 901, in response to a request received by a data processing (DP) accelerator, sent by an application (the application may be hosted by a host 104) to apply a watermark algorithm to an artificial intelligence (AI) model by the DP accelerator, processing logic determines that the watermark algorithm is not available at a watermark unit of the DP accelerator. At block 902, processing logic sends a request for a watermark algorithm. At block 903, processing logic receives the watermark algorithm by the DP accelerator. At block 904, processing logic configures the watermark unit with the watermark algorithm at runtime for causing the watermark algorithm to be used by the DP accelerator.

[0076] In one embodiment, the processing logic also embeds a watermark based on the watermark algorithm into the AI ​​model. The processing logic returns the AI ​​model with the embedded watermark to the application, where the watermark is used to authenticate the AI ​​model. In another embodiment, the processing logic also signs the AI ​​model with the watermark to generate a signature, where the signature is used to authenticate the AI ​​with the watermark for two-factor authentication.

[0077] In one embodiment, the watermark algorithm is provided by a cloud provider, or a user of the cloud provider, or an application on a user device, or an application hosted by a host machine of a cloud provided by the cloud provider. In one embodiment, the watermark algorithm is received via a dedicated communication channel, wherein the communication channel is configured by the cloud provider.

[0078] In one embodiment, the processing logic also generates an AI model by training the AI ​​model based on a training data set received from the application via the host. In one embodiment, the processing logic also receives a pre-trained AI model and retrains the AI ​​model by training the AI ​​model based on the training data set.

[0079] In the previous paragraphs, some types of watermarking algorithms were discussed, including "Type 1" watermarking algorithms that add virtual layers and / or change the weights of the AI ​​model and "Type 2" watermarking algorithms that change the training subroutine to include a hidden training data set for training, where the hidden training data set is not retrievable by the user / operator / trainer of the AI ​​model. The following is the pseudo code for these watermarking algorithms.

[0080] Pseudocode for Type 1: (Input: AI model; Output: watermarked AI model)

[0081] Adding virtual layers after some selective original layers, and

[0082] For each layer:

[0083] Add dummy weights at specific locations, and / or

[0084] Edit weights.

[0085] Pseudocode for Type 2: (Input: training subroutine and / or training input data; Output: modified training subroutine and / or training input data) Insert hidden training input data or adjust training variables of training subroutine or training input data.

[0086] The verification process of verification type 1 can check the weights and / or layers for a specific signature as a verification of the watermark, for example, by adding or subtracting some extremely small values ​​(such as 0.000001221 or 0.0000001221), the weights at a specific position or layer have a specific pattern of trailing decimal values. The verification process of verification type 2 can perform inference using input data that infers output data with a machine-readable watermark. The verification process will then check the output data for the presence of the watermark for verification. Because the watermark algorithm and the verification process can affect the inference accuracy or the degree to which the watermark is embedded in the AI ​​model, the watermark algorithm can evolve / adapt to new AI models.

[0087] Fig.10 is a block diagram showing a DP accelerator with a watermark algorithm trainer according to one embodiment. Fig.10 In one embodiment, the DP accelerator 105 includes a memory / storage 280, which includes a watermark algorithm trainer 1001. The algorithm trainer 1001 can train a baseline watermark algorithm to generate an additional or derived watermark algorithm. In one embodiment, the algorithm trainer 1001 includes a generation module 1003, a training standard determiner 1005, an inference accuracy determiner 1007, a compatibility determiner 1009, a verification module 1011, and a configuration module 1013. The generation module 1003 can generate a new watermark algorithm using a previous watermark algorithm. The training standard determiner 1005 can determine a set of training standards for watermark algorithm training. The inference accuracy determiner 1007 can determine the inference accuracy of an AI model with or without a watermark. The compatibility determiner 1009 can determine a compatibility score and assign the compatibility score to a watermark / AI model pair. For example, a score of 0.3 may be assigned to compatibility for adding a virtual layer to a multi-layer deep neural network, a score of 0.7 may be assigned to compatibility for changing one or more weights of an AI model by a very small value (a very small value compared to the actual weight), and a score of 1 may be assigned to compatibility for training an AI model using an additional hidden input data set. The verification module 1011 may verify whether the AI ​​model includes a watermark and may determine the version of the watermark. The configuration module 1013 may configure the watermark unit using a watermark algorithm at runtime so that the watermark algorithm can be used for the DP accelerator.

[0088] Fig.11 1 is a flow chart illustrating a method performed by a DP accelerator according to one embodiment. Process 1100 may be performed by processing logic, which may include software, hardware, or a combination thereof. For example, process 1100 may be performed by Fig.10 The DP accelerator 105 performs the reference Fig.11At block 1101, processing logic trains a watermark algorithm by a DP accelerator based on a predetermined set of criteria, wherein the watermark algorithm is trained to generate variations of the watermark algorithm. The training may be a machine learning type training or a regression type training to maximize some training criteria through an objective function for a baseline watermark algorithm (e.g., an initial watermark algorithm from which the additional algorithm may be derived). The objective function may be a weighted combination of multiple criteria, such as the inference accuracy of the AI ​​model with the watermark, and / or a compatibility score indicator (indicating the compatibility of the AI ​​model with the watermark), etc. The objective function may be optimized via an iterative optimization algorithm, such as a gradient descent algorithm. The training will iteratively adjust watermark algorithm parameters (e.g., weight values, number of virtual layers added, training epochs, training input data sets) along the gradient (taking steps proportional to the gradient) to maximize / minimize the objective function, or until a predetermined threshold or number of iterations has occurred. Since there are different types or categories of AI models (e.g., single or multi-layer perceptrons, deep learning neural networks, convolutional neural networks, recurrent neural networks, etc.), different watermark algorithms can be trained to generate different AI models to maximize the inference accuracy and compatibility of the watermark / AI model pairs.

[0089] At block 1102, processing logic configures a watermark unit at runtime using a variation of a watermark algorithm for use by the DP accelerator. In one embodiment, the watermark algorithm may be versioned and assigned a globally unique identifier (GUID) or any other identifier that identifies the watermark algorithm. In this case, verification of the watermark for the AI ​​model may be performed on any of the AI ​​models as long as the identifier for the watermark algorithm is provided for verification.

[0090] In one embodiment, the set of criteria includes changes in the inference accuracy (degradation or improvement) of an artificial intelligence (AI) model with the watermark to be trained, and pairing compatibility of the watermark and the AI ​​model.

[0091] Fig.12 1 is a flow chart illustrating a method performed by a DP accelerator according to one embodiment. Process 1200 may be performed by processing logic, which may include software, hardware, or a combination thereof. For example, process 1200 may be performed by Fig.10 The DP accelerator 105 performs the reference Fig.12 At block 1201, processing logic also embeds the watermark into one or more AI models, and at block 1202, scores the AI ​​models and watermarks for pairing compatibility based on a compatibility scoring criterion. At block 1203, processing logic determines the inference accuracy of the AI ​​model without the watermark. At block 1204, processing logic determines the inference accuracy of the AI ​​model embedded with the watermark. At block 1205, processing logic determines a change (degradation or improvement) in the inference accuracy of the pairing.

[0092] In one embodiment, training is performed until a predetermined condition is reached for a set of criteria or until a predetermined number of training iterations. In one embodiment, in response to a request received by a data processing (DP) accelerator, sent by an application to apply a watermark algorithm to an AI model by a DP accelerator, the processing logic also applies the watermark algorithm to generate a watermark for the AI ​​model, embeds the watermark into the AI ​​model, and returns the AI ​​model with the watermark to the application, wherein the watermark is used to authenticate the AI ​​model. In one embodiment, the processing logic generates an AI model by training the AI ​​model based on a training data set.

[0093] Fig.13 FIG. 1 is a block diagram showing a DP accelerator with a watermark verification module according to one embodiment. Fig.13 In one embodiment, the DP accelerator 135 includes a memory / storage 280, which includes a watermark verification module 1301. The watermark verification module can verify the watermark and allow inference if the policy applied to the AI ​​model is met. In one embodiment, the watermark verification module 1301 includes a watermark extraction module 1303, a proof sender module 1305, a verification module 1307, and a policy module 1309. The watermark extraction module 1303 can extract the watermark from the AI ​​model / watermark pair. The proof sender module 1305 can generate a proof that the AI ​​model contains the watermark based on the extraction. The verification module 1307 can read the watermark and determine that the watermark comes from a trusted source. The policy module 1309 can apply one or more policies to the AI ​​model.

[0094] Fig.14 1400 is a flow chart illustrating a process of verifying a watermark according to one embodiment. Process 1400 may be performed by processing logic, which may include software, hardware, or a combination thereof. For example, process 1400 may be performed by Fig.13 The DP accelerator 105 performs the reference Fig.14At block 1401, an application of the host 104 sends an inference request to the DP accelerator 105. The inference request may include an inference input data set. At block 1402, the DP accelerator 105 extracts a watermark from the AI ​​model / watermark pair based on the inference request to determine whether the AI ​​model satisfies one or more policies before inference. A policy is a protocol scheme to evaluate whether training or inference is allowed to be applied to the AI ​​model / watermark by a specific DP accelerator. Examples of policies may be: the AI ​​model / watermark must contain a watermark generated by some baseline watermark algorithm or some version (e.g., the latest version) of an algorithm derived from the baseline watermark algorithm, the AI ​​model / watermark must contain a specific type of watermark algorithm (e.g., type 1 or type 2), the AI ​​model must contain a verifiable watermark from a specific source (e.g., a watermark corresponding to a specific organization, group, team, etc.), and / or if the signature for the AI ​​model / watermark is verified to be from a trusted source, such as generated by the DP accelerator 105 or other DP accelerators communicating with the host 104. In one embodiment, one or more policies may be applied to a category of AI models or a specific single AI model. In another embodiment, the DP accelerator may have a different policy set or the same policy set as another DP accelerator of the host 104 .

[0095] At block 1403, the DP Accelerator 105 verifies the watermark based on one or more strategies. Fig.13 The watermark verification module 1301 may maintain a mapping table that maps the AI ​​model and / or the watermark of the AI ​​model to one or more policies. The DP accelerator 105 may then retrieve the policy for verification based on the AI ​​model and / or the watermark.

[0096] If satisfied, then at block 1404, the DP accelerator 105 applies the AI ​​model / watermark pair to the inference input data to generate an inference result. At block 1405, the DP accelerator 105 sends the verification proof and / or the inference result to the requesting application (via the host 104). The verification proof may be a signature that includes an encrypted hash of the verification result, the AI ​​model / watermark, and / or the inference result, encrypted based on the private key of the security key pair of the DP accelerator 105. The verification proof is then decrypted by the host 104 or any third party verifying the proof through the public key of the security key pair.

[0097] Fig.15 is a flow chart illustrating a method performed by a DP accelerator according to one embodiment. Process 1550 may be performed by processing logic, which may include software, hardware, or a combination thereof. For example, process 1550 may be performed by Fig.13 The DP accelerator 105 performs the reference Fig.15At block 1551, processing logic receives an inference request from an application. At block 1552, processing logic extracts a watermark from an artificial intelligence (AI) model with a watermark through a data processing (DP) accelerator. At block 1553, processing logic verifies the extracted watermark based on a policy. At block 1554, processing logic applies the AI ​​model with the watermark to an inference input set to generate an inference result. At block 1555, processing logic sends the verification proof and the inference result to the application.

[0098] In one embodiment, after the extracted watermark is successfully verified based on the policy, an inference result is generated. In one embodiment, the policy includes that the AI ​​model contains a verifiable watermark generated by the DP accelerator and the watermark includes a machine-readable watermark for verification.

[0099] In one embodiment, the policy includes criteria for successfully verifying a signature for an AI model with a watermark. In one embodiment, different policies are applied to different types of watermark algorithms. In one embodiment, different policies are applied to different versions of a watermark algorithm. In one embodiment, the verification proof includes a signature for a result of an inference request.

[0100] Fig.16 1600 is a flow chart illustrating a method performed by an application of a host according to one embodiment. Process 1600 may be performed by processing logic, which may include software, hardware, or a combination thereof. For example, process 1600 may be performed by an application of a user, such as Figure 3 An application hosted on a host server 104. Fig.16 At block 1601, the processing logic sends an inference request to a data processing (DP) accelerator through an application, requesting to generate an inference result by applying an artificial intelligence (AI) model with a watermark to an inference input, wherein the request includes a request for the DP accelerator to extract a watermark from the AI ​​model with a watermark and to verify the extracted watermark based on a policy. At block 1604, the processing logic receives a verification certificate and / or an inference result through the application.

[0101] In one embodiment, after the extracted watermark is successfully verified based on the policy, an inference result is generated. In one embodiment, the policy includes that the AI ​​model contains a verifiable watermark generated by the DP accelerator and the watermark includes a machine-readable watermark for verification.

[0102] In one embodiment, the policy includes criteria for successfully verifying a signature for an AI model with a watermark. In one embodiment, different policies are applied to different types of watermark algorithms. In one embodiment, different policies are applied to different versions of a watermark algorithm. In one embodiment, the verification proof includes a signature for a result of an inference request.

[0103] Note that some or all of the components shown and described above may be implemented in software, hardware, or a combination thereof. For example, such components may be implemented as software installed and stored in a permanent storage device, which may be loaded by a processor (not shown) and executed in a memory to perform the entire process or operation described herein. Alternatively, such components may be implemented as executable code programmed or embedded into dedicated hardware such as an integrated circuit (e.g., an IC or ASIC for a specific application), a digital signal processor (DSP), or a field programmable gate array (FPGA), which may be accessed from an application via a corresponding driver and / or an operating system. In addition, such components may be implemented as specific hardware logic in a processor or processor core as part of an instruction set that may be accessed by a software component via one or more specific instructions.

[0104] Fig.17 1500 is a block diagram illustrating an example of a data processing system that may be used with one embodiment of the present invention. For example, system 1500 may represent any of the above-described data processing systems that perform any of the above-described processes or methods, such as, for example, the above-described client devices, hosts, or DP accelerators, such as, for example, the above-described Figure 1 Client 101-102, host server 104 or DP accelerator 105.

[0105] System 1500 may include many different components. These components may be implemented as integrated circuits (ICs), portions thereof, discrete electronic devices, or other modules suitable for circuit boards (such as the motherboard or add-in cards of a computer system), or as components otherwise incorporated into the chassis of a computer system.

[0106] It should also be noted that system 1500 is intended to illustrate a high-level view of many components of a computer system. However, it should be understood that additional components may be present in some implementations, and in addition, different arrangements of the components shown may occur in other implementations. System 1500 may represent a desktop computer, a laptop computer, a tablet computer, a server, a mobile phone, a media player, a personal digital assistant (PDA), a smart watch, a personal communicator, a gaming device, a network router or hub, a wireless access point (AP) or a repeater, a set-top box or a combination thereof. In addition, although only a single machine or system is shown, the term "machine" or "system" should also be considered to include any set of machines or systems that execute instruction sets (or multiple instruction sets) individually or collectively to perform any one or more methods discussed herein.

[0107] In one embodiment, system 1500 includes a processor 1501, a memory 1503, and devices 1505-1508 via a bus or interconnect 1510. Processor 1501 may represent a single processor or multiple processors including a single processor core or multiple processor cores. Processor 1501 may represent one or more general-purpose processors, such as a microprocessor, a central processing unit (CPU), etc. More specifically, processor 1501 may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements other instruction sets, or a processor that implements a combination of instruction sets. Processor 1501 may also be one or more special-purpose processors, such as an application-specific integrated circuit (ASIC), a cellular or baseband processor, a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, a graphics processor, a network processor, a communication processor, a cryptographic processor, a coprocessor, an embedded processor, or any other type of logic capable of processing instructions.

[0108] Processor 1501 can be a low-power multi-core processor socket (such as an ultra-low voltage processor) that can act as a main processing unit and a central hub for communicating with various components of the system. Such a processor can be implemented as a system on a chip (SoC). Processor 1501 is configured to execute instructions for performing the operations and steps discussed herein. System 1500 can also include a graphical interface that communicates with an optional graphics subsystem 1504, which can include a display controller, a graphics processor, and / or a display device.

[0109] Processor 1501 may communicate with memory 1503, which in one embodiment may be implemented via multiple memory devices to provide a given amount of system memory. Memory 1503 may include one or more volatile storage (or memory) devices, such as random access memory (RAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), static RAM (SRAM), or other types of storage devices. Memory 1503 may store information including sequences of instructions to be executed by processor 1501 or any other device. For example, executable code and / or data for various operating systems, device drivers, firmware (e.g., an input-output basic system or BIOS), and / or applications may be loaded into memory 1503 and executed by processor 1501. The operating system may be any type of operating system, such as, for example, from of Operating system, Mac from Apple From of Or other real-time or embedded operating systems (such as VxWorks).

[0110] System 1500 may also include IO devices such as devices 1505-1508, including (one or more) network interface devices 1505, (one or more) optional input devices 1506, and (one or more) other optional IO devices 1507. Network interface device 1505 may include a wireless transceiver and / or a network interface card (NIC). The wireless transceiver may be a WiFi transceiver, an infrared transceiver, a Bluetooth transceiver, a WiMax transceiver, a wireless cellular telephone transceiver, a satellite transceiver (e.g., a global positioning system (GPS) transceiver) or other radio frequency (RF) transceiver, or a combination thereof. The NIC may be an Ethernet card.

[0111] Input device(s) 1506 may include a mouse, a touchpad, a touch-sensitive screen (which may be integrated with display device 1504), a pointer device such as a stylus, and / or a keyboard (e.g., a physical keyboard or a virtual keyboard displayed as part of a touch-sensitive screen). For example, input device 1506 may include a touch screen controller coupled to a touch screen. The touch screen and touch screen controller may, for example, detect contact and movement or interruption thereof using any of a variety of touch-sensitive technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch screen.

[0112] IO devices 1507 may include audio devices. Audio devices may include speakers and / or microphones to facilitate voice-enabled functions such as voice recognition, voice replication, digital recording, and / or telephone functions. Other IO devices 1507 may also include (one or more) universal serial bus (USB) ports, (one or more) parallel ports, (one or more) serial ports, printers, network interfaces, bus bridges (e.g., PCI-PCI bridges), (one or more) sensors (e.g., motion sensors such as accelerometers, gyroscopes, magnetometers, light sensors, compasses, proximity sensors, etc.), or combinations thereof. Device 1507 may also include an imaging processing subsystem (e.g., a camera), which may include an optical sensor such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) optical sensor for facilitating camera functions such as recording photos and video clips. Certain sensors may be coupled to interconnect 1510 via a sensor hub (not shown), while other devices such as a keyboard or thermal sensor may be controlled by an embedded controller (not shown), depending on the specific configuration or design of system 1500.

[0113] In order to provide permanent storage of information such as data, applications, one or more operating systems, etc., mass storage (not shown) may also be coupled to the processor 1501. In various embodiments, in order to achieve a thinner and lighter system design and improve system responsiveness, such mass storage may be implemented via a solid-state device (SSD). However, in other embodiments, the mass storage may be implemented primarily using a hard disk drive (HDD) with a smaller number of SSD storage devices to act as an SSD cache that enables non-volatile storage of context states and other such information during a power-off event, so that a fast power-on can occur when restarting system activities. A flash memory device may also be coupled to the processor 1501, for example, via a serial peripheral interface (SPI). The flash memory device may provide non-volatile storage of system software, including basic input / output software (BIOS) and other firmware for the system.

[0114] The storage device 1508 may include a computer-accessible storage medium 1509 (also referred to as a machine-readable storage medium or a computer-readable medium) on which is stored one or more sets of instructions or software (e.g., modules, units, and / or logic 1528) embodying any one or more of the methods or functions described herein. The processing module / unit / logic 1528 may represent any of the above components, such as, for example, the above-described Figure 1-Figure 3 Host server 104 or Figure 1-Figure 3 , Figure 7 , Fig.10 , Fig.13 DP accelerator 105. During the execution of processing module / unit / logic 1528 by data processing system 1500, processing module / unit / logic 1528 may also reside in memory 1503 and / or processor 1501 in whole or in part, and memory 1503 and processor 1501 also constitute machine-accessible storage media. Processing module / unit / logic 1528 may also be transmitted or received over a network via network interface device 1505.

[0115] Computer-readable storage medium 1509 can also be used to permanently store some of the above-mentioned software functions. Although computer-readable storage medium 1509 is shown as a single medium in the exemplary embodiment, the term "computer-readable storage medium" should be considered to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) storing one or more instruction sets. The term "computer-readable storage medium" should also be considered to include any medium capable of storing or encoding an instruction set for execution by a machine and causing the machine to perform any one or more methods of the present invention. Therefore, the term "computer-readable storage medium" should be considered to include, but is not limited to, solid-state memory, and optical and magnetic media, or any other non-temporary machine-readable medium.

[0116] The processing modules / units / logic 1528, components and other features described herein may be implemented as discrete hardware components or integrated into the functionality of hardware components such as ASICS, FPGAs, DSPs or similar devices. In addition, the processing modules / units / logic 1528 may be implemented as firmware or functional circuits within a hardware device. Furthermore, the processing modules / units / logic 1528 may be implemented in any combination of hardware devices and software components.

[0117] Note that although system 1500 is shown as having various components of a data processing system, it is not intended to represent any particular architecture or manner of interconnecting these components; as such details are not relevant to embodiments of the present invention. It will also be appreciated that network computers, handheld computers, mobile phones, servers, and / or other data processing systems having fewer or perhaps more components may also be used with embodiments of the present invention.

[0118] Some portions of the foregoing detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of operations leading to a desired result. Operations are those requiring physical manipulations of physical quantities.

[0119] It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise apparent from the above discussion, it should be understood that throughout the description, discussions utilizing terms such as those set forth in the claims refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within computer system registers and memories into other data similarly represented as physical quantities within computer system memories or registers or other such information storage, transmission or display devices.

[0120] The techniques shown in the figures may be implemented using code and data stored and executed on one or more electronic devices. Such electronic devices use computer-readable media such as non-transitory computer-readable storage media (e.g., magnetic disks, optical disks, random access memory; read-only memory; flash memory devices; phase change memory) and transitory computer-readable transmission media (e.g., electrical, optical, acoustic, or other forms of propagated signals such as carrier waves, infrared signals, digital signals) to store and communicate (internally and / or over a network with other electronic devices) code and data.

[0121] The processes or methods depicted in the foregoing figures may be performed by processing logic including hardware (e.g., circuits, dedicated logic, etc.), firmware, software (e.g., embodied on a non-transitory computer-readable medium), or a combination thereof. Although the processes or methods are described above based on some sequential operations, it should be understood that some of the described operations may be performed in a different order. Moreover, some operations may be performed in parallel rather than sequentially.

[0122] In the foregoing description, embodiments of the present invention have been described with reference to specific exemplary embodiments of the present invention. It will be apparent that various modifications may be made thereto without departing from the broader spirit and scope of the present invention as set forth in the appended claims. Therefore, the description and drawings should be regarded as illustrative rather than restrictive.

Claims

1. A computer-implemented method for processing data by a data processing DP accelerator, the method include: receiving an inference request from an application executed by a host; Extracting a watermark from an artificial intelligence (AI) model having a watermark by a data processing DP accelerator, wherein the DP accelerator maintains a mapping table that maps the AI ​​model and / or the watermark to one or more policies, wherein each policy is used to evaluate whether the DP accelerator maintaining the policy is allowed to apply training or inference to the AI ​​model or watermark mapped by the policy; the policy includes: the AI ​​model or watermark must contain a watermark generated by some baseline watermark algorithm or some version of an algorithm derived from the baseline watermark algorithm, and the AI ​​model or watermark must contain a specific type of watermark algorithm; the specific type of watermark algorithm includes a "type 1" watermark algorithm that adds a virtual layer and / or changes the weight of the AI ​​model and a "type 2" watermark algorithm that changes the training subroutine to include a hidden training data set for training; Retrieve strategies for verification based on AI models and / or watermarks through DP accelerators; verifying the extracted watermark based on the retrieved policy; Applying the watermarked AI model to the inference input set to generate an inference result; and Send the verification proof and inference results to the application.

2. The method of claim 1, wherein after the extracted watermark is successfully verified based on the policy, an inference result is generated.

3. The method of claim 1, wherein the policy includes the AI ​​model containing a verifiable watermark generated by the DP accelerator and the watermark includes criteria for verifying the machine-recognizable watermark.

4. The method of claim 1, wherein the policy includes criteria for successfully verifying a signature for the watermarked AI model. The method of claim 1 , wherein different strategies are applied to different types of watermarking algorithms. The method of claim 1 , wherein different strategies are applied to different versions of the watermark algorithm.

7. The method of claim 1, wherein the verification proof comprises a signature of a result of the inference request.

8. A device for processing data by means of a data processing DP accelerator, include: a receiving module for receiving an inference request from an application executed by a host; An extraction module, for extracting a watermark from an artificial intelligence (AI) model having a watermark by a data processing (DP) accelerator, wherein the DP accelerator maintains a mapping table that maps the AI ​​model and / or the watermark to one or more policies, wherein each policy is used to evaluate whether the DP accelerator maintaining the policy is allowed to apply training or inference to the AI ​​model or watermark mapped by the policy; the policy includes: the AI ​​model or watermark must contain a watermark generated by some baseline watermark algorithm or some version of an algorithm derived from the baseline watermark algorithm, and the AI ​​model or watermark must contain a specific type of watermark algorithm; the specific type of watermark algorithm includes a "type 1" watermark algorithm that adds a virtual layer and / or changes the weight of the AI ​​model and a "type 2" watermark algorithm that changes the training subroutine to include a hidden training data set for training; Retrieve strategies for verification based on AI models and / or watermarks through DP accelerators; a verification module for verifying the extracted watermark based on the retrieved policy; An application module, for applying the watermarked AI model to the inference input set to generate an inference result; and The sending module is used to send the verification proof and inference results to the application.

9. A data processing DP accelerator, include: processor; as well as A memory coupled to the processor to store instructions which, when executed by the processor, cause the processor to perform a method according to any one of claims 1-7.

10. A computer-implemented method for processing data by a data processing DP accelerator, the method include: An inference request is sent by an application to a data processing DP accelerator, requesting to generate an inference result by applying an artificial intelligence AI model with a watermark to an inference input, wherein the request includes a request for performing the following operations: Extracting a watermark from an AI model with a watermark by a DP accelerator, wherein the DP accelerator maintains a mapping table that maps the AI ​​model and / or the watermark to one or more policies, wherein each policy is used to evaluate whether the DP accelerator maintaining the policy is allowed to apply training or inference to the AI ​​model or watermark mapped by the policy, wherein the policy includes: the AI ​​model or watermark must include a watermark generated by some baseline watermark algorithm or some version of an algorithm derived from the baseline watermark algorithm, and the AI ​​model or watermark must include a specific type of watermark algorithm; the specific type of watermark algorithm includes a "type 1" watermark algorithm that adds a virtual layer and / or changes the weight of the AI ​​model and a "type 2" watermark algorithm that changes the training subroutine to include a hidden training data set for training; Retrieve strategies for verification based on AI models and / or watermarks through the DP accelerator; and verifying the extracted watermark based on the retrieved policy; and Receive verification proof and inference results through the application. The method of claim 10 , wherein after the extracted watermark is successfully verified based on the policy, an inference result is generated.

12. The method of claim 10, wherein the policy includes the AI ​​model containing a verifiable watermark generated by the DP accelerator and the watermark includes criteria for verifying the machine-recognizable watermark.

13. The method of claim 10, wherein the policy includes criteria for successfully verifying a signature for the watermarked AI model.

14. The method of claim 10, wherein different strategies are applied to different types of watermarking algorithms.

15. The method of claim 10, wherein different strategies are applied to different versions of the watermark algorithm.

16. The method of claim 10, wherein the verification proof comprises a signature of a result of the inference request.

17. A device for processing data by a data processing DP accelerator, include: A sending module, configured to send an inference request to a data processing DP accelerator through an application, requesting to generate an inference result by applying an artificial intelligence (AI) model with a watermark to an inference input, wherein the request includes a request for performing the following operations: extracting a watermark from an AI model with a watermark through a DP accelerator, wherein the DP accelerator maintains a mapping table that maps the AI ​​model and / or watermark to one or more policies, wherein each policy is used to evaluate whether the DP accelerator maintaining the policy is allowed to apply training or inference to the AI ​​model or watermark mapped by the policy, wherein the policy includes: the AI ​​model or watermark must include a watermark generated by some baseline watermark algorithm or some version of an algorithm derived from the baseline watermark algorithm, and the AI ​​model or watermark must include a specific type of watermark algorithm; the specific type of watermark algorithm includes a "type 1" watermark algorithm that adds a virtual layer and / or changes the weight of the AI ​​model and a "type 2" watermark algorithm that changes a training subroutine to include a hidden training data set for training; Retrieve strategies for verification based on AI models and / or watermarks through the DP accelerator; and verifying the extracted watermark based on the retrieved policy; and The receiving module is used to receive the verification proof and the inference result through the application.

18. A data processing DP accelerator, include: processor; as well as A memory coupled to the processor to store instructions which, when executed by the processor, cause the processor to perform a method according to any one of claims 10-16.

19. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1-7 or 10-16.

Images