Model protection system and non-transitory machine-readable medium

Abstract

The present invention provides a model protection system comprising a processor. The processor is arranged to execute a guest virtual machine (VM), a host VM and a hypervisor. The guest virtual machine comprises a model and is arranged to send at least one command to a command hub. The host VM is arranged to manage and configure protection settings for a protected model with reference to the at least one command sent from the command hub; wherein the protected model is derived from the model. The hypervisor is arranged to receive a security setting command sent by the host VM and manage and configure a security protection component in accordance with the security setting command to set a read-only mode for the protected model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence (AI). Background Technology

[0002] In the field of artificial intelligence (AI), it is crucial to develop a protection scheme to safeguard compromised AI models. Traditionally, AI models can be injected into the kernel of an operating system (OS; for example, Android) or the Linux kernel for inference, and the AI ​​model can be driven within the kernel. However, since the AI ​​models of all the different applications running on the operating system are driven within the kernel, the AI ​​model is exposed when injected into or driven within the kernel. Encrypted AI models can be decrypted and executed in isolated execution environments (such as trusted execution environments, TEEs), which requires the support of a secure operating system and typically suffers from significant latency and poor flexibility, posing problems for mass production. Therefore, a new model protection architecture is urgently needed. Summary of the Invention

[0003] Therefore, one object of the present invention is to provide a model protection system and a non-transitory machine-readable medium for storing program code, wherein model protection can be provided when the program code is executed, in order to solve the above-mentioned problems.

[0004] According to at least one embodiment of the present invention, a model protection system is provided. The system may include a processor. The processor may be configured to execute a guest virtual machine (VM), a host VM, and a hypervisor. The guest VM may include a model and may be configured to send at least one command to a command hub. The host VM may be configured to receive at least one command sent from the command hub, and, with reference to the at least one command, manage and configure protection settings for the protected model, and output a security setting command, wherein the protected model is derived from the model. The hypervisor may be configured to receive the security setting command sent by the host VM, and, according to the security setting command, manage and configure security protection components to set the protected model to read-only mode.

[0005] According to at least one embodiment of the present invention, a non-transitory machine-readable medium for storing program code is provided. When a processor loads and executes the program code, the program code instructs the processor to execute: a guest virtual machine (VM), a host VM, and a hypervisor. The guest VM may include a model and may be arranged to send at least one command to a command hub. The host VM may be arranged to receive the sent at least one command from the command hub and to manage and configure protection settings for a protected model, wherein the protected model is derived from the model, with reference to the at least one command. The hypervisor may be used to receive security setting commands sent by the host VM and to manage and configure security protection components according to the security setting commands to set a read-only mode for the protected model.

[0006] One advantage of this invention is that by setting a read-only mode for the protected model used by DMA circuits (e.g., accelerated processing units, APUs), the protected model will not be modified during its lifetime (i.e., the integrity of the protected model is protected). Thus, since the Android system or its Linux kernel does not modify the protected model during inference, other unauthorized users may not be able to exploit the protected model to attack the APU, and other unauthorized users will not be able to attack the protected model while it is performing computations. Furthermore, by setting a no-read / write mode for the protected model used by the central processing unit (CPU), the protected model will not be read or written during its lifetime, and its contents will be protected.

[0007] These and other objects of the invention will undoubtedly become apparent to those skilled in the art after reading the following detailed description of the preferred embodiments shown in the various accompanying drawings. Attached Figure Description

[0008] Figure 1 This is a schematic diagram illustrating an electronic device according to an embodiment of the present invention.

[0009] Figure 2 This is a schematic diagram illustrating a model protection system according to an embodiment of the present invention.

[0010] Figure 3 This is a schematic diagram illustrating a model protection system according to another embodiment of the present invention.

[0011] Figure 4This is a schematic diagram illustrating the relationship between a model protection system according to an embodiment of the present invention and a verifier included in an isolated execution environment.

[0012] Figure 5 This is a schematic diagram illustrating the relationship between a model protection system according to an embodiment of the present invention and a verifier included in a primary virtual machine (VM).

[0013] Figure 6 This is a schematic diagram illustrating the relationship between a model protection system and an isolated execution environment for model decryption according to an embodiment of the present invention.

[0014] Figure 7 This is a schematic diagram illustrating the relationship between the master VM of the model protection system and the model decryption system in an embodiment of the present invention. Detailed Implementation

[0015] Certain terms used in the following description and claims refer to specific components. As those skilled in the art will understand, electronic device manufacturers may use different names to refer to a component. This application is not intended to distinguish between components with different names but the same function. In the following description and claims, the terms "comprising" and "including" are used in an open-ended manner and should therefore be interpreted as meaning "including but not limited to...".

[0016] Figure 1 This is a schematic diagram illustrating an electronic device 10 according to an embodiment of the present invention. By way of example and not limitation, the electronic device 10 may be a portable device, such as a smartphone or tablet. The electronic device 10 may include a processor 12, a storage device 14, and hardware circuitry 16. The processor 12 may be a single-core processor or a multi-core processor. The storage device 14 is a non-transitory machine-readable medium for storing computer program code PROG. The processor 12 is equipped with software execution capabilities. The computer program code PROG may include multiple software modules. Therefore, when loaded and executed by the processor 12, the computer program code PROG instructs the processor 12 to perform specified functions of the software modules. The electronic device 10 can be considered as a computer system using a computer program product, which includes a computer-readable medium containing the computer program code PROG. The hardware circuitry 16 is pure hardware, may consist only of logic gates, and can perform specified functions without executing software. Regarding the model protection system proposed in this invention, it can be implemented on the electronic device 10. For example, the model protection system may include software-based functions implemented by the computer program code PROG running on the processor 12, and hardware-based functions implemented by the hardware circuitry 16.

[0017] Figure 2This is a schematic diagram of a model protection system 20 according to an embodiment of the present invention. System 20 may include a processor (e.g., acting as a...). Figure 1 The processor 12 shown includes a central processing unit (CPU) 250, a transfer interface 230, and direct memory access (DMA) circuitry (e.g., an accelerated processing unit (APU) 240). The processor can be arranged to execute software modules, including a guest VM 200, a host VM 210, and a hypervisor 220. Furthermore, the processor can also be arranged to execute a command hub 224, which is a software module integrated into the hypervisor 220, but the invention is not limited thereto. In some embodiments, the command hub 224 can be a hardware component external to the hypervisor 220 executing on the processor, and the command hub 224 can be... Figure 1 This is a portion of the hardware circuitry 16 shown. The client VM 200 may include a model (e.g., an AI model) 202 and may be configured to send model protection commands (MPCs) to the command hub 224. For example, an operating system (OS), such as Android using a Linux kernel, may run on the client VM 200, where at least one client 201 in the Linux kernel may send model protection commands (MPCs) to the command hub 224. The master VM 210 may include an agent 212, a model protector 214, and a protection manager 216. The command hub 224 may be configured to broadcast model protection commands (MPCs) to the agent 212 and perform communication between the client VM 200 and the master VM 210.

[0018] Agent 212 may be configured to receive Model Protection Command (MPC) sent by Command Hub 224. Model Protector 214 may be configured to receive the Model Protection Command (MPC) from Agent 212 and refer to the MPC to manage and configure protection settings for protected model 203, which is derived from model 202. In this embodiment, model 202 may be injected into the Linux kernel of the Android system for inference, and protected model 203 may be derived from model 202 injected into the Linux kernel. In some embodiments, model 202 may be injected into a shared memory area shared by the Android system running on guest VM 200 and the Linux kernel of the Android system for inference, and protected model 203 may be derived from model 202 injected into the shared memory area. In some embodiments, model 202 may be an encrypted model, and protected model 203 may be generated by decrypting model 202. Further details will be described later with reference to the accompanying drawings.

[0019] Protection manager 216 can be configured to manage and configure security protection components according to protection settings provided by model protector 214, wherein the security protection components may include memory management unit (MMU) 225 and / or memory protection unit (MPU) 226. Furthermore, host VM 210 can be configured to send the security setting command SAFETY_MODEL_COMMAND to hypervisor 220 via transport interface 230. Figure 2 (marked as "SMC" in Chinese).

[0020] The transport interface 230 can be configured to bind the host VM 210 to the hypervisor 220 and perform communication between the host VM 210 and the hypervisor 220. In this way, only the host VM 210 bound to the hypervisor 220 by the transport interface 230 can communicate with the hypervisor 220 (e.g., sending the security setting command SAFETY_MODEL_COMMAND to the hypervisor 220 for subsequent operations), thereby preventing other unauthorized users from using the system 20. It should be noted that the transport interface 230 can be implemented in software or hardware. That is, the transport interface 230 can be a software module executed by the processor, or it can be... Figure 1 A portion of the hardware circuit 16 shown.

[0021] The hypervisor 220 may include a protection engine 222, which may be configured to receive the security setting command SAFETY_MODEL_COMMAND sent by the host VM 210 through the transport interface 230, and to manage and configure security protection components (e.g., MMU 225 and / or MPU 226) according to the security setting command SAFETY_MODEL_COMMAND, to set the read-only mode of the protected model 203 for the APU 240 (in... Figure 2 (marked as "RO" in some embodiments). In some embodiments, the protection engine 222 can also be configured to set a non-read / write mode for the protected model 203 of the CPU 250 by managing and configuring security protection components (e.g., MMU 225 and / or MPU 226) according to the security setting command SAFETY_MODEL_COMMAND, and the protected model 203 cannot be read or written during its lifetime (e.g., the contents of the protected model 203 can be protected). For example, while the CPU 250 is running the protected model 203, the contents of the protected model 203 cannot be read or written, thus protecting the contents of the protected model 203. By setting the protected model 203 of the APU 240 to a read-only mode, the protected model 203 can remain unmodified during its lifetime (e.g., the integrity of the protected model 203 can be protected). For example, when the protected model 203 is running on the APU 240, the protected model 203 cannot be modified, thus protecting its integrity. In this way, since the Android system or Linux kernel can avoid modifying the protected model 203 during inference, other unauthorized users cannot exploit the protected model 203 to attack the APU 240, and other unauthorized users cannot attack the protected model 203 when the APU 240 performs computations on it.

[0022] Figure 3 This is a schematic diagram of a model protection system 30 according to another embodiment of the present invention. System 30 may include a processor (e.g., acting as a...). Figure 1The processor 12 shown includes a CPU 350, a transfer interface 330, and DMA circuitry (e.g., an APU 340). The processor can be configured to execute software modules including a guest VM 300, a host VM 310, and a hypervisor 320, wherein a command hub 321 is a software module integrated into the hypervisor 320. An Android system and a Linux kernel can run on the guest VM 300, which can include a model (e.g., an AI model) 302. At least one client 301 in the Android system can send a model protection command (MPC) to the host VM 310 via the command hub 321. In this embodiment, the model 302 can be injected from the Android system running on the guest VM 300 into a shared memory area for inference, where the shared memory area is a memory area shared by the Android system and the Android system's Linux kernel. For example, the shared memory area 304 can be bound to a specific application (APP), such as a legitimate application of system 30, and only models of that specific APP can be injected into the shared memory area 304.

[0023] The primary VM 310 can be configured to: receive Model Protection Commands (MPCs) from the command hub 321; manage and configure protection settings (i.e., model protection) for the protected model 303, which is derived from model 302 injected into the shared memory area 304, by referring to the Model Protection Commands (MPCs); and manage and configure security protection components (e.g., MMU 360 and / or MPU 361) according to the protection settings. For brevity, Figure 3 The term "model protection" can include the agent, model protector, protection manager, and the aforementioned operations of the host VM 310. Furthermore, the host VM 310 can be configured to send the security setting command SAFETY_MODEL_COMMAND to the hypervisor 320 via the transport interface 330. Figure 3 (marked as "SMC" in the original text). The hypervisor 320 can be configured to receive the security setting command SAFETY_MODEL_COMMAND sent by the host VM 310 via the transport interface 330, and manage and configure security protection components (such as MMU360 and / or MPU 361) according to the security setting command SAFETY_MODEL_COMMAND (for brevity). Figure 3 The “protection manager” marked in the text may include the aforementioned operations of the protection engine and management program 320, to set the read-only mode for the protected model 303 of the APU 340 (in...). Figure 3 (marked as "RO" in Chinese). For the sake of brevity, descriptions similar to those in the previous embodiments are omitted here.

[0024] Figure 4This is a schematic diagram illustrating the relationship between a model protection system according to an embodiment of the present invention and a verifier included in an isolated execution environment. System 40 may include a processor (e.g., acting as...). Figure 1 The processor 12 shown includes a CPU 450, a transfer interface 430, and DMA circuitry (e.g., an APU 440). The processor can be configured to execute software modules including a guest VM 400, a host VM 410, and a hypervisor 420, wherein a command hub 421 is a software module integrated into the hypervisor 420. An Android system and a Linux kernel can run on the guest VM 400, which can include a model (e.g., an AI model) 402 within the Android system. At least one client 401 in the Linux kernel can send Model Protection Commands (MPCs) to the host VM 410 via the command hub 421. In this embodiment, the model 402 can be injected from the Android system into the Linux kernel for inference. The processor can also be configured to execute an isolated execution environment, such as a trusted execution environment (TEE) 404, wherein the TEE 404 may include a verifier 406, and the verifier 406 may be configured to verify the signature of the protected model 403 (the protected model 403 is obtained from the model 402 injected into the Linux kernel of the Android system running on the guest VM 400) to ensure the security 440 of the protected model 403 before the APU executes the computation on the protected model 403.

[0025] The master VM 410 can be configured to: receive model protection commands (MPCs) from the command hub 421; manage and configure protection settings (i.e., model protection) for the protected model 403 based on the model protection commands (MPCs); and manage and configure security protection components (e.g., MMU 460 and / or MPU 461) according to these protection settings. For brevity, Figure 4 The term "model protection" can include the agent, model protector, protection manager, and the aforementioned operations of the host VM 410. Additionally, the host VM 410 can be configured to send the security setting command SAFETY_MODEL_COMMAND to the hypervisor 420 via the transport interface 430. Figure 4 (marked as "SMC" in Chinese).

[0026] The hypervisor 420 can be configured to receive the security setting command SAFETY_MODEL_COMMAND sent by the host VM 410 via the transport interface 430, and manage and configure security protection components (e.g., MMU 460 and / or MPU 461) according to the security setting command SAFETY_MODEL_COMMAND (for brevity, Figure 4 The “protection manager” marked in the text may include the aforementioned operations of the protection engine and management program 420, to set the protected model 403 for APU 440 to read-only mode (in... Figure 4 (marked as "RO" in the original text). The integrity of the protected model 403 can be protected by setting it to read-only mode. In addition to the read-only mode for the protected model 403 of the APU 440, the hypervisor 420 can also be configured to set a non-read / write mode for the protected model of the CPU 450 (in...). Figure 4 (Marked as "no-RW" in the original text). The contents of the protected model 403 can be protected by setting a non-read / write mode for the protected model 403 of the CPU 450. For the sake of brevity, descriptions similar to those in the above embodiments are omitted here.

[0027] Figure 5 This is a schematic diagram illustrating the relationship between a model protection system 50 and a checker included in the host VM according to an embodiment of the present invention. System 50 may include a processor (e.g., acting as...). Figure 1 The processor 12 shown includes a CPU 550, a transfer interface 530, and DMA circuitry (e.g., an APU 540). The processor can be configured to execute software modules including a guest VM 500, a host VM 510, and a hypervisor 520, wherein a command hub 521 is a software module integrated into the hypervisor 520. An Android system and a Linux kernel can run on the guest VM 500, which can include a model (e.g., an AI model) 502 within the Android system. At least one client 501 in the Linux kernel can send model protection commands (MPCs) to the host VM 510 via the command hub 521, and the model 502 can be injected from the Android system into the Linux kernel for inference.

[0028] Figure 4 The system 40 shown is Figure 5The difference in system 50 shown is that, in system 50, the processor (e.g., CPU 550) is modified not to execute an isolated execution environment. At least one client 501 in the Linux kernel can also send model verification commands (MVC) to the host VM 510 via a command hub 521, which is integrated into the hypervisor 520. The host VM 510 may also include a verifier 511, which can be used to verify the signature of the protected model 503 (obtained from model 502 injected into the Linux kernel of the Android system, which runs on the guest VM 500) according to the model verification commands (MVC), to ensure the security of the protected model 503 before the APU 550 performs calculations on the protected model 503. For brevity, descriptions similar to those in the above embodiments are omitted here.

[0029] Figure 6 This is a schematic diagram illustrating the relationship between a model protection system 60 and an isolated execution environment for model decryption according to an embodiment of the present invention. System 60 may include a processor (e.g., acting as...). Figure 1 The processor 12 shown includes a CPU 650, a transfer interface 630, and DMA circuitry (e.g., an APU 640). The processor can be configured to execute software modules including a guest VM 600, a host VM 610, and a hypervisor 620, wherein a command hub 621 is a software module integrated into the hypervisor 620. An Android system and a Linux kernel can run on the guest VM 600, which can include an encrypted model (e.g., an encrypted AI model) 602 from the Android system. At least one client 601 in the Linux kernel can send a model protection command (MPC) 610 to the host VM via the command hub 621. In this embodiment, the encrypted model 602 can be injected from the Android system into the Linux kernel for inference. The processor can also be configured to execute an isolated execution environment, such as a trusted execution environment (TEE) 604, and the TEE 604 can be configured to decrypt the encrypted model 602 to generate a protected model 603.

[0030] The primary VM 610 can be configured to: receive Model Protection Commands (MPCs) from the command hub 621; manage and configure protection settings (e.g., model protection) for the protected model 603 by referring to the MPCs; and manage and configure security protection components (e.g., MMU 660 and / or MPU 661) based on these protection settings. For brevity, Figure 6The term "model protection" can include the agent, model protector, protection manager, and the aforementioned operations of the host VM 610. Additionally, the host VM 610 can be configured to send the security setting command SAFETY_MODEL_COMMAND to the hypervisor 620 via the transport interface 630. Figure 6 (marked as "SMC" in Chinese).

[0031] The hypervisor 620 can be configured to receive the security setting command SAFETY_MODEL_COMMAND sent by the host VM 610 via the transport interface 630, and manage and configure security protection components (such as MMU 660 and / or MPU 661) according to the security setting command SAFETY_MODEL_COMMAND (for brevity). Figure 6 The “protection manager” marked in the text may include the aforementioned operations of the protection engine and management program 620 to set the protected model 603 for APU 640 to read-only mode. Figure 6 (marked as "RO" in the original text). In addition to the read-only mode for the protected model 603 of the APU 640, the hypervisor 620 can also be configured to set the protected model 603 of the CPU 650 to a non-read / write mode (in...). Figure 6 (marked as "no-RW" in the original text). For the sake of brevity, similar descriptions to those in the previous embodiments are omitted here.

[0032] Figure 7 This is a schematic diagram illustrating the relationship between the model protection system 70 and the master VM of the model decryption system according to an embodiment of the present invention. System 70 may include a processor (e.g., acting as a...). Figure 1 The processor 12 shown includes a CPU 750, a transfer interface 730, and DMA circuitry (e.g., an APU 740). The processor can be configured to execute software modules including a guest VM 700, a host VM 710, and a hypervisor 720, wherein a command hub 721 is a software module integrated into the hypervisor 720. An Android system and a Linux kernel can run on the guest VM 700, which can include an encrypted model (e.g., an encrypted AI model) 702. At least one client 701 in the Linux kernel can send model protection commands (MPCs) to the host VM 710 via the command hub 721, and the encrypted model 702 can be injected from the Android system into the Linux kernel for inference.

[0033] Figure 6 The system 60 shown is Figure 7The difference between the systems 70 shown is that in system 70, the processor (e.g., CPU 750) is modified not to execute an isolated execution environment, and at least one client 701 in the Linux kernel can further send model decryption commands (MDC) to the host VM 710 via a command hub 721, which is integrated into the hypervisor 720. The host VM 710 can be equipped with model decryption capabilities and can be configured to decrypt the encrypted model 702 to generate a protected model 703. For brevity, similar descriptions to those in the previous embodiments are omitted here.

[0034] Those skilled in the art will readily observe that many modifications and alterations can be made to the apparatus and method while retaining the teachings of the present invention. Therefore, the foregoing disclosure should be construed as being limited only by the scope and limits of the appended claims.

Claims

1. A model protection system, characterized in that, include: Processor, used to execute: A guest virtual machine (VM), wherein the guest VM includes a model, and the guest VM is used to send at least one command to a command hub; A master VM, wherein the master VM is configured to receive at least one command sent from the command hub and to manage and configure protection settings for a protected model in reference to the at least one command; wherein the protected model is obtained by one of the following methods: from a model injected into the kernel of an operating system running on the guest VM; from a model injected into a shared memory area of ​​the operating system and the kernel; or by decrypting a model that is an encrypted model; and The management program is used to receive security setting commands sent by the master VM, and manage and configure security protection components according to the security setting commands to set the read-only mode of the protected model.

2. The system as described in claim 1, characterized in that, The processor is used to execute the command hub, which is a software module integrated into the management program.

3. The system as described in claim 1, characterized in that, Also includes: The command hub; The command hub is a hardware component external to the management program that executes on the processor.

4. The system as described in claim 1, characterized in that, Also includes: A transmission interface is used to bind the master VM to the management program and to perform communication between the master VM and the management program.

5. The system as described in claim 1, characterized in that, The processor is also used to perform: An isolated execution environment is used to verify the signature of the protected model in order to ensure the security of the protected model before computations on the protected model are performed by the direct memory access DMA circuit.

6. The system according to claim 5 or 1, characterized in that, The management program is also used to set the unreadable / write mode of the protected model by managing and configuring the security protection components according to the security setting commands.

7. The system according to claim 1, characterized in that, The at least one command includes a first command for model protection and a second command for model verification, and the main VM further includes: A verifier is used to verify the signature of the protected model according to the second command to ensure the security of the protected model before the direct memory access DMA circuit performs calculations on the protected model.

8. The system as described in claim 1, characterized in that, When the model is an encrypted model, the processor is also used to perform: An isolated execution environment is used to decrypt the encrypted model to generate the protected model.

9. The system according to claim 1, characterized in that, When the model is an encrypted model, the at least one command includes a first command for model protection and a second command for model decryption, and the master VM is also used to perform decryption on the encrypted model according to the second command to generate the protected model.

10. The system according to claim 1, characterized in that, The security protection components include a memory management unit (MMU) or a memory protection unit (MPU).

11. A non-transitory machine-readable medium for storing program code, wherein when loaded and executed by a processor, the program code instructs the processor to perform: The guest virtual machine (VM) is one of the following: The client VM includes a model, and the client VM is used to send at least one command to the command hub; A master VM, wherein the master VM is configured to receive the at least one command sent from the command hub and to manage and configure protection settings for a protected model with reference to the at least one command; the protected model is obtained by one of the following methods: from a model injected into the kernel of an operating system running on the guest VM; or from a model injected into a shared memory area of ​​the operating system and the kernel; Alternatively, it can be obtained by decrypting the model used as an encryption model; as well as The management program is used to receive security setting commands sent by the master VM, and manage and configure the security protection components according to the security setting commands to set the read-only mode of the protected model.

12. The non-transitory machine-readable medium according to claim 11, characterized in that, The program code further instructs the processor to execute the command hub, which is a software module integrated into the management program.

13. The non-transitory machine-readable medium as claimed in claim 11, characterized in that, The command hub is implemented by hardware components.

14. The non-transitory machine-readable medium according to claim 11, characterized in that, When communication is performed between the master VM and the hypervisor, the master VM is bound to the hypervisor.

15. The non-transitory machine-readable medium according to claim 11, characterized in that, The program code also instructs the processor to execute an isolated execution environment, which is used to verify the signature of the protected model and ensure the security of the protected model before computations on the protected model are performed.

16. The non-transitory machine-readable medium as described in claim 11 or 15, characterized in that, The management program is also used to set the unreadable / write mode of the protected model by managing and configuring the security protection components according to the security setting commands.

17. The non-transitory machine-readable medium according to claim 11, characterized in that, When the model is an encrypted model, the program code further instructs the processor to execute an isolated execution environment, and the isolated execution environment is used to decrypt the encrypted model to generate the protected model.

Images