Model Protection Method and Related Products

Abstract

Embodiments of this application disclose a model protection method and related products. The method includes: determining at least one AI model corresponding to an AI application, where the AI model includes an unencrypted AI model and an encrypted AI model; for each encrypted AI model, establishing a virtual machine corresponding to the encrypted AI model in the secure world of the CPU, establishing an instance corresponding to the encrypted AI model in the GNPU, and establishing a secure channel between the virtual machine and the instance corresponding to the encrypted AI model; for at least one unencrypted AI model, establishing an instance corresponding to the at least one unencrypted AI model in the GNPU, and establishing a normal channel between the at least one unencrypted AI model and the instance between the non-secure world of the CPU and the GNPU. Using the embodiments of this application can improve service efficiency in the case of realizing model protection through multiple instances of the GNPU.

Description

Technical Field

[0001] This application relates to the technical field of model protection, and in particular, to a model protection method and related products. Background Art

[0002] Data has three states throughout its life cycle: At-Rest, In-Transit, and In-Use. If there are active or passive security issues during the use of Artificial Intelligence (AI), the consequences will be unimaginable. Therefore, AI security is a persistent security requirement. Among them, in the protection of AI models, it is necessary to be used in the three states of data.

[0003] In the technologies for implementing AI model protection in the In-Use state, generally, the application and the underlying software on which the application depends are deployed in the Secure World. For example, for the AI models of AI applications such as fingerprint or face unlocking, the tasks can be deployed to the graphics processing unit (GPU) through the hardware-based Trusted Execution Environment (TEE). However, at this time, the GPU can only serve the Secure World. If it is necessary to switch the service to the Normal World, it is necessary to save and restore the context and clean up the running environment, resulting in low efficiency. Summary of the Invention

[0004] The embodiments of this application provide a model protection method and related products, which can utilize the multi-instantiation of GNPU, reuse the computing power of GNPU, and achieve simultaneous service for the Secure World and the Normal World; in the case of implementing model protection through GNPU multi-instances, it is beneficial to improve service efficiency.

[0005] In a first aspect, the embodiments of this application provide a model protection method, which is applied to an electronic device. The electronic device includes a Central Processing Unit (CPU) and a General Neural Network Processing Unit (GNPU). The method includes:

[0006] Determine at least one AI model corresponding to the AI application, where the AI model is applied in the In-use state, and the AI model includes a non-encrypted AI model and an encrypted AI model;

[0007] For each of the encrypted AI models, a virtual machine corresponding to the encrypted AI model is established in the secure world of the CPU, an instance corresponding to the encrypted AI model is established within the GNPU, and a secure channel is established between the virtual machine corresponding to the encrypted AI model and the instance, where the secure channel is used to support data interaction between the virtual machine and the instance;

[0008] For at least one of the non-encrypted AI models, an instance corresponding to the at least one non-encrypted AI model is established within the GNPU, and a normal channel is established between the non-secure world of the CPU and the GNPU for data interaction between the at least one non-encrypted AI model and the instance, where the normal channel is used to support data interaction between the at least one non-encrypted AI model and the instance.

[0009] In a second aspect, an embodiment of the present application provides a model protection device, which is applied to an electronic device, the electronic device includes a central processing unit CPU and a general neural network processor GNPU, and the device includes: a determination unit and a establishment unit, where,

[0010] The determination unit is configured to determine at least one AI model corresponding to an AI application, where the AI model is in the In-use state, and the AI model includes a non-encrypted AI model and an encrypted AI model;

[0011] The establishment unit is configured to, for each of the encrypted AI models, establish a virtual machine corresponding to the encrypted AI model in the secure world of the CPU, establish an instance corresponding to the encrypted AI model within the GNPU, and establish a secure channel between the virtual machine corresponding to the encrypted AI model and the instance, where the secure channel is used to support data interaction between the virtual machine and the instance;

[0012] The establishment unit is further configured to, for at least one of the non-encrypted AI models, establish an instance corresponding to the at least one non-encrypted AI model within the GNPU, and establish a normal channel between the non-secure world of the CPU and the GNPU for data interaction between the at least one non-encrypted AI model and the instance, where the normal channel is used to support data interaction between the at least one non-encrypted AI model and the instance.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and are configured to be executed by the processor, and the programs include instructions for performing the steps in any method of the first aspect of the embodiments of the present application.

[0014] Fourthly, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program for electronic data exchange. The computer program enables a computer to execute some or all of the steps described in any of the methods in the first aspect of the embodiments of the present application.

[0015] Fifthly, an embodiment of the present application provides a computer program product containing instructions. The computer program product includes a non-transitory computer-readable storage medium storing a computer program. When the computer program product runs on an electronic device, it enables the electronic device to execute some or all of the steps described in any of the methods in the first aspect of the embodiments of the present application. The computer program product can be a software installation package.

[0016] It can be seen that in the embodiments of the present application, at least one AI model corresponding to an AI application is determined. The AI model is applied in the In-use state and includes an unencrypted AI model and an encrypted AI model. For each encrypted AI model, a virtual machine corresponding to the encrypted AI model is established in the secure world of the CPU, an instance corresponding to the encrypted AI model is established in the GNPU, and a secure channel is established between the virtual machine corresponding to the encrypted AI model and the instance. The secure channel is used to support data interaction between the virtual machine and the instance. For at least one unencrypted AI model, an instance corresponding to the at least one unencrypted AI model is established in the GNPU, and a normal channel is established between the non-secure world of the CPU and the GNPU for data interaction between the at least one unencrypted AI model and the instance. In this way, the deployment of different AI models can be achieved through the embodiments of the present application. By utilizing the multi-instantiation of the GNPU, the protection of different AI models can be realized, and at the same time, the computing power of the GNPU can be reused to provide services for the Secure World and the Normal World. In the case of realizing model protection through GNPU multi-instances, it is beneficial to improve service efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1It is a schematic structural diagram of a model protection system provided by an embodiment of the present application;

[0019] Figure 2 It is a schematic flowchart of a model protection method provided by an embodiment of the present application;

[0020] Figure 3 It is a schematic structural diagram of a TEE deployed in a CPU provided by an embodiment of the present application;

[0021] Figure 4A It is a schematic structural diagram of a model protection system provided by an embodiment of the present application;

[0022] Figure 4B It is a schematic structural diagram of a model protection system provided by an embodiment of the present application;

[0023] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0024] Figure 6A It is a block diagram of the functional units of a model protection device provided by an embodiment of the present application;

[0025] Figure 6B It is a block diagram of the functional units of a model protection device provided by an embodiment of the present application. Detailed implementation manners

[0026] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0027] The terms "first", "second", etc. in the specification and claims of the present application and the above accompanying drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices.

[0028] References to "embodiments" in this specification mean that a particular feature, structure, or characteristic described in connection with the embodiments can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0029] The electronic device can be a portable electronic device that also includes other functions such as a personal digital assistant and / or music player functions, such as a mobile phone, a tablet computer, a terminal device, a wearable electronic device with wireless communication functions (such as a smart watch, smart glasses), a vehicle-mounted device, etc. Exemplary embodiments of the portable electronic device include, but are not limited to, portable electronic devices running on the IOS system, Android system, Microsoft system, or other operating systems. The above portable electronic devices can also be other portable electronic devices, such as a laptop computer. It should also be understood that in some other embodiments, the above electronic device may not be a portable electronic device, but a desktop computer.

[0030] In the protection of AI models, it is necessary to use them in three states of data. Currently, most of the protection for models exists in the life cycle of two states: at-rest and in-transit. For the technologies that implement AI model protection in the in-use state, or for data protection in the in-use state, due to limitations in the operating environment and performance, they have not been widely applied.

[0031] For the technologies that implement AI model protection in the in-use state, currently, the application and the underlying software it depends on are generally deployed in a secure environment or a secure world. For example, based on the hardware-based Trusted Execution Environment (TEE) or Software Guard Extensions formintel (SGX). These technologies have a high dependence, a large difficulty and workload in transplanting larger models, and in the scenario where the protected AI application frequently interacts between the secure world and the normal world, the performance consumption caused by the switching between the secure world and the normal world is unacceptable in most application scenarios.

[0032] In addition, for example, for an AI model for fingerprint or face unlock AI applications, the task can be deployed to a graphics processing unit (GPU) or a neural network processing unit (NPU) through the TEE. At this time, the GPU or NPU is dedicated to the TEE (deployed in the secure world) and cannot be used by the non-secure world. Moreover, there is no isolation between the AI models running in the NPU at the same time. If there is a malicious model, the models in the coexistence environment will be at risk of being damaged in terms of confidentiality and integrity by the malicious model.

[0033] Furthermore, for the above situation, if it is necessary to switch the service to the non-secure world (Normal World), it is necessary to save and restore the context and clean up the running environment, resulting in low efficiency.

[0034] Therefore, in view of the above problems, the present application proposes a model protection method and related device, which will be described in detail below.

[0035] As Figure 1 shown, it is a schematic structural diagram of a model protection system. As shown in the figure, the model protection system may include: a central processing unit (CPU) and a general neural network processor unit (GNPU).

[0036] Among them, the above-mentioned central processing unit CPU includes a secure world and a non-secure world (Normal World). The TEE can be deployed in the secure world, and the protection of the AI model in the In-use state can be achieved through the TEE, which is beneficial to improving the security level.

[0037] Among them, the above-mentioned GNPU can be used to create multiple instances, and each instance can be used to provide computing power for its corresponding AI model.

[0038] Exemplarily, the AI models can be divided into encrypted AI models and non-encrypted AI models, and virtual machines can be deployed for each encrypted AI model in the TEE.

[0039] In a possible example, an electronic device may determine at least one AI model corresponding to an AI application. Among them, the AI model is applied in the In-use state, and the AI model includes an unencrypted AI model and an encrypted AI model; for each of the encrypted AI models, a virtual machine corresponding to the encrypted AI model is established in the secure world of the CPU, an instance corresponding to the encrypted AI model is established in the GNPU, and a secure channel is established between the virtual machine corresponding to the encrypted AI model and the instance, where the secure channel is used to support data interaction between the virtual machine and the instance; for at least one of the unencrypted AI models, an instance corresponding to the at least one unencrypted AI model is established in the GNPU, and a normal channel is established between the non-secure world of the CPU and the GNPU for data interaction between the at least one unencrypted AI model and the instance, where the normal channel is used to support data interaction between the at least one unencrypted AI model and the instance. In this way, the deployment of different AI models can be achieved through the embodiments of the present application. By using the multi-instantiation of the GNPU, the protection of different AI models can be realized, and at the same time, the computing power of the GNPU can be reused to provide services for the Secure World and the Normal World; in the case of realizing model protection through GNPU multi-instances, it is beneficial to improve service efficiency.

[0040] It should be noted that in this application, "a plurality of" may refer to two or more, and will not be elaborated hereinafter.

[0041] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of a model protection method provided by an embodiment of the present application, applied to an electronic device. The electronic device includes a central processing unit CPU and a general neural network processor GNPU. As shown in the figure, the model protection method includes the following operations.

[0042] S201. Determine at least one AI model corresponding to the AI application. Among them, the AI model is applied in the In-use state, and the AI model includes an unencrypted AI model and an encrypted AI model.

[0043] Among them, the above AI application may include applications such as biometric, autonomous driving, intelligent diagnosis, mathematical modeling, etc., which are not limited here. Specifically, it may include fingerprint recognition, voiceprint recognition, traffic violation recognition, intelligent diagnosis, intelligent obstacle avoidance, natural language generation, digital modeling, emotion recognition, path planning, etc.

[0044] Among them, the above at least one AI model is used to support the implementation of AI applications. In different AI applications, one or more AI models may be included to implement the AI application through the AI model. For example, for AI applications such as autonomous driving or perception, it is necessary to detect obstacles, classify and identify the obstacles, track the vehicle in front, etc. Thus, different functional requirements above can be respectively implemented through a classification AI model, a path planning AI model, a tracking AI model, etc. to complete the implementation of the autonomous driving AI application.

[0045] Among them, the AI model can include two types: non-encrypted AI model and encrypted AI model. For the non-encrypted AI model, it can include open-source models, models with low commercial value, etc., which are not limited here. For the encrypted AI model, it can include third-party models, models with high commercial value, malicious models, etc., which are not limited here.

[0046] Optionally, since the performance of the non-safe world is better than that of the safe world, some AI models with high requirements for computing performance, etc. can also be deployed in the non-safe world.

[0047] Among them, the embodiments of the present application can be applied to the technology of implementing AI model protection in the In-Use state. Of course, it is also applicable to AI models in other states.

[0048] S202. For each of the encrypted AI models, establish a virtual machine corresponding to the encrypted AI model in the secure world of the CPU, establish an instance corresponding to the encrypted AI model in the GNPU, and establish a secure channel between the virtual machine corresponding to the encrypted AI model and the instance, where the secure channel is used to support data interaction between the virtual machine and the instance.

[0049] Among them, a TEE can be deployed in the CPU, and the AI model in the TEE is in the secure world.

[0050] Among them, the electronic device can establish an instance corresponding to each AI model in the GNPU. Of course, the GNPU can also perform sharding processing on one or more instances to evenly use the computing power of the GNPU.

[0051] Exemplarily, as Figure 3 shown, it is a schematic structural diagram of a TEE deployed in the CPU proposed by the present application. In this TEE, multiple virtual machines can be included, and each virtual machine can correspond to an encrypted AI model. For example, virtual machine 1 can be deployed for the first encrypted AI model, virtual machine 2 can be deployed for the second encrypted AI model, and virtual machine 3 can be deployed for the third encrypted AI model. As Figure 4A shown, it is a schematic structural diagram of a model protection system. The model protection system can include asFigure 3 The TEE shown may further include a GNPU, and corresponding instances can be established in the GNPU for each encrypted AI model. For example, corresponding instance 1 is established in the GNPU for the first encrypted AI model mentioned above, corresponding instance 2 is established in the GNPU for the second encrypted AI model mentioned above, and corresponding instance 3 is established in the GNPU for the third encrypted AI model mentioned above.

[0052] Furthermore, for each encrypted AI model, a secure channel can be created between its corresponding virtual machine and instance. That is, a secure channel 1 can be created between corresponding instance 1 and virtual machine 1 for the first encrypted AI model, a secure channel 2 can be created between corresponding instance 2 and virtual machine 2 for the second encrypted AI model, and a secure channel 3 can be created between corresponding instance 3 and virtual machine 3 for the third encrypted AI model.

[0053] Still further, if the first encrypted AI model is a malicious model, isolation between AI models can be achieved through the secure channel and the instance, effectively blocking the attack of the first encrypted AI model on the second encrypted AI model and / or the third encrypted AI model, which is beneficial to ensuring the integrity and confidentiality of the second encrypted AI model and / or the third encrypted AI model.

[0054] It can be seen that in this example, the TEE can be used to protect each encrypted AI model in the In-use state, which is beneficial to improving the security level, and the GNPU can be used to isolate between models, blocking the attack path of malicious models, which is beneficial to ensuring the confidentiality and integrity of AI models in a coexistence environment and avoiding the attack and destruction of malicious models on other encrypted AI models or subsequent non-encrypted AI models.

[0055] S203. For at least one of the non-encrypted AI models, establish an instance corresponding to the at least one non-encrypted AI model in the GNPU, and establish a normal channel between the non-secure world of the CPU and the GNPU for data interaction between the at least one non-encrypted AI model and the instance, where the normal channel is used to support data interaction between the at least one non-encrypted AI model and the instance.

[0056] Among them, the above at least one non-encrypted AI model can be deployed in the non-secure world of the CPU, and an instance of the whole of the at least one non-encrypted AI model is established in the GNPU, and a normal channel is established between the at least one non-encrypted AI model and the GNPU to support data interaction between the at least one non-encrypted AI model and the instance.

[0057] It can be seen that in this example, an encrypted AI model is deployed in the secure world, and a non-encrypted AI model is deployed in the non-secure world. Isolation is also established between different AI models. In particular, deploying a malicious model in the TEE also avoids attacks and damage by the malicious model on the non-encrypted AI model, which is beneficial to maintaining the confidentiality and integrity of the AI model.

[0058] For example, Figure 4B As shown, it is a schematic structural diagram of a model protection system. In addition to including the TEE and GNPU as in Figure 4A , the non-secure world (Normal World) is also included; in this non-secure world, there may be a dependency relationship between at least one non-encrypted AI model. The above at least one non-encrypted AI model can share a common channel to support data interaction between each non-encrypted AI model in the at least one non-encrypted AI model and Instance 4.

[0059] It can be seen that the model protection method described in the embodiments of the present application determines at least one AI model corresponding to the AI application. Among them, the AI model is applied in the In-use state, and the AI model includes a non-encrypted AI model and an encrypted AI model; for each encrypted AI model, a virtual machine corresponding to the encrypted AI model is established in the secure world of the CPU, an instance corresponding to the encrypted AI model is established in the GNPU, and a secure channel is established between the virtual machine corresponding to the encrypted AI model and the instance, where the secure channel is used to support data interaction between the virtual machine and the instance; for at least one non-encrypted AI model, an instance corresponding to the at least one non-encrypted AI model is established in the GNPU, and a common channel is established between the non-secure world of the CPU and the GNPU for data interaction between the at least one non-encrypted AI model and the instance, where the common channel is used to support data interaction between the at least one non-encrypted AI model and the instance. In this way, the deployment of different AI models can be realized through the embodiments of the present application, and the protection of different AI models can be realized by using the multi-instantiation of the GNPU. At the same time, the computing power of the GNPU is reused to provide services for the Secure World and the Normal World; in the case of realizing model protection through GNPU multi-instances, it is beneficial to improve service efficiency.

[0060] In a possible example, the instance corresponding to the encrypted AI model is managed by the TEE in the CPU, and the GNPU driver module of the GNPU is deployed in the TEE.

[0061] Among them, the GNPU driver module of the above GNPU can be deployed in the TEE. The electronic device can realize the fragmentation of resources in the GNPU, or the generation or division of instances through the GNPU driver module in the TEE, so as to realize the management of instances corresponding to the encrypted AI model.

[0062] In a possible example, establishing a virtual machine corresponding to the encrypted AI model in the secure world of the CPU, establishing an instance corresponding to the encrypted AI model in the GNPU, and establishing a secure channel between the virtual machine corresponding to the encrypted AI model and the instance. The above method may include the following steps: triggering the TEE, creating a virtual machine corresponding to the encrypted AI model in the TEE, and deploying the encrypted AI model into the virtual machine; in the virtual machine, decrypting the encrypted AI model to obtain the decrypted target AI model; sending a resource application request to the GNPU through the GNPU driver module; in response to the resource application request, establishing an instance of the encrypted AI model or the target AI model in the GNPU through the GNPU; obtaining the key between the virtual machine corresponding to the encrypted AI model and the secure channel; creating a secure channel between the virtual machine corresponding to the encrypted AI model and the instance according to the key.

[0063] Among them, in order to ensure the privacy of the data of the encrypted AI model, the electronic device can establish a key between the corresponding virtual machine and the corresponding secure channel for each encrypted AI model. This key can be uniquely used to identify the secure channel corresponding to the encrypted AI model, so as to realize the establishment of the secure channel for each encrypted AI model, thereby effectively isolating the data interaction or processing between different encrypted AI models and the GNPU. The setting of this key can be set by the user himself or by the system default, which is not limited here; for example, for different third-party AI models, different keys can be set by the manufacturer or the system itself.

[0064] Among them, the above resource application request is used to apply for fragmented resources or instances in the GNPU for this encrypted AI model.

[0065] It can be seen that in this example, the protection of each encrypted AI model in the In-use state can be realized through the TEE, which is beneficial to improving the security level, blocking the attack path of malicious models, and is beneficial to ensuring the confidentiality and integrity of AI models in the coexistence environment, and avoiding the attack and destruction of malicious models on other encrypted AI models or subsequent non-encrypted AI models. And the computing performance of the encrypted AI model in the TEE is extended by means of the GNPU, which is beneficial to improving the performance under the TEE or in the secure mode. And through the multi-instantiation of the GNPU, the isolation between models is realized, and the computing power of the GNPU can be reused, which is beneficial to realizing the provision of secure world services.

[0066] In a possible example, the non-secure world includes a task scheduling module, and the method may further include the following steps: for multiple non-encrypted AI models, through the task scheduling module, perform topological sorting on the multiple non-encrypted AI models to obtain a topological sorting table, and according to the topological sorting table, perform task scheduling on the multiple non-encrypted AI models, and during the task scheduling process of each non-encrypted AI model, support data interaction between the non-encrypted AI model and the instance through the ordinary channel.

[0067] Among them, in the case of applying this example to multiple non-encrypted AI models, topological sorting can be performed on the multiple non-encrypted AI models, so that the electronic device can implement scheduling for different non-encrypted AI models according to the topological sorting table.

[0068] For example, for AI applications such as autonomous driving or perception, it is necessary to detect obstacles, classify and identify the obstacles, complete path planning, and track the vehicle in front, etc. Thus, the above different functional requirements can be realized through a classification AI model, a path planning AI model, a tracking AI model, etc. In this process, the above three AI models can be deployed in the non-secure world, that is, as non-encrypted AI models, and a topological sorting table between the classification AI model, the path planning AI model, and the tracking AI model can be established, which can instruct the electronic device to schedule the classification AI model first, then the path planning AI model, and finally the tracking AI model to complete tasks such as classification and identification of obstacles, path planning, and tracking of the vehicle in front.

[0069] It can be seen that in this example, topological sorting can be performed on at least one non-encrypted AI model, and the non-encrypted AI models with scheduling requirements share an ordinary channel, which is beneficial to realizing isolation between different non-encrypted AI models.

[0070] In a possible example, for the topological sorting of the multiple non-encrypted AI models, the method may include the following steps: determine the dependency relationship between the multiple non-encrypted AI models; according to the dependency relationship, perform topological sorting on the multiple non-encrypted AI models.

[0071] Among them, the above dependency relationship may refer to a data dependency relationship or a dependency relationship of the model processing order, etc., which is not limited here.

[0072] For example, for AI applications such as autonomous driving or perception, it is necessary to detect obstacles, classify and identify the obstacles, complete path planning, and track the vehicle ahead, etc. It can be seen that there is a model dependency relationship among the classification AI model, the path planning AI model, and the tracking AI model. Therefore, topological sorting can be performed according to this dependency relationship.

[0073] It can be seen that in this example, topological sorting can be performed on multiple non-encrypted AI models with dependency relationships, and during the use of each non-encrypted AI model, data interaction between the non-encrypted AI model and the corresponding instance can be achieved through the same ordinary channel, which is beneficial to saving the computing power of the GNPU.

[0074] Optionally, when there is no dependency relationship among the above-mentioned multiple non-encrypted AI models, a non-encrypted AI model can be randomly selected first for interaction with the instance to support the computing power of the CPU through the GNPU. Further, if there is a dependency relationship among any N non-encrypted AI models among the above-mentioned M non-encrypted AI models, where M is a positive integer greater than 2 and N is a positive integer greater than or equal to 2, the above-mentioned M non-encrypted AI models can also be classified so that the N non-encrypted AI models with dependency relationships form a group, and the remaining non-encrypted AI models are divided into another group. The N non-encrypted AI models are preferentially deployed in the non-secure world. After the N non-encrypted AI models are used up, the remaining non-encrypted AI models are added to the non-secure world. If the remaining non-encrypted AI models need to be deployed in advance or run simultaneously with the N non-encrypted AI models, the remaining non-encrypted AI models can be deployed in the TEE to complete the joint deployment and operation of the M non-encrypted AI models to ensure the normal use of the AI applications corresponding to the M non-encrypted AI models.

[0075] In a possible example, the GNPU provides computing power support services for both the TEE or the secure world and the non-secure world at the same time.

[0076] Please refer to Figure 5 , Figure 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As shown in the figure, the electronic device includes a processor, a memory, a communication interface, and one or more programs, which are applied to the electronic device. The electronic device includes a central processing unit CPU and a general neural network processor GNPU. Among them, the above one or more programs are stored in the above memory, and the above one or more programs are configured with instructions for the above processor to execute the following steps:

[0077] Determine at least one AI model corresponding to the AI application, where the AI model is in the In-use state, and the AI model includes a non-encrypted AI model and an encrypted AI model;

[0078] For each of the encrypted AI models, a virtual machine corresponding to the encrypted AI model is established in the secure world of the CPU, an instance corresponding to the encrypted AI model is established within the GNPU, and a secure channel is established between the virtual machine corresponding to the encrypted AI model and the instance, where the secure channel is used to support data interaction between the virtual machine and the instance;

[0079] For at least one of the non-encrypted AI models, an instance corresponding to the at least one non-encrypted AI model is established within the GNPU, and a normal channel is established between the non-secure world of the CPU and the GNPU for data interaction between the at least one non-encrypted AI model and the instance, where the normal channel is used to support data interaction between the at least one non-encrypted AI model and the instance.

[0080] It can be seen that the electronic device described in the embodiments of the present application determines at least one AI model corresponding to an AI application, where the AI model is applied in the In-use state, and the AI model includes a non-encrypted AI model and an encrypted AI model; for each of the encrypted AI models, a virtual machine corresponding to the encrypted AI model is established in the secure world of the CPU, an instance corresponding to the encrypted AI model is established within the GNPU, and a secure channel is established between the virtual machine corresponding to the encrypted AI model and the instance, where the secure channel is used to support data interaction between the virtual machine and the instance; for at least one of the non-encrypted AI models, an instance corresponding to the at least one non-encrypted AI model is established within the GNPU, and a normal channel is established between the non-secure world of the CPU and the GNPU for data interaction between the at least one non-encrypted AI model and the instance, where the normal channel is used to support data interaction between the at least one non-encrypted AI model and the instance. In this way, the deployment of different AI models can be achieved through the embodiments of the present application, and the protection of different AI models can be realized by using the multi-instantiation of the GNPU. At the same time, the computing power of the GNPU can be reused to provide services for the Secure World and the Normal World; in the case of realizing model protection through GNPU multi-instances, it is beneficial to improve service efficiency.

[0081] In a possible example, the instance corresponding to the encrypted AI model is managed by the TEE in the CPU, and the GNPU driver module of the GNPU is deployed in the TEE.

[0082] In a possible example, in establishing a virtual machine corresponding to the encrypted AI model in the secure world of the CPU, establishing an instance corresponding to the encrypted AI model within the GNPU, and establishing a secure channel between the virtual machine corresponding to the encrypted AI model and the instance, the above program includes instructions for performing the following steps:

[0083] Trigger the TEE, create a virtual machine corresponding to the encrypted AI model in the TEE, and deploy the encrypted AI model into the virtual machine;

[0084] In the virtual machine, decrypt the encrypted AI model to obtain the decrypted target AI model;

[0085] Send a resource application request to the GNPU through the GNPU driver module;

[0086] In response to the resource application request, establish an instance of the encrypted AI model or the target AI model within the GNPU through the GNPU;

[0087] Obtain the key between the virtual machine corresponding to the encrypted AI model and the secure channel;

[0088] Create a secure channel between the virtual machine corresponding to the encrypted AI model and the instance according to the key.

[0089] In a possible example, the non-secure world includes a task scheduling module, and the above program further includes instructions for performing the following steps:

[0090] For multiple non-encrypted AI models, through the task scheduling module, perform topological sorting on the multiple non-encrypted AI models to obtain a topological sorting table, and according to the topological sorting table, perform task scheduling on the multiple non-encrypted AI models, and during the task scheduling process of each non-encrypted AI model, support data interaction between the non-encrypted AI model and the instance through the ordinary channel.

[0091] In a possible example, in performing topological sorting on the multiple non-encrypted AI models, the above program further includes instructions for performing the following steps:

[0092] Determine the dependency relationship between the multiple non-encrypted AI models;

[0093] Perform topological sorting on the multiple non-encrypted AI models according to the dependency relationship.

[0094] In a possible example, the GNPU provides computing power support services for both the TEE or the secure world and the non-secure world simultaneously.

[0095] The above mainly introduced the solution of the embodiment of the present application from the perspective of the execution process on the method side. It can be understood that in order for an electronic device to implement the above functions, it includes the corresponding hardware structure and / or software module for executing each function. Those skilled in the art should easily realize that, combining the units and algorithm steps of each example described in the embodiments provided in this article, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0096] The embodiment of the present application can divide the functional units of the electronic device according to the above method examples. For example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. It should be noted that the division of units in the embodiment of the present application is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0097] In the case of dividing each functional module corresponding to each function, Figure 6A The schematic diagram of the model protection device is shown, as Figure 6A shown, the device is applied to an electronic device, the electronic device includes a central processing unit CPU and a general neural network processor GNPU, and the model protection device 600 may include: a determination unit 601 and a establishment unit 602, wherein,

[0098] The determination unit 601 is configured to determine at least one AI model corresponding to an AI application, wherein the AI model is applied in the In-use state, and the AI model includes an unencrypted AI model and an encrypted AI model;

[0099] The establishment unit 602 is configured to, for each of the encrypted AI models, establish a virtual machine corresponding to the encrypted AI model in the secure world of the CPU, establish an instance corresponding to the encrypted AI model in the GNPU, and establish a secure channel between the virtual machine corresponding to the encrypted AI model and the instance, wherein the secure channel is used to support data interaction between the virtual machine and the instance;

[0100] The establishing unit 602 is further configured to, for at least one of the non-encrypted AI models, establish an instance corresponding to the at least one non-encrypted AI model within the GNPU, and establish a normal channel between the non-secure world of the CPU and the GNPU for the at least one non-encrypted AI model and the instance, where the normal channel is used to support data interaction between the at least one non-encrypted AI model and the instance.

[0101] It can be seen that the model protection device provided by the embodiment of the present application determines at least one AI model corresponding to the AI application, where the AI model is applied in the In-use state, and the AI model includes a non-encrypted AI model and an encrypted AI model; for each of the encrypted AI models, a virtual machine corresponding to the encrypted AI model is established in the secure world of the CPU, an instance corresponding to the encrypted AI model is established within the GNPU, and a secure channel is established between the virtual machine corresponding to the encrypted AI model and the instance, where the secure channel is used to support data interaction between the virtual machine and the instance; for at least one of the non-encrypted AI models, an instance corresponding to the at least one non-encrypted AI model is established within the GNPU, and a normal channel is established between the non-secure world of the CPU and the GNPU for the at least one non-encrypted AI model and the instance, where the normal channel is used to support data interaction between the at least one non-encrypted AI model and the instance. In this way, the deployment of different AI models can be achieved through the embodiment of the present application, and the protection of different AI models can be realized by using the multi-instantiation of the GNPU. At the same time, the computing power of the GNPU is reused to provide services for the Secure World and the Normal World; in the case of realizing model protection through GNPU multi-instances, it is beneficial to improve service efficiency.

[0102] In a possible example, in terms of establishing a virtual machine corresponding to the encrypted AI model in the secure world of the CPU, establishing an instance corresponding to the encrypted AI model within the GNPU, and establishing a secure channel between the virtual machine corresponding to the encrypted AI model and the instance, the above-mentioned establishing unit 602 is specifically configured to:

[0103] Trigger the TEE, create a virtual machine corresponding to the encrypted AI model in the TEE, and deploy the encrypted AI model into the virtual machine;

[0104] In the virtual machine, decrypt the encrypted AI model to obtain the decrypted target AI model;

[0105] Send a resource application request to the GNPU through the GNPU driver module;

[0106] In response to the resource application request, an instance of the encrypted AI model or the target AI model is established within the GNPU by the GNPU;

[0107] Obtain the key between the virtual machine corresponding to the encrypted AI model and the secure channel;

[0108] According to the key, create a secure channel between the virtual machine corresponding to the encrypted AI model and the instance.

[0109] In a possible example, the non-secure world includes a task scheduling module, and the establishing unit 602 is specifically further configured to:

[0110] For multiple non-encrypted AI models, through the task scheduling module, perform topological sorting on the multiple non-encrypted AI models to obtain a topological sorting table, and according to the topological sorting table, perform task scheduling on the multiple non-encrypted AI models, and during the task scheduling process of each non-encrypted AI model, support data interaction between the non-encrypted AI model and the instance through the ordinary channel.

[0111] In a possible example, Figure 6B shows a schematic diagram of the model protection device, as Figure 6B shown, on the basis of Figure 6A The model protection device 600 may further include: a sorting unit 603. In terms of performing topological sorting on the multiple non-encrypted AI models, the above sorting unit 603 is configured to:

[0112] Determine the dependency relationship between the multiple non-encrypted AI models;

[0113] According to the dependency relationship, perform topological sorting on the multiple non-encrypted AI models.

[0114] It should be noted that all relevant contents of each step involved in the above method embodiment can be cited in the function description of the corresponding functional module, and will not be repeated here.

[0115] The electronic device provided in this embodiment is used to execute the above model protection method, and thus can achieve the same effect as the above implementation method.

[0116] In the case of adopting an integrated unit, the electronic device may include a processing module, a storage module, and a communication module. Among them, the processing module may be used to control and manage the operations of the electronic device. For example, it may be used to support the electronic device in executing the steps performed by the above-mentioned network structure determination unit 601, establishment unit 602, and sorting unit 603. The storage module may be used to support the electronic device in executing storage of program codes and data, etc. The communication module may be used to support the communication of the electronic device with other devices.

[0117] Among them, the processing module may be a processor or a controller. It may implement or execute various exemplary logic blocks, modules, and circuits described in combination with the disclosure of the present application. The processor may also be a combination that realizes computing functions, such as a combination including one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, and so on. The storage module may be a memory. The communication module may specifically be a device for interacting with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0118] An embodiment of the present application further provides a computer storage medium. Among them, the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any of the methods described in the above method embodiments. The above computer includes an electronic device.

[0119] An embodiment of the present application further provides a computer program product containing instructions. The above computer program product includes a non-transitory computer-readable storage medium storing a computer program. When the computer program product runs on an electronic device, it enables the electronic device to execute part or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the above computer includes an electronic device.

[0120] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps may be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0121] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0122] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical or other form.

[0123] The units described above as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0125] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the above-mentioned methods in each embodiment of the present application. And the aforementioned memory includes: USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks or optical disks and other various media that can store program codes.

[0126] Those of ordinary skill in the art can understand that all or part of the steps in the above-mentioned various methods can be completed by instructing relevant hardware through a program. This program can be stored in a computer-readable memory. The memory can include: flash drives, read-only memories, random access memories, magnetic disks or optical disks, etc.

[0127] The above has introduced the embodiments of the present application in detail. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A model protection method, applied to an electronic device, characterized in that The electronic device includes a Central Processing Unit (CPU) and a General Neural Network Processing Unit (GNPU), and the method includes: Determine at least one AI model corresponding to an AI application, where the AI model is applied in the In-use state, and the AI model includes an unencrypted AI model and an encrypted AI model; For each of the encrypted AI models, create a virtual machine corresponding to the encrypted AI model in the secure world of the CPU, create an instance corresponding to the encrypted AI model within the GNPU, and establish a secure channel between the virtual machine corresponding to the encrypted AI model and the instance corresponding to the encrypted AI model, where the secure channel is used to support data interaction between the virtual machine and the instance corresponding to the encrypted AI model; For at least one of the unencrypted AI models, create an instance corresponding to the at least one unencrypted AI model within the GNPU, and establish a normal channel between the non-secure world of the CPU and the GNPU for data interaction between the at least one unencrypted AI model and the instance corresponding to the unencrypted AI model, where the normal channel is used to support data interaction between the at least one unencrypted AI model and the instance corresponding to the unencrypted AI model.

2. The method according to claim 1, characterized in that, The instance corresponding to the encrypted AI model is managed by the Trusted Execution Environment (TEE) in the CPU, and the GNPU driver module of the GNPU is deployed in the TEE.

3. The method according to claim 2, wherein The step of creating a virtual machine corresponding to the encrypted AI model in the secure world of the CPU, creating an instance corresponding to the encrypted AI model within the GNPU, and establishing a secure channel between the virtual machine corresponding to the encrypted AI model and the instance corresponding to the encrypted AI model includes: Trigger the TEE, create a virtual machine corresponding to the encrypted AI model in the TEE, and deploy the encrypted AI model into the virtual machine; In the virtual machine, decrypt the encrypted AI model to obtain the decrypted target AI model; Send a resource application request to the GNPU through the GNPU driver module; In response to the resource application request, create an instance of the encrypted AI model or the target AI model within the GNPU through the GNPU; Obtain the key between the virtual machine corresponding to the encrypted AI model and the secure channel; Create a secure channel between the virtual machine corresponding to the encrypted AI model and the instance corresponding to the encrypted AI model according to the key.

4. The method according to claim 1, wherein The non-secure world includes a task scheduling module, and the method further includes: For multiple unencrypted AI models, perform topological sorting on the multiple unencrypted AI models through the task scheduling module to obtain a topological sorting table, and perform task scheduling on the multiple unencrypted AI models according to the topological sorting table. During the task scheduling process of each unencrypted AI model, support data interaction between the unencrypted AI model and the instance corresponding to the unencrypted AI model through the normal channel.

5. The method according to claim 4, characterized in that, The step of performing topological sorting on the multiple unencrypted AI models includes: Determine the dependency relationships among the multiple non-encrypted AI models; According to the dependency relationships, perform topological sorting on the multiple non-encrypted AI models.

6. The method according to any one of claims 1-5, characterized in that, The GNPU simultaneously provides computing power support services for both the TEE or the secure world and the non-secure world.

7. A model protection device, characterized in that, The device is applied to an electronic device, the electronic device includes a central processing unit CPU and a general neural network processor GNPU, and the device includes: a determination unit and an establishment unit, where The determination unit is configured to determine at least one AI model corresponding to an AI application, where the AI model is in the In-use state, and the AI model includes a non-encrypted AI model and an encrypted AI model; The establishment unit is configured to, for each of the encrypted AI models, establish a virtual machine corresponding to the encrypted AI model in the secure world of the CPU, establish an instance corresponding to the encrypted AI model within the GNPU, and establish a secure channel between the virtual machine corresponding to the encrypted AI model and the instance corresponding to the encrypted AI model, where the secure channel is used to support data interaction between the virtual machine and the instance corresponding to the encrypted AI model; The establishment unit is further configured to, for at least one of the non-encrypted AI models, establish an instance corresponding to the at least one non-encrypted AI model within the GNPU, and establish a normal channel between the non-secure world of the CPU and the GNPU for data interaction between the at least one non-encrypted AI model and the instance corresponding to the non-encrypted AI model.

8. An electronic device, characterized in that, Comprising a processor, a memory, a communication interface, and one or more programs, the one or more programs are stored in the memory and are configured to be executed by the processor, and the programs include instructions for performing the steps in the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Store a computer program for electronic data exchange, where the computer program causes a computer to execute the method according to any one of claims 1-6.

10. A computer program product comprising instructions, characterized in that, When the computer program product runs on an electronic device, the electronic device is caused to execute the method according to any one of claims 1-6.

Images