Model reasoning method, model dynamic encryption method and device

Abstract

The invention discloses a model reasoning method, a model dynamic encryption method and a model dynamic encryption device, and belongs to the technical field of data processing. The model reasoning method comprises the steps of obtaining a model reasoning request, wherein the model reasoning request is used for requesting a first model to perform model reasoning; obtaining a target model fragment related to the model reasoning request and fragment attribute information of the target model fragment from a trusted execution environment (TEE) storage space, wherein the fragment attribute information is used for indicating at least one of the following items of a calculation layer included in the corresponding model fragment: a layer type, a calculation amount and sensitivity to private data; determining a first model fragment and a second model fragment according to fragment attribute information of the target model fragment and system state information of the electronic equipment; transmitting the second model fragment to a rich execution environment (REE); and based on a scheduler between the TEE and the REE, calling the first model fragment in the TEE, and calling the second model fragment in the REE to execute a processing process corresponding to the model reasoning request.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, specifically relating to a model reasoning method, a dynamic encryption method for models, and an apparatus. Background Technology

[0002] With the rapid development of large-scale model technology in artificial intelligence (AI), more and more mobile electronic devices are beginning to incorporate related applications. At the same time, model protection and privacy protection are receiving increasing attention.

[0003] In related technologies, large-scale AI model inference on mobile devices requires deploying the complete model on electronic devices. In most scenarios, the model files are statically encrypted to protect the model assets, such as using algorithms like Advanced Encryption Standard (AES) to encrypt the complete model files, and then decrypting and loading the entire file into memory at runtime.

[0004] In addition, in a few scenarios, the model is deployed into a Trusted Execution Environment (TEE) for deeper protection.

[0005] Among them, the method of statically encrypting model files still allows attackers to obtain model files from memory, rendering the protection mechanism ineffective. While deploying the model into a TEE provides some protection for the model, it hinders the efficient use of the overall large model. Summary of the Invention

[0006] The purpose of this application is to provide a model inference method, a dynamic encryption method and apparatus for models, which can solve the problems in related technologies where attackers can still obtain model files from memory and the overall large model has low utilization efficiency.

[0007] In a first aspect, embodiments of this application provide a model reasoning method applied to an electronic device, the method comprising: Obtain a model inference request, the model inference request being used to request a first model to perform model inference, the first model comprising at least two model fragments; Obtain the target model fragment and the fragment attribute information of the target model fragment related to the model inference request from the Trusted Execution Environment (TEE) storage space. The target model fragment includes a partial model fragment of the first model. The fragment attribute information is used to indicate at least one of the following of the computation layers contained in the corresponding model fragment: layer type, computational load, and sensitivity to private data. Based on the fragmentation attribute information of the target model fragment and the system state information of the electronic device, the first model fragment and the second model fragment are determined; The second model fragment is transferred to the Rich Execution Environment (REE). Based on the scheduler between the TEE and the REE, the first model shard is invoked in the TEE and the second model shard is invoked in the REE to execute the processing procedure corresponding to the model inference request.

[0008] Secondly, embodiments of this application provide a dynamic encryption method for a model, applied to a server, the method comprising: Obtain the model file for the first model; The model file is parsed to obtain the directed acyclic graph (DAG) of the first model, wherein the DAG includes at least two nodes and directed edges between the nodes, and the nodes correspond to the computational layers in the first model; A partitioning candidate set is determined based on the DAG, wherein the partitioning candidate set includes at least one partitioning scheme, each partitioning scheme includes at least two model partitions obtained by the first model, and partitioning attribute information of each of the at least two model partitions; the partitioning attribute information is used to indicate at least one of the following of the computational layers contained in the corresponding model partition: layer type, computational load, and sensitivity to private data; Obtain system status information from electronic devices; Based on the system status information, a target sharding scheme is determined from the sharding candidate set; Each model fragment in the target fragmentation scheme is sent to the electronic device.

[0009] Thirdly, embodiments of this application provide a model inference apparatus for an electronic device, the apparatus comprising: The first acquisition module is used to acquire a model inference request, the model inference request being used to request a first model to perform model inference, the first model including at least two model fragments; The second acquisition module is used to acquire the target model fragment and the fragment attribute information of the target model fragment related to the model inference request from the Trusted Execution Environment (TEE) storage space. The target model fragment includes a partial model fragment of the first model. The fragment attribute information is used to indicate at least one of the following of the computing layers contained in the corresponding model fragment: layer type, computational load, and sensitivity to private data. The first determining module is used to determine the first model fragment and the second model fragment based on the fragmentation attribute information of the target model fragment and the system state information of the electronic device; The first transmission module is used to transmit the second model fragment to the rich execution environment REE; The scheduling module is used to call the first model shard in the TEE and the second model shard in the REE based on the scheduler between the TEE and the REE, so as to execute the processing procedure corresponding to the model inference request.

[0010] Fourthly, embodiments of this application provide a dynamic encryption device for a model, used on a server, the device comprising: The sixth acquisition module is used to acquire the model file of the first model; The second parsing module is used to parse the model file to obtain the directed acyclic graph (DAG) of the first model, wherein the DAG includes at least two nodes and directed edges between the nodes, and the nodes correspond to the computation layers in the first model. The fourth determining module is used to determine a partitioning candidate set based on the DAG, wherein the partitioning candidate set includes at least one partitioning scheme, each partitioning scheme includes at least two model partitions obtained by the first model, and partitioning attribute information of each of the at least two model partitions; the partitioning attribute information is used to indicate at least one of the following of the computational layers contained in the corresponding model partition: layer type, computational load, and sensitivity to private data; The seventh acquisition module is used to acquire system status information of the electronic device; The fifth determining module is used to determine the target sharding scheme from the sharding candidate set based on the system status information; The third processing module is used to encrypt each model fragment in the target sharding scheme and send each model fragment in the target sharding scheme to the electronic device.

[0011] Fifthly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the first aspect.

[0012] In a sixth aspect, embodiments of this application provide a server including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the second aspect.

[0013] In a seventh aspect, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first or second aspect.

[0014] Eighthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the methods described in the first or second aspect.

[0015] Ninthly, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method as described in the first or second aspect.

[0016] In this embodiment, the first model is first divided into at least two model fragments and stored in the TEE storage space. One part of the model fragments, the first model fragment, is invoked within the TEE, while the other part, the second model fragment, is invoked within the REE. Thus, when responding to model inference requests on the electronic device, on the one hand, since it is not necessary to invoke the complete first model within the TEE, the computational resources consumed by the TEE are reduced, which is beneficial for improving the execution efficiency of the first model fragment. On the other hand, since each model fragment is pre-stored in the TEE storage space, it is difficult for attackers to obtain the model file from the TEE storage space. Moreover, during the execution of model inference, only the second model fragment needs to be invoked in the REE, without needing to invoke the complete first model within the REE, ensuring that the first model is not fully exposed in the REE, and attackers cannot obtain the complete model file of the first model from the REE. In summary, this embodiment can improve the security and efficiency of the first model. Attached Figure Description

[0017] Figure 1 This is a flowchart of a model reasoning method provided in an embodiment of this application; Figure 2 This is a flowchart of another model reasoning method provided in the embodiments of this application; Figure 3 This is a flowchart of a dynamic encryption method for a model provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a model inference device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a dynamic encryption device for a model provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0019] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0020] It should be noted that the model inference method and the dynamic encryption method for the model provided in this application embodiment have multiple application scenarios, including but not limited to at least one of the following: IoT edge computing: used for security model updates between distributed devices; Medical impact analysis: used for privacy-preserving inference in accordance with the Health Insurance Portability and Accountability Act (HIPAA) standards; Autonomous driving systems: Safety assurance for multi-sensor data fusion; Financial transaction verification: a federated learning system that incorporates blockchain.

[0021] The model inference method, model dynamic encryption method, model inference device, and model dynamic encryption device provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0022] It should be noted that the execution subject of the model reasoning method in this application embodiment is an electronic device, which can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope.

[0023] See Figure 1 The model reasoning method provided in this application includes the following steps: Step 101: Obtain a model inference request. The model inference request is used to request the first model to perform model inference. The first model includes at least two model fragments.

[0024] Step 102: Obtain the target model shard and the shard attribute information of the target model shard related to the model inference request from the Trusted Execution Environment (TEE) storage space. The target model shard includes a partial model shard of the first model. The shard attribute information is used to indicate at least one of the following of the computation layers contained in the corresponding model shard: layer type, computational load, and sensitivity to private data.

[0025] In some implementations, the target model shard can be a partial shard of the first model. For example, the first model can be divided into 10 shards, but only 5 of them are needed to perform the inference process related to the model inference request. In this way, the remaining model shards that are not related to the corresponding model inference request continue to be stored in the TEE storage space, and the attacker cannot obtain the complete first model during the model inference process.

[0026] Optionally, the target model shard represents the model shards that are relied upon when performing model inference request-related processing. For example, assuming the first model is a sequence model, the model shards are loaded in the chronological order of the data stream for performing model inference request-related processing; for a graph model, the model shards are loaded in the topological order of the data stream for performing model inference request-related processing.

[0027] In some implementations, the sizes of at least two model slices obtained from the first model partitioning are non-uniform. For example, in a Convolutional Neural Network (CNN) model, convolutional layers and pooling layers may be combined into one model slice, while fully connected layers are treated as a separate model slice. This non-uniform partitioning increases the randomness of each model slice, making it more difficult for attackers to infer individual model slices and the overall model, thereby improving the security performance of the model slices.

[0028] In some implementations, the sharding scheme of the first model can be derived by analyzing the logical architecture of the first model.

[0029] For example, each computational layer in the first model, such as convolutional layers, fully connected layers, and activation functions, is treated as a node in a Directed Acyclic Graph (DAG). Directed edges are constructed between DAG nodes based on the logical relationships between the computational layers, thus forming the DAG graph of the first model. Meta-information of the DAG nodes is recorded, such as layer type, parameter size, input / output tensor dimensions, and whether hardware acceleration is supported. Data dependency types between DAG nodes are recorded based on the directed edges, such as direct dependencies, branch merging, and residual connections. Communication bandwidth requirements for data transmission between DAG nodes are also recorded based on the directed edges, such as the amount of data transmitted across shards. Then, model topology analysis is performed on the DAG graph to derive the sharding scheme for the first model. This includes analyzing the characteristics and dependencies of the computational layers based on the DAG graph. These characteristics include computational layer type (e.g., linear computation, nonlinear activation), parameter quantity (e.g., weight matrix size), connection density (e.g., number of input / output edges), and computation latency (estimated execution time on the target device). Finally, graph analysis algorithms are used to identify the boundaries where model sharding can be performed to determine the sharding scheme.

[0030] In some implementations, there may be at least two candidate sharding schemes, each of which divides the DAG graph of the first model into multiple subgraphs. Finally, one of them can be selected as the final sharding scheme based on the actual system state of the electronic device and the requirements of the model inference request, and the target model shard can be selected from that sharding scheme.

[0031] For example, performing topology analysis on a DAG graph yields a candidate set of partitioning schemes for the first model. This candidate set includes at least two partitioning schemes, each of which divides the DAG graph of the first model into multiple subgraphs. In this case, if there are many idle computing resources in the TEE of the electronic device, the selected partitioning scheme can partition the computational layer with a large amount of computation and the computational layer involving private data processing into the same model partition. If there are few idle computing resources in the TEE of the electronic device, the selected partitioning scheme can partition the computational layer with a large amount of computation and the computational layer involving private data processing into different model partitions.

[0032] In some implementations, the fragmentation attribute information can be determined based on the metadata of the DAG nodes contained in the corresponding model fragment, the data dependency type between DAG nodes recorded by the directed edges in the model fragment, and the communication bandwidth requirements for data transmission between DAG nodes. Based on the layer type, computational load, and sensitivity to private data of the model fragment indicated by the fragmentation attribute information, it can be determined in step 103 whether the model fragment is called in TEE or REE.

[0033] Step 103: Determine the first model fragment and the second model fragment based on the fragmentation attribute information of the target model fragment and the system status information of the electronic device.

[0034] It is worth noting that the target model fragment includes the first model fragment and the second model fragment.

[0035] In some implementations, model fragments with high sensitivity to private data and low computational cost can be preferentially called in TEE; while model fragments with low sensitivity to private data and high computational cost can be preferentially called in REE.

[0036] In other implementations, the layer type of the model fragment invoked in the TEE can be specified, or the layer type of the model fragment invoked in the REE can be specified.

[0037] In some implementations, model fragments called in the TEE and REE can be dynamically allocated based on the system status information of the electronic device, such as TEE memory availability, available safe memory size, CPU utilization, CPU utilization of REE and TEE respectively, network latency, and system status such as cloud interaction and battery power.

[0038] For example, when there is sufficient TEE memory and CPU idle time, model fragments that are highly sensitive to private data and model fragments with low computational load can be called in the TEE; when there is less TEE memory and CPU idle time, only model fragments that are highly sensitive to private data can be called in the TEE, and all remaining model fragments that do not involve private data processing can be called in the REE.

[0039] Step 104: Transmit the second model fragment to the REE.

[0040] In some implementations, the ciphertext and key of the second model fragment can be transmitted to the REE, and the decryption and retrieval of the second model fragment can be completed in the REE. This can improve the data security of the second model fragment transmitted between the TEE and the REE.

[0041] Of course, the second model fragment can also be transmitted from TEE to REE in plaintext.

[0042] In some implementations, model fragments and their attribute information can be encrypted independently on a fragment-by-fragment basis, with each fragment using a unique key. The encrypted ciphertext and the corresponding key are then stored in the TEE storage space. This further enhances the information security of the first model.

[0043] As an optional implementation, obtaining the target model shard and its shard attribute information related to the model inference request from the Trusted Execution Environment (TEE) storage space includes: Retrieve the ciphertext of the target model fragment related to the model inference request from the Trusted Execution Environment (TEE) storage space; The target model fragment is decrypted within the TEE to obtain the plaintext of the target model fragment and the fragment attribute information of the target model fragment; The step of transmitting the second model fragment to the REE includes: The ciphertext of the second model fragment is transmitted from the TEE to the REE; The ciphertext of the second model fragment is decrypted in the REE to obtain the plaintext of the second model fragment.

[0044] In some implementations, the key used to decrypt the model fragment ciphertext can be accessed through a secure application programming interface (API) such as TEE, reducing the risk of the key and the model fragment ciphertext being exported.

[0045] In some implementations, when the TEE transmits the ciphertext of the second model fragment to the REE, it can also transmit the corresponding key to the REE, or send the calculation method or calculation parameters of the corresponding key to the REE. In this way, the REE can use the key to decrypt the ciphertext of the second model fragment and perform calculations based on the plaintext of the second model fragment.

[0046] Optionally, a scheduler can ensure secure data transmission between the TEE and REE, for example, by using temporary encrypted channels, such as session key-based encryption, to encrypt data before transmission and decrypt it at the receiving end. Furthermore, tasks executed within the REE employ data encryption and code segment integrity verification throughout their execution. This enables secure protection and attack detection of the second model fragments within the REE.

[0047] In this embodiment, when performing step 103, the ciphertext of the target model fragment is decrypted in the TEE to obtain the plaintext of the target model fragment and the fragmentation attribute information of the target model fragment; when performing step 104, the ciphertext and key of the second model fragment can be transmitted to the REE through encryption, such as session encryption, and the ciphertext of the second model fragment is decrypted in the REE to obtain the plaintext of the second model fragment and the fragmentation attribute information of the second model fragment. In this way, the security of each model fragment of the first model in the TEE can be improved, and the security of the second model fragment transmitted between the TEE and the REE can also be improved.

[0048] Step 105: Based on the scheduler between the TEE and the REE, the first model shard is invoked in the TEE and the second model shard is invoked in the REE to execute the processing procedure corresponding to the model inference request.

[0049] In some implementations, the scheduler can serialize the target model fragments into inference tasks based on the fragmentation attribute information of the target model fragments and the system states of the TEE and REE environments, construct a heterogeneous pipeline between the TEE and REE, and optimize parallelism. For example, if the output of model fragment A is the input of model fragment B, then model fragment B will be arranged after model fragment A in the inference task sequence. The main tasks of the TEE include model fragment decryption, lightweight computation and security verification, and invocation of model fragments highly sensitive to private information. Specific computations may include model forward propagation, activation function calculation, etc. The main tasks of the REE include invocation of computationally intensive computation layers.

[0050] For example, based on the fragmentation attribute information of the model fragment, the sensitivity of the model fragment to private information can be determined. Then, according to Table 1 below, based on this sensitivity and the current system state information of the electronic device, it can be determined whether the model fragment is the first model fragment or the second model fragment. Table 1 The model sharding can be calculated based on the parameters in Table 1 above. i The index value, , in, =0.4, representing the weight of sensitivity; β=0.25, representing the weight of current TEE load; γ=0.2, representing the weight of network condition; This indicates the weight of the security situation; Representation model partitioning i The sensitivity index is represented by L; the current TEE load index is represented by N; the network status index is represented by M; and the security posture index is represented by M.

[0051] In some embodiments, the method further includes: After executing the third model fragment, the plaintext of the third model fragment is destroyed. The third model fragment includes each model fragment in the target model fragment.

[0052] As an optional implementation, the plaintext of the second model fragment in the REE can be destroyed immediately after step 105, reducing the risk of the second model fragment being leaked in the REE storage space.

[0053] For example, after the ciphertext of the second model segment is decrypted, a short lifetime, such as milliseconds, can be set for the plaintext of the second model segment, so that it is destroyed immediately after the second model segment completes the calculation.

[0054] As another optional implementation, after executing step 105, the plaintext of the second model fragment in the REE and the plaintext of the first and second model fragments in the TEE can be destroyed immediately to reduce the risk of the first and second model fragments being leaked.

[0055] For example, after the ciphertext of the target model fragment is decrypted, the plaintext of the target model fragment can be stored in the protected area of ​​the TEE memory, and a short lifespan, such as milliseconds, can be set for the plaintext of the target model fragment. In this way, after the first model fragment and the second model fragment are determined based on the fragmentation attribute information of the target model fragment, the plaintext of the second model fragment can be destroyed immediately, and it will be destroyed immediately after the first model fragment is completed.

[0056] In some implementations, model segmentation adjustments can be dynamically triggered under certain conditions.

[0057] For example, model sharding adjustment can be triggered when at least one of the following conditions is met, and steps 102 to 105 can be executed based on the adjusted model sharding: The remaining TEE memory is below a threshold, such as 50MB; Leakage risk detected in some model fragments; The first model was detected to be under attack, such as a side-channel attack.

[0058] In some embodiments, the method further includes: Obtain the second output data obtained by calling the second model segmentation in the REE; The second output data is transmitted from the REE to the TEE; In the TEE, the inference result of the model inference request is obtained based on the first output data and the second output data, where the first output data is the output data of the first model segment.

[0059] Optionally, input data, such as model inference requests, can be decrypted within the TEE before use, and the output of the first model segment can be re-encrypted within the TEE or directly passed to the next model segment in the inference task sequence. For computations within the REE, before computation, the second model segment is securely transmitted from the TEE to the REE: the segmented data of the second model segment and the output data of the first model segment are encrypted at the TEE and decrypted at the REE. After the second model segment completes computation, the output result is returned to the TEE for verification or subsequent processing.

[0060] In some implementations, computations within the TEE are primarily lightweight to avoid memory overflows, for example, by using memory pool management. Computations of model slices with low sensitivity to private data are performed within the REE, thus leveraging the REE's high-performance computing resources, such as Graphics Processing Units (GPUs) and Neural Processing Units (NPUs), to perform computationally intensive computational layers.

[0061] Alternatively, REE computations can use acceleration libraries, such as TensorFlow Lite, to improve processing efficiency.

[0062] In this implementation, the REE transmits the processing results of the second model shard to the TEE. The TEE aggregates the processing results of all target model shards and performs post-processing accordingly. This aggregation process ensures data integrity and allows for hash verification to check whether the shard outputs have been tampered with. Finally, the inference result of the model inference request is obtained based on the TEE's post-processing, thus enhancing the privacy of the inference result.

[0063] In some implementations, the inference result of a model inference request can be encrypted within the TEE using a session key, which is either input by the client or generated by the system. The encrypted inference result is then returned to the REE, which decrypts it in a secure environment before returning it to the client.

[0064] In some embodiments, the method further includes: Upon detection of a side-channel attack event, protective measures are implemented, wherein the protective measures include at least one of the following: Pseudo-operations or random delays are inserted into the calculations performed in the TEE and the REE; Random noise is added to the intermediate values ​​in the processing of the model inference request; Adjust the execution order of the target model fragments.

[0065] The aforementioned side-channel attack events may include events such as detecting abnormal power consumption or time offset.

[0066] Optionally, a random delay of ±15% of the baseline value can be inserted into the calculations performed in the TEE and REE. Of course, the random delay can be any range other than ±15% of the baseline value, and is not specifically limited here.

[0067] In some implementations, the above-mentioned protective measures can be adaptively adjusted according to the intensity or level of the security threat received by the first model. For example, when the first model is subjected to a high-intensity security threat, the number of protective measures can be increased from one to two or even three; or, when the first model is subjected to a high-intensity security threat, the amount of random delay inserted into the calculations performed in the TEE and REE can be increased.

[0068] In this implementation, by inserting pseudo-operations or random delays into the computations performed in the TEE and REE, the time channel can be disrupted; by adding random noise to intermediate values, the effectiveness of power consumption analysis can be reduced; and by dynamically adjusting the execution order of the target model fragments, the uncertainty of the fragment sequence can be increased. Ultimately, the security of model fragments invoked in the TEE and REE can be reduced.

[0069] In some embodiments, the method further includes: During the reasoning process corresponding to the model reasoning request, first information is obtained, which includes at least one of the following: the usage frequency of each model segment of the first model, the error rate of each model segment of the first model, and the abnormal access information of each model segment of the first model. Based on the first information, the fourth model fragment is updated. The fourth model fragment includes model fragments in the first model that have security risks or some model fragments in the first model.

[0070] In one optional implementation, it can be determined based on the first information whether there are any model fragments in the model fragments of the first model that have been attacked or have a risk of leakage. For model fragments that have been attacked or have a risk of leakage, these model fragments can be updated individually without updating the model fragments of the entire first model.

[0071] In another alternative implementation, a portion of the model fragments of the first model can be updated periodically, for example, a random portion of the model fragments of the first model can be updated every day or every week.

[0072] In this embodiment, the model fragments of the first model can be dynamically updated, which not only improves the security of the first model, but also reduces the resource consumption caused by updating the model fragments of the entire first model.

[0073] It is worth mentioning that the process of dividing the first model into at least two model fragments can be implemented on an electronic device that executes the model inference method of the present application embodiment, or it can be implemented on other devices, and the fragments can be input by the other devices into the electronic device that executes the model inference method of the present application embodiment for storage.

[0074] As an optional implementation, before obtaining the model inference request, the method further includes: Obtain the model file of the first model; The model file is parsed to obtain the directed acyclic graph (DAG) of the first model, wherein the DAG includes at least two nodes and directed edges between the nodes, and the nodes correspond to the computational layers in the first model; A candidate set of partitions is determined based on the DAG, wherein the candidate set of partitions includes at least one partitioning scheme, each partitioning scheme includes at least two model partitions obtained by the first model partitioning, and partitioning attribute information of each of the at least two model partitions; Based on the system status information of the electronic device, a target sharding scheme is determined from the sharding candidate set; The model file is segmented according to the target segmentation scheme, and each model segment in the target segmentation scheme is encrypted and stored in the TEE storage space.

[0075] In this embodiment, the electronic device executing the model inference method of this application first analyzes the model file of the first model and obtains candidate sharding schemes for the first model based on the DAG graph obtained from the analysis. Finally, based on the system state of the electronic device when receiving the model inference request, the optimal scheme is selected from the candidate sharding schemes as the target sharding scheme. The model of the first model is then sharded according to the target sharding scheme, and each model shard is encrypted and stored in the TEE storage space. Finally, the target model shard on which the model inference request depends is selected from the various model shards. In this way, the entire process from dynamic sharding and dynamic encryption to model inference can be realized on the electronic device, improving the efficiency of using the first model.

[0076] As another optional implementation, other devices can analyze the DAG graph of the first model and determine a candidate set of fragments for the first model based on the DAG graph. When the electronic device executing the model inference method of this application receives a model inference request, it can report its current system status to the other device. Based on the system status of the electronic device executing the model inference method of this application, the other device selects the optimal fragmentation scheme from the candidate fragmentation schemes as the target fragmentation scheme, and fragments the first model according to the target fragmentation scheme. Each fragment is then encrypted and transmitted to the electronic device executing the model inference method of this application. Finally, the electronic device executing the model inference method of this application selects the target model fragment from all the fragments of the first model that is required to respond to the model inference request. This allows for dynamic fragmentation and encryption of the model on other devices with higher security, thereby improving the security of the first model.

[0077] In some implementations, the candidate sharding set can cover all possible sharding schemes for the first model. Subsequently, the optimal target sharding scheme can be selected from the candidate sharding set based on the following strategy: selecting the target sharding scheme from the candidate sharding set based on security weights and performance weights. The security weights are used to constrain the security of the sharding scheme, such as non-uniform size of model shards and random boundaries of model shards; and nodes that are highly sensitive to private data must be included in the model shards called by the TEE, etc. The performance weights are used to constrain the computational efficiency, computational reliability, and other performance of the sharding scheme, such as minimizing the total communication volume between model shards and balancing the decryption time of model shards, etc.

[0078] For example, in high-security mode, choose a scheme with a large number of model fragments and large size differences; in high-performance mode, choose a scheme with fewer model fragments and smaller computationally intensive star model fragments.

[0079] In this embodiment, the first model is first divided into at least two model fragments and stored in the TEE storage space. One part of the model fragments, the first model fragment, is invoked within the TEE, while the other part, the second model fragment, is invoked within the REE. Thus, when responding to model inference requests on the electronic device, on the one hand, since it is not necessary to invoke the complete first model within the TEE, the computational resources consumed by the TEE are reduced, which is beneficial for improving the execution efficiency of the first model fragment. On the other hand, since each model fragment is pre-stored in the TEE storage space, it is difficult for attackers to obtain the model file from the TEE storage space. Moreover, during the execution of model inference, only the second model fragment needs to be invoked in the REE, without needing to invoke the complete first model within the REE, ensuring that the first model is not fully exposed in the REE, and attackers cannot obtain the complete model file of the first model from the REE. In summary, this embodiment can improve the security and efficiency of the first model.

[0080] For ease of understanding, as shown in Figure 2 The interactive schematic diagram shown is used as an example to illustrate the model reasoning method provided in this application embodiment.

[0081] like Figure 2 As shown, the model inference method provided in this application includes the following steps: 1) The REE of the electronic device receives the user's model inference request; 2) The REE encrypts the model inference request before forwarding it to the TEE; 3) The TEE reads the metadata of each model shard from secure storage; 4) Secure storage returns a list of model shards and the encrypted metadata of each model shard to the TEE; 5) TEE decrypts the metadata of the model shards and verifies the data hash chain; 6) The TEE determines the target model shard based on the model shard's metadata and requests the target model shard from the secure storage. 7) Securely store the encrypted data of the target model fragment returned to the TEE; 8) TEE decryption of target model fragments; 9) The TEE determines the first model fragment and the second model fragment from the target model fragment; 10) For the first model segment, perform the computation in the TEE; 11) The TEE sends the ciphertext of the second model fragment to the REE; 12) REE decrypts the second model fragment and executes the calculation process for the second model fragment; 13) The REE returns the output of the second model segment to the TEE; 14) TEE aggregates the output results of the first model segment and the output results of the second model segment; 15) The TEE determines whether the model shards need to be updated based on the first information during the model inference process. If an update is triggered, steps 5) to 14) are repeated after the model shards are updated. 16) The TEE returns the inference result to the REE; 17) Output the reasoning results to the client via REE.

[0082] In some implementations, the methods for triggering model shard updates may include at least one of the following: Time-driven update: It adopts a dual time window mechanism with a base period of 24-hour fixed interval and random jitter of ±(0-30) minutes random offset; Event-driven updates: for example, when the number of model fragment decryption failures exceeds 3 times, or when the memory access pattern matches the characteristics of an attack. Space-driven updates: For example, when the distance the electronic device moves is greater than a preset threshold or the number of base station handovers performed by the electronic device is greater than 3 times per hour.

[0083] In some implementations, model sharding updates can be adjustments to the model sharding scheme, such as adjusting the boundaries of model shards. The shards to be updated can be a portion of the first model, such as a model shard that poses a security threat, or a random portion of the model shards.

[0084] In this way, the model sharding update process can be carried out without affecting other normal model shards, achieving minimally invasive updates and reducing the resource consumption of model sharding updates.

[0085] In some implementations, the updated new model fragments can replace the old model fragments, meaning that while storing the updated new model fragments, the updated old model fragments can also be safely destroyed.

[0086] In this embodiment, the model sharding update is dynamically triggered when the system detects a threat or performs routine maintenance. This ensures long-term system security, enabling rapid self-updating to protect model files in the event of attacks or other scenarios. It supports minute-level sharding policy updates and defends against zero-day attacks.

[0087] See Figure 3 This application also provides a dynamic encryption method for a model, wherein the execution entity of the method can be a server, such as... Figure 3 As shown, the method includes: Step 301: Obtain the model file of the first model.

[0088] Step 302: Parse the model file to obtain the directed acyclic graph (DAG) of the first model, wherein the DAG includes at least two nodes and directed edges between the nodes, and the nodes correspond to the computation layers in the first model.

[0089] Step 303: Determine a candidate set of partitions based on the DAG, wherein the candidate set of partitions includes at least one partitioning scheme, each partitioning scheme includes at least two model partitions obtained by the first model, and partitioning attribute information of each of the at least two model partitions; the partitioning attribute information is used to indicate at least one of the following of the computational layers contained in the corresponding model partition: layer type, computational load, and sensitivity to private data.

[0090] Step 304: Obtain the system status information of the electronic device.

[0091] Step 305: Determine the target sharding scheme from the sharding candidate set based on the system status information.

[0092] Step 306: Send each model fragment in the target fragmentation scheme to the electronic device.

[0093] It should be noted that in this embodiment, the process of the server determining the candidate fragment set of the first model based on the model file of the first model, and finally the system state of the electronic device, selecting the optimal solution from the candidate fragment set as the target fragment scheme, and fragmenting the model fragments of the first model according to the target fragment scheme, adopts the same principle as the process of the electronic device determining the target fragment scheme in the previous embodiment of this application. To avoid repetition, it will not be described again here.

[0094] In this embodiment, the server parses the model file of the first model and obtains a candidate fragment set for the first model based on the analyzed DAG graph. Finally, based on the system state of the electronic device when receiving the model inference request, the server selects the optimal solution from the candidate fragment set as the target fragmentation solution, fragments the first model according to the target fragmentation solution, and encrypts each fragment before sending it to the electronic device. This allows the electronic device to store the model fragments of the first model in its TEE storage space. In this way, the model file of the first model is unknown to the electronic device, reducing the risk of the model file being exposed on the electronic device.

[0095] The model inference method provided in this application can be executed by a model inference device or an electronic device containing a model inference device. This application uses the execution of the model inference method by a model inference device as an example to illustrate the model inference device provided in this application.

[0096] See Figure 4 The model inference device 400 of this application embodiment includes the following modules: The first acquisition module 401 is used to acquire a model inference request, the model inference request being used to request a first model to perform model inference, the first model including at least two model fragments; The second acquisition module 402 is used to acquire the target model fragment and the fragment attribute information of the target model fragment related to the model inference request from the Trusted Execution Environment (TEE) storage space. The target model fragment includes a partial model fragment of the first model. The fragment attribute information is used to indicate at least one of the following of the computing layers contained in the corresponding model fragment: layer type, computational load, and sensitivity to private data. The first determining module 403 is used to determine the first model fragment and the second model fragment based on the fragmentation attribute information of the target model fragment and the system state information of the electronic device. The first transmission module 404 is used to transmit the second model fragment to the rich execution environment REE; The scheduling module 405 is used to call the first model shard in the TEE and the second model shard in the REE based on the scheduler between the TEE and the REE, so as to execute the processing procedure corresponding to the model inference request.

[0097] In some implementations, the second acquisition module 402 includes: The first acquisition unit is used to acquire the ciphertext of the target model fragment related to the model inference request from the Trusted Execution Environment (TEE) storage space. The first decryption unit is used to decrypt the target model fragment within the TEE to obtain the plaintext of the target model fragment and the fragment attribute information of the target model fragment. The first transmission module 404 includes: The first transmission unit is used to transmit the ciphertext of the second model fragment from the TEE to the REE; The second decryption unit is used to decrypt the ciphertext of the second model fragment in the REE to obtain the plaintext of the second model fragment.

[0098] In some embodiments, the model inference device 400 further includes: The destruction module is used to destroy the plaintext of the third model fragment after the third model fragment is executed. The third model fragment includes each model fragment in the target model fragment.

[0099] In some embodiments, the model inference device 400 further includes: The third acquisition module is used to acquire the second output data obtained by calling the second model segmentation in the REE; The second transmission module is used to transmit the second output data from the REE to the TEE; A first processing module is configured to, within the TEE, derive the inference result of the model inference request based on first output data and second output data, wherein the first output data is the output data of the first model segment.

[0100] In some embodiments, the model inference device 400 further includes: An execution module is configured to perform protective measures upon detection of a side-channel attack event, wherein the protective measures include at least one of the following: Pseudo-operations or random delays are inserted into the calculations performed in the TEE and the REE; Random noise is added to the intermediate values ​​in the processing of the model inference request; Adjust the execution order of the target model fragments.

[0101] In some embodiments, the model inference device 400 further includes: The fourth acquisition module is used to acquire first information during the inference process corresponding to the model inference request. The first information includes at least one of the following: the usage frequency of each model segment of the first model, the error rate of each model segment of the first model, and the abnormal access information of each model segment of the first model. The update module is used to update the fourth model fragment based on the first information. The fourth model fragment includes model fragments in the first model that have security risks or some model fragments in the first model.

[0102] In some embodiments, the model inference device 400 further includes: The fifth acquisition module is used to acquire the model file of the first model; The first parsing module is used to parse the model file to obtain the directed acyclic graph (DAG) of the first model, wherein the DAG includes at least two nodes and directed edges between the nodes, and the nodes correspond to the computation layers in the first model. The second determining module is used to determine a partitioning candidate set based on the DAG, wherein the partitioning candidate set includes at least one partitioning scheme, each partitioning scheme includes at least two model partitions obtained by the first model partitioning, and partitioning attribute information of each of the at least two model partitions; The third determining module is used to determine the target sharding scheme from the sharding candidate set based on the system status information of the electronic device; The second processing module is used to segment the model file according to the target segmentation scheme and store each model segment in the target segmentation scheme into the TEE storage space.

[0103] The model inference device 400 in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific type of device.

[0104] The model inference device 400 in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit its use.

[0105] The model reasoning device 400 provided in this application embodiment can achieve... Figure 1 or Figure 2 The various processes implemented in the method embodiments shown are all capable of achieving the same beneficial effects, and will not be described again here to avoid repetition.

[0106] The dynamic encryption method for models provided in this application can be executed by a dynamic encryption device for the model or a server containing such a device. This application uses the example of a dynamic encryption device executing the dynamic encryption method for the model to illustrate the dynamic encryption device provided in this application.

[0107] See Figure 5 The dynamic encryption device 500 of the model in this application embodiment includes the following modules: The sixth acquisition module 501 is used to acquire the model file of the first model; The second parsing module 502 is used to parse the model file to obtain the directed acyclic graph (DAG) of the first model, wherein the DAG includes at least two nodes and directed edges between the nodes, and the nodes correspond to the computation layers in the first model. The fourth determining module 503 is used to determine a partitioning candidate set based on the DAG, wherein the partitioning candidate set includes at least one partitioning scheme, each partitioning scheme includes at least two model partitions obtained by the first model, and partitioning attribute information of each of the at least two model partitions; the partitioning attribute information is used to indicate at least one of the following of the computational layers contained in the corresponding model partition: layer type, computational load, and sensitivity to private data; The seventh acquisition module 504 is used to acquire system status information of the electronic device; The fifth determining module 505 is used to determine a target sharding scheme from the sharding candidate set based on the system status information; The third processing module 506 is used to encrypt each model fragment in the target sharding scheme and send each model fragment in the target sharding scheme to the electronic device.

[0108] The dynamic encryption device 500 for the model provided in this application embodiment can achieve... Figure 3 The various processes implemented in the method embodiments shown are all capable of achieving the same beneficial effects, and will not be described again here to avoid repetition.

[0109] Optionally, such as Figure 6 As shown, this application embodiment also provides an electronic device 600, including a processor 601 and a memory 602. The memory 602 stores a program or instructions that can run on the processor 601. When the program or instructions are executed by the processor 601, they implement the various steps of the above-described model reasoning method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0110] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0111] Figure 7 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0112] The electronic device 700 includes, but is not limited to, components such as: radio frequency unit 701, network module 702, audio output unit 703, input unit 704, sensor 705, display unit 706, user input unit 707, interface unit 708, memory 709, and processor 710.

[0113] Those skilled in the art will understand that the electronic device 700 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 710 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0114] The input unit 704 or the user input unit 707 is used to obtain a model inference request, which is used to request a first model to perform model inference. The first model includes at least two model fragments. Processor 710 is configured to obtain target model fragments and fragment attribute information of the target model fragments related to the model inference request from the Trusted Execution Environment (TEE) storage space, wherein the target model fragments include partial model fragments of the first model, and the fragment attribute information is used to indicate at least one of the following of the computation layers contained in the corresponding model fragments: layer type, computational load, and sensitivity to private data; The processor 710 is further configured to determine the first model fragment and the second model fragment based on the fragmentation attribute information of the target model fragment and the system state information of the electronic device; The processor 710 is also used to transmit the second model fragment to the rich execution environment REE via the interface unit 708; The processor 710 is further configured to, based on a scheduler between the TEE and the REE, invoke the first model shard in the TEE and the second model shard in the REE to execute the processing procedure corresponding to the model inference request.

[0115] In some implementations, the step of processor 710 retrieving the target model shard and the shard attribute information of the target model shard related to the model inference request from the Trusted Execution Environment (TEE) storage space includes: Retrieve the ciphertext of the target model fragment related to the model inference request from the Trusted Execution Environment (TEE) storage space; The target model fragment is decrypted within the TEE to obtain the plaintext of the target model fragment and the fragment attribute information of the target model fragment; The process of transmitting the second model fragment to the REE via interface unit 708, executed by processor 710, includes: The encrypted text of the second model fragment is transmitted from the TEE to the REE through the interface unit 708; The ciphertext of the second model fragment is decrypted in the REE to obtain the plaintext of the second model fragment.

[0116] In some implementations, the processor 710 is further configured to control the memory 709 to destroy the plaintext of the third model fragment after executing the third model fragment, the third model fragment including each model fragment in the target model fragment.

[0117] In some implementations, the processor 710 is also used for: Obtain the second output data obtained by calling the second model segmentation in the REE; The second output data is transmitted from the REE to the TEE via interface unit 708; In the TEE, the inference result of the model inference request is obtained based on the first output data and the second output data, where the first output data is the output data of the first model segment.

[0118] In some embodiments, the processor 710 is further configured to perform protective measures upon detecting a side-channel attack event, wherein the protective measures include at least one of the following: Pseudo-operations or random delays are inserted into the calculations performed in the TEE and the REE; Random noise is added to the intermediate values ​​in the processing of the model inference request; Adjust the execution order of the target model fragments.

[0119] In some implementations, the processor 710 is also used for: During the reasoning process corresponding to the model reasoning request, first information is obtained, which includes at least one of the following: the usage frequency of each model segment of the first model, the error rate of each model segment of the first model, and the abnormal access information of each model segment of the first model. Based on the first information, the fourth model fragment is updated. The fourth model fragment includes model fragments in the first model that have security risks or some model fragments in the first model.

[0120] In some implementations, before the input unit 704 or the user input unit 707 executes the request to obtain the model inference, the processor 710 is further configured to: Obtain the model file of the first model; The model file is parsed to obtain the directed acyclic graph (DAG) of the first model, wherein the DAG includes at least two nodes and directed edges between the nodes, and the nodes correspond to the computational layers in the first model; A candidate set of partitions is determined based on the DAG, wherein the candidate set of partitions includes at least one partitioning scheme, each partitioning scheme includes at least two model partitions obtained by the first model partitioning, and partitioning attribute information of each of the at least two model partitions; Based on the system status information of the electronic device, a target sharding scheme is determined from the sharding candidate set; The model file is divided into shards according to the target sharding scheme, and each model shard in the target sharding scheme is stored in the TEE storage space.

[0121] The electronic device 700 provided in this application embodiment is capable of performing the following... Figure 6 The electronic device shown illustrates the functions of each module and its ability to obtain data. Figure 7 The same beneficial effects of the electronic devices shown will not be repeated here to avoid repetition.

[0122] It should be understood that, in this embodiment, the input unit 704 may include a graphics processing unit (GPU) 7041 and a microphone 7042. The GPU 7041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 706 may include a display panel 7061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0123] The memory 709 can be used to store software programs and various data. The memory 709 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 709 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 709 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0124] Processor 710 may include one or more processing units; optionally, processor 710 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 710.

[0125] This application provides a server including a processor and a memory. The memory stores programs or instructions that can run on the processor. When the programs or instructions are executed by the processor, they implement the various processes of the dynamic encryption method embodiment of the above model and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0126] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described model inference method or dynamic encryption method of the model, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0127] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0128] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described model inference method or model dynamic encryption method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0129] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0130] This application provides a computer program product that is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described model inference method or model dynamic encryption method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0131] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0132] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0133] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A model reasoning method, characterized in that, Applied to electronic devices, the method includes: Obtain a model inference request, the model inference request being used to request a first model to perform model inference, the first model comprising at least two model fragments; Obtain the target model fragment and the fragment attribute information of the target model fragment related to the model inference request from the Trusted Execution Environment (TEE) storage space. The target model fragment includes a partial model fragment of the first model. The fragment attribute information is used to indicate at least one of the following of the computation layers contained in the corresponding model fragment: layer type, computational load, and sensitivity to private data. Based on the fragmentation attribute information of the target model fragment and the system state information of the electronic device, the first model fragment and the second model fragment are determined; The second model fragment is transferred to the REE rich execution environment; Based on the scheduler between the TEE and the REE, the first model shard is invoked in the TEE and the second model shard is invoked in the REE to execute the processing procedure corresponding to the model inference request.

2. The method according to claim 1, characterized in that, The step of obtaining the target model shard and the shard attribute information of the target model shard related to the model inference request from the Trusted Execution Environment (TEE) storage space includes: Retrieve the ciphertext of the target model fragment related to the model inference request from the Trusted Execution Environment (TEE) storage space; The target model fragment is decrypted within the TEE to obtain the plaintext of the target model fragment and the fragment attribute information of the target model fragment; The step of transmitting the second model fragment to the REE includes: The ciphertext of the second model fragment is transmitted from the TEE to the REE; The ciphertext of the second model fragment is decrypted in the REE to obtain the plaintext of the second model fragment.

3. The method according to claim 2, characterized in that, The method further includes: After executing the third model fragment, the plaintext of the third model fragment is destroyed. The third model fragment includes each model fragment in the target model fragment.

4. The method according to claim 1, characterized in that, The method further includes: Obtain the second output data obtained by calling the second model segmentation in the REE; The second output data is transmitted from the REE to the TEE; In the TEE, the inference result of the model inference request is obtained based on the first output data and the second output data, where the first output data is the output data of the first model segment.

5. The method according to claim 1, characterized in that, The method further includes: Upon detection of a side-channel attack event, protective measures are implemented, wherein the protective measures include at least one of the following: Pseudo-operations or random delays are inserted into the calculations performed in the TEE and the REE; Random noise is added to the intermediate values ​​in the processing of the model inference request; Adjust the execution order of the target model fragments.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: During the reasoning process corresponding to the model reasoning request, first information is obtained, which includes at least one of the following: the usage frequency of each model segment of the first model, the error rate of each model segment of the first model, and the abnormal access information of each model segment of the first model. Based on the first information, the fourth model fragment is updated. The fourth model fragment includes model fragments in the first model that have security risks or some model fragments in the first model.

7. The method according to any one of claims 1 to 5, characterized in that, Prior to obtaining the model inference request, the method further includes: Obtain the model file of the first model; The model file is parsed to obtain the directed acyclic graph (DAG) of the first model, wherein the DAG includes at least two nodes and directed edges between the nodes, and the nodes correspond to the computational layers in the first model; A candidate set of partitions is determined based on the DAG, wherein the candidate set of partitions includes at least one partitioning scheme, each partitioning scheme includes at least two model partitions obtained by the first model partitioning, and partitioning attribute information of each of the at least two model partitions; Based on the system status information of the electronic device, a target sharding scheme is determined from the sharding candidate set; The model file is divided into shards according to the target sharding scheme, and each model shard in the target sharding scheme is stored in the TEE storage space.

8. A dynamic encryption method for a model, characterized in that, Applied to a server, the method includes: Obtain the model file for the first model; The model file is parsed to obtain the directed acyclic graph (DAG) of the first model, wherein the DAG includes at least two nodes and directed edges between the nodes, and the nodes correspond to the computational layers in the first model; A partitioning candidate set is determined based on the DAG, wherein the partitioning candidate set includes at least one partitioning scheme, each partitioning scheme includes at least two model partitions obtained by the first model, and partitioning attribute information of each of the at least two model partitions; the partitioning attribute information is used to indicate at least one of the following of the computational layers contained in the corresponding model partition: layer type, computational load, and sensitivity to private data; Obtain system status information from electronic devices; Based on the system status information, a target sharding scheme is determined from the sharding candidate set; Each model fragment in the target fragmentation scheme is sent to the electronic device.

9. A model reasoning device, characterized in that, For an electronic device, the device includes: The first acquisition module is used to acquire a model inference request, the model inference request being used to request a first model to perform model inference, the first model including at least two model fragments; The second acquisition module is used to acquire the target model fragment and the fragment attribute information of the target model fragment related to the model inference request from the Trusted Execution Environment (TEE) storage space. The target model fragment includes a partial model fragment of the first model. The fragment attribute information is used to indicate at least one of the following of the computing layers contained in the corresponding model fragment: layer type, computational load, and sensitivity to private data. The first determining module is used to determine the first model fragment and the second model fragment based on the fragmentation attribute information of the target model fragment and the system state information of the electronic device; The first transmission module is used to transmit the second model fragment to the rich execution environment REE; The scheduling module is used to call the first model shard in the TEE and the second model shard in the REE based on the scheduler between the TEE and the REE, so as to execute the processing procedure corresponding to the model inference request.

10. The apparatus according to claim 9, characterized in that, The second acquisition module includes: The first acquisition unit is used to acquire the ciphertext of the target model fragment related to the model inference request from the Trusted Execution Environment (TEE) storage space. The first decryption unit is used to decrypt the target model fragment within the TEE to obtain the plaintext of the target model fragment and the fragment attribute information of the target model fragment. The first transmission module includes: The first transmission unit is used to transmit the ciphertext of the second model fragment from the TEE to the REE; The second decryption unit is used to decrypt the ciphertext of the second model fragment in the REE to obtain the plaintext of the second model fragment.

11. The apparatus according to claim 10, characterized in that, The device further includes: The destruction module is used to destroy the plaintext of the third model fragment after the third model fragment is executed. The third model fragment includes each model fragment in the target model fragment.

12. The apparatus according to claim 9, characterized in that, The device further includes: The third acquisition module is used to acquire the second output data obtained by calling the second model segmentation in the REE; The second transmission module is used to transmit the second output data from the REE to the TEE; A first processing module is configured to, within the TEE, derive the inference result of the model inference request based on first output data and second output data, wherein the first output data is the output data of the first model segment.

13. The apparatus according to claim 9, characterized in that, The device further includes: An execution module is configured to perform protective measures upon detection of a side-channel attack event, wherein the protective measures include at least one of the following: Pseudo-operations or random delays are inserted into the calculations performed in the TEE and the REE; Random noise is added to the intermediate values ​​in the processing of the model inference request; Adjust the execution order of the target model fragments.

14. The apparatus according to any one of claims 9 to 13, characterized in that, The device further includes: The fourth acquisition module is used to acquire first information during the inference process corresponding to the model inference request. The first information includes at least one of the following: the usage frequency of each model segment of the first model, the error rate of each model segment of the first model, and the abnormal access information of each model segment of the first model. The update module is used to update the fourth model fragment based on the first information. The fourth model fragment includes model fragments in the first model that have security risks or some model fragments in the first model.

15. The apparatus according to any one of claims 9 to 13, characterized in that, The device further includes: The fifth acquisition module is used to acquire the model file of the first model; The first parsing module is used to parse the model file to obtain the directed acyclic graph (DAG) of the first model, wherein the DAG includes at least two nodes and directed edges between the nodes, and the nodes correspond to the computation layers in the first model. The second determining module is used to determine a partitioning candidate set based on the DAG, wherein the partitioning candidate set includes at least one partitioning scheme, each partitioning scheme includes at least two model partitions obtained by the first model partitioning, and partitioning attribute information of each of the at least two model partitions; The third determining module is used to determine the target sharding scheme from the sharding candidate set based on the system status information of the electronic device; The second processing module is used to segment the model file according to the target segmentation scheme and store each model segment in the target segmentation scheme into the TEE storage space.

16. A dynamic encryption device for a model, characterized in that, For use with a server, the device includes: The sixth acquisition module is used to acquire the model file of the first model; The second parsing module is used to parse the model file to obtain the directed acyclic graph (DAG) of the first model, wherein the DAG includes at least two nodes and directed edges between the nodes, and the nodes correspond to the computation layers in the first model. The fourth determining module is used to determine a partitioning candidate set based on the DAG, wherein the partitioning candidate set includes at least one partitioning scheme, each partitioning scheme includes at least two model partitions obtained by the first model, and partitioning attribute information of each of the at least two model partitions; the partitioning attribute information is used to indicate at least one of the following of the computational layers contained in the corresponding model partition: layer type, computational load, and sensitivity to private data; The seventh acquisition module is used to acquire system status information of the electronic device; The fifth determining module is used to determine the target sharding scheme from the sharding candidate set based on the system status information; The third processing module is used to encrypt each model fragment in the target sharding scheme and send each model fragment in the target sharding scheme to the electronic device.

Images