TEE-based edge deep neural network model reasoning method and system

Abstract

The invention discloses a TEE-based edge deep neural network model reasoning method and system. The method comprises the steps of obtaining a specified sensitive layer in a target deep neural network model; performing memory demand derivation on the sensitive layer operator to determine the memory demand of the sensitive layer operator; completing the training of the target deep neural network model, encrypting the weight parameter information of the sensitive layer, and exporting a model file; deploying and submitting the model file to target side end equipment with TEE and REE; sensitive layer task segmentation is carried out on the target edge equipment according to the memory dependence information of the sensitive layer, a non-sensitive layer in the target deep neural network model is constructed in REE, n sub-tasks generated by sensitive layer segmentation are constructed in TEE, and construction of the target deep neural network model is completed for model reasoning. The invention aims to provide efficient operator-level confidential protection by using TEE, and ensure that a confidential calculation module is seamlessly fused into a mainstream machine learning workflow at the same time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data security of edge computing, and in particular to an edge-end deep neural network model inference method and system based on TEE. BACKGROUND

[0002] Edge computing reduces the dependence on cloud data transmission through localized data processing, and its low latency and privacy protection characteristics promote its wide deployment in various intelligent applications. However, edge devices usually involve multiple untrusted entities and rely on complex software and hardware infrastructure, so that the deep neural network deployed thereon faces potential security threats. An effective model security protection scheme is to use the hardware isolation mechanism of ARM TrustZone to protect the confidentiality and integrity of the model sensitive layer inference process. Compared with traditional privacy protection methods such as homomorphic encryption, there is no need to perform ciphertext conversion, noise injection or model function fitting, reducing the computational overhead and avoiding precision loss. ARM TrustZone is one of the mainstream hardware implementations for building TEEs. Compared with other TEE solutions such as Intel SGX and AMD SEV, TrustZone relies on a lightweight hardware isolation mechanism and low power consumption characteristics, and has higher adaptability and deployment advantages in resource-constrained edge computing scenarios. Specifically, TrustZone divides the system execution environment into the normal world and the secure world at the hardware architecture level. The normal world (REE) runs general operating systems (such as Linux, Android) and normal application programs, while the secure world runs an independent secure operating system (such as OP-TEE) and trusted applications for processing security-sensitive tasks. The security monitor manages the world switching and is responsible for saving and restoring the processor context when entering and exiting the secure world to ensure state consistency. The two worlds are physically isolated through memory, preventing data leakage. The system is verified through a chain of trust at startup to ensure that the systems and applications loaded in the secure world are legal and trusted. These mechanisms collectively form the security foundation of ARM TrustZone, ensuring that the secure world remains confidential and complete even if the normal world is compromised. However, existing TEE-based secure inference schemes still face challenges (1) Difficulty in fully reusing the highly optimized computing ecosystem in mainstream deep learning frameworks: Deploying trusted applications in TEE requires explicitly specifying the stack space (TA_STACK_SIZE) and data area size (TA_DATA_SIZE) before compilation. This static memory allocation makes it impossible for the system to dynamically expand the secure memory during runtime. This makes it difficult for TEE to be compatible with the runtime dynamic memory management mechanism relied on by mainstream deep learning frameworks (such as PyTorch). This compatibility gap not only increases the development cost of the model, but also reduces the execution efficiency of the non-sensitive layers on the REE side. (2) TEE lacks parallelization support, making the inference of sensitive layers a bottleneck for system throughput: Due to the core design goal of minimizing the trusted computing base (TCB) and strengthening isolation, TEE only supports single-threaded execution. While this design avoids the expansion of attack surfaces caused by state sharing in multi-threaded execution, it also leads to a performance gap between operator execution in TEE and on the REE side, increasing the overall model inference latency. SUMMARY

[0003] The technical problem solved by the present application: In view of the above problems of the prior art, the present application provides a TEE-based edge deep neural network model inference method and system. The present application aims to provide efficient operator-level confidential protection using TEE, while ensuring that the confidential computing module seamlessly integrates into mainstream machine learning workflows.

[0004] To solve the above technical problems, the technical scheme adopted by the present application is: A TEE-based edge deep neural network model inference method, comprising the following steps: S101, obtaining a specified sensitive layer in a target deep neural network model, and the remaining layers are non-sensitive layers; S102, performing memory requirement derivation on the sensitive layer operator to determine the memory requirement of the sensitive layer operator; S103, completing the training of the target deep neural network model; S104, encrypting the weight parameter information of the sensitive layer in the target deep neural network model after training; S105, model serializing the weight parameter information of the non-sensitive layer of the target deep neural network model, the encrypted weight parameter information of the sensitive layer, and the memory dependency information of the sensitive layer to generate a model file in a specified format; S106, deploying and submitting the model file to a target edge device with TEE and REE; S107, performing sensitive layer task segmentation according to the memory dependency information of the sensitive layer on the target edge device, and dividing the sensitive layer task into n sub-tasks based on a preset number of parallel optimization cores n; S108, constructing the non-sensitive layer in the target deep neural network model in REE, and constructing the n sub-tasks generated by sensitive layer segmentation in TEE, to complete the construction of the target deep neural network model; S109, calling the system interface of the target deep neural network model on the REE side of the target edge device for model inference; S110, calling the system interface of the target deep neural network model on the REE side of the target edge device to obtain the model inference result of the target deep neural network model.

[0005] Optionally, in step S102, when deriving the memory requirements of sensitive layer operators to determine their memory needs, the process includes determining the static resident memory required by the sensitive layer operators based on the structural parameters declared by the sensitive layer operators. These structural parameters include some or all of the weights, biases, mean, and variance. The process also involves deriving the dynamic temporary storage memory required by all sensitive layer operators based on the computation graph of the target deep neural network model and the dimensionality information of the input tensor. This dynamic temporary storage memory includes the memory occupied by intermediate activation values ​​and the workspace. The derivation of the dynamic temporary storage memory required by all sensitive layer operators based on the computation graph of the target deep neural network model and the dimensionality information of the input tensor includes: S201, obtain the directed acyclic graph DAG=(V,E) and input tensor set X of the target deep neural network model, where V is the set of nodes, E is the set of edges, nodes are the computation operations of neural network operators, and edges are the data transmission between computation operations; S202, perform topological sorting on the directed acyclic graph (DAG) and generate queues according to the dependency order between nodes; S203, determine whether there are any unprocessed nodes in the topologically sorted queue; if so, obtain an unprocessed node as the current node v; if there are no unprocessed nodes, jump to step S209. S204, determine whether the in-degree of the current node v is 0; if so, select the corresponding input tensor x from the input tensor set X; otherwise, aggregate the output tensors of all its predecessor nodes as the input tensor x of the current node v. S205, determine whether the current node v belongs to the sensitive layer. If it does not belong to the sensitive layer, jump to step S208; otherwise, jump to step S206. S206, Obtain the dimension information x of the input tensor x. d Based on the dimensional information x of the input tensor x d The memory size s required to determine the intermediate activation value of the current node v v ; S207, determine whether the current node v depends on the workspace. If it does not depend on the workspace, then set the memory size w required by the workspace of the entry node v. v Set to 0; otherwise, based on the dimension information x of the input tensor x. d Determine the memory size w required for the workspace of entry node v. v ; S208, Perform forward propagation operation on the current node v, calculate the output based on the input tensor x, and store the result for use by its successor nodes, then jump to step S203; S209, Returns the dynamic temporary memory dependency information of all accumulated sensitive layer nodes, End.

[0006] Optionally, in step S206, the dimension information x is based on the input tensor x. d The memory size s required to determine the intermediate activation value of the current node v v Includes: dimensional information x based on the input tensor x d Based on the derivation rules of the output size of the operator corresponding to the current node v, the shape of the output tensor of the current node v is determined, and the memory size s required for the output tensor of the current node v, i.e., the intermediate activation value, is calculated in combination with the data type of the output tensor. v In step S207, the dimension information x is based on the input tensor x. d Determine the memory size w required for the workspace of entry node v. v Includes: dimensional information x based on the input tensor d Based on the computational characteristics of the operator corresponding to the entry node v, the size of the temporary buffer space required to support intermediate computations during execution is determined as the workspace memory size w. v The accelerated computing operation includes the im2col conversion.

[0007] Optionally, encrypting the weight parameter information of the sensitive layer in the target deep neural network model after training in step S104 includes: generating a temporary elliptic curve key pair consisting of a temporary public key PK_temp and a temporary private key SK_temp; performing ECDH key negotiation with the temporary public key PK_temp and the fixed elliptic curve public key PK_tee provided in the TEE environment of the target edge device to obtain a shared key K_shared and its derived symmetric key; using the symmetric key derived from the shared key K_shared to locally encrypt the weight parameters of the sensitive layer in the target deep neural network model after training; and when decrypting the encryption result of the weight parameters of the sensitive layer, the target edge device's TEE environment uses its fixed elliptic curve private key SK_tee and the received temporary public key PK_temp to perform ECDH key negotiation again to obtain the same shared key K_shared and its derived symmetric key, and uses the derived symmetric key to decrypt the encryption result of the weight parameters of the sensitive layer, finally recovering the original weight parameters of the sensitive layer in the TEE environment of the target edge device.

[0008] Optionally, step S107 includes: S301, Obtain the sensitive layer operator linked list consisting of all sensitive layers in the target deep neural network model. , and the preset number of parallel optimization cores n; S302, check the legality of the preset parallel optimization core number n: n must be greater than 1 and less than or equal to the minimum of the core number on the physical device and the sensitive layer operator output channel, if n does not meet the condition, an error is reported and step S308 is skipped; otherwise, step S303 is skipped; S303, based on the core number n of parallel optimization, n empty sub-chains are constructed in the REE on the target edge device Each sub-chain represents a sub-task after being divided, and serves as the basis for constructing the sensitive layer of the target deep neural network model in the TEE; S304, judge whether there are unprocessed operators in the sensitive layer operator chain , if so, get the current operator of the unprocessed operator and jump to step S305; otherwise, jump to step S308; S305, judge whether there are weight parameters that need to be divided in the output tensor channel dimension of the current operator, if so, jump to step S306; if not, jump to step S307; S306, divide the weight parameters of the current operator based on the output channel to generate the divided sub-operators, which are inserted into each sub-chain in turn; S307, insert a copy of the current operator into each sub-chain ; S308, return the final output, including n sensitive layer sub-lists generated by n sub-chains , and the process ends.

[0009] Optionally, step S108 includes: S401, deserialize the neural network model file into the REE memory on the target edge device side using the TorchScript interface in PyTorch, wherein the non-sensitive layers of the model will be delegated to the PyTorch underlying runtime management and initialized during the deserialization process; S402, based on the parallel optimization core number n checked for legality, create n sub-threads in the REE of the target edge device, each sub-thread corresponds to manage a sensitive layer sub-list generated by task division, and each sub-thread will run on a different core; S403, each sub-thread creates an independent session for communication with the TEE and calls SMC to embed the core where it is located into the TEE, and sends the parameters of all sensitive operators in the sub-list after task division to the TEE in turn, including structural parameters and dynamic temporary memory demand information; the structural parameters include part or all of the weights, biases, means, and variances; the dynamic temporary memory demand information includes intermediate activation value memory size and workspace memory size; S404, the inference framework in the TEE of the target edge device accepts parameters for constructing the model sensitive layer from the REE side, and completes the initialization operation of the sensitive layer operator.

[0010] Optionally, the system interface of the target deep neural network model on the REE side on the target edge device in step S109 is called for model inference, including: S501, obtaining an input tensor as an input of the target deep neural network model on the REE side, and initializing an empty sensitive layer operator execution sequence on the REE side, and setting the values of two flag bits Flag_E and Flag_D to False to respectively indicate whether the intermediate result tensor needs to be encrypted or decrypted during the inference process; S502, determining whether there is an operator to be executed in the target deep neural network model, if there is an operator to be executed, obtaining the next operator to be executed as the current to-be-executed operator based on the model architecture of the target deep neural network model; if not, jump to step S518; otherwise, jump to step S503; S503, determining whether the current sensitive layer operator execution sequence is empty, if empty, jump to step S504; if not empty, jump to step S507; S504, determining whether the current to-be-executed operator is a non-sensitive layer operator, if the to-be-executed operator is a non-sensitive layer operator, jump to step S505; if the current to-be-executed operator is a sensitive layer operator, jump to step S506; S505, calling the PyTorch underlying runtime to perform inference calculation on the to-be-executed non-sensitive operator, returning and saving the intermediate result for constructing the input of subsequent operator inference, resetting the values of the two flag bits Flag_E and Flag_D to False, and jumping to step S502; S506, adding the sensitive layer operator to the sensitive layer operator execution sequence, and jumping to step S502; S507, determining whether the current to-be-executed operator satisfies the conditions of belonging to a sensitive layer operator and not having a full connection characteristic for the input tensor, if both conditions are met, jump to step S506; otherwise, jump to step S508; S508, determining whether the current to-be-executed operator satisfies the conditions of belonging to a sensitive layer operator and having a full connection characteristic for the input tensor, if both conditions are met, setting the value of the flag bit Flag_E to True; S509, sending the sensitive layer operator execution sequence, the input tensor, and the two flag bits Flag_E and Flag_D to each sub-thread; S510, each sub-thread calls SMC in parallel on different cores and embeds an independent TEE, and passes in parameters including a sequence of sensitive layer operator execution, input tensors, and two flags Flag_E and Flag_D; S511, the inference framework inside the TEE on each core determines whether the value of the flag Flag_D is True, and if True, uses a pre-set symmetric key to decrypt the input tensors; S512, the inference framework inside the TEE on each core completes parallel inference of the continuous sensitive layers in the execution sequence and obtains output tensors, and determines whether the value of the flag Flag_E is True, and if True, uses a pre-set symmetric key to encrypt the output tensors to generate new output tensors; S513, the TEE on each core returns the output tensors obtained by sensitive layer inference to the corresponding sub-thread on the REE side, and each sub-thread aggregates the output tensors in the channel dimension to generate new output tensors as intermediate results for constructing inputs for subsequent operator inference; S514, the sensitive layer operator execution sequence is emptied; S515, it is determined whether there are still operators to be executed, and if not, the step S519 is jumped to; S516, it is determined whether the current operator to be executed is a non-sensitive layer operator, and if so, the step S505 is jumped to; otherwise, the step S517 is jumped to; S517, the value of the flag Flag_D is set to Flag_E, and the value of the flag Flag_E is reset to False and the step S506 is jumped to; S518, it is determined whether the current sensitive layer operator execution sequence is empty, and if not, the step S509 is jumped to; otherwise, the step S519 is jumped to; S519, the latest output tensors of the target deep neural network model are obtained as the final inference results.

[0011] In addition, the application also provides a TEE-based edge deep neural network model inference system, which comprises a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the TEE-based edge deep neural network model inference method.

[0012] In addition, the application also provides a computer readable storage medium, which stores a computer program or instructions, and the computer program or instructions are programmed or configured to execute the TEE-based edge deep neural network model inference method by a processor.

[0013] In addition, the present application also provides a computer program product comprising a computer program or instructions programmed or configured to execute the TEE-based edge deep neural network model inference method by a processor.

[0014] Compared with the prior art, the present application mainly has the following beneficial effects: 1. The present application includes memory requirement derivation of sensitive layer operators to determine the memory requirements of the sensitive layer operators, and by constructing a static memory allocation plan for the sensitive layer, it can avoid the additional runtime overhead and memory fragmentation risk brought by PyTorch instant mapping for TEE environment.

[0015] 2. The present application includes sensitive layer task segmentation according to the memory dependency information of the sensitive layer on the target edge device, based on the preset parallel optimization core number n, the sensitive layer task is divided into n sub-tasks, according to the mapping relationship between the operator input and output and the calculation characteristics, the calculation task of the sensitive layer operator is adaptively divided and distributed to multiple independent sessions for parallel execution, thereby improving the execution efficiency of the sensitive layer and relieving the performance pressure of TEE.

[0016] 3. The design of the present application is compatible with the inference framework PyTorch, aiming to provide efficient operator-level confidential protection using TEE, while ensuring that the confidential computing module seamlessly integrates into the mainstream machine learning workflow. Based on the ARM TrustZone hardware security extension technology, the present application has the advantages of high execution efficiency, smaller safe memory occupation, simple model script construction and stronger universality under the premise of ensuring the confidentiality and integrity of the model sensitive layer data. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The basic flowchart of the method of the embodiment of the present application is shown.

[0018] Figure 2 The system structure diagram in the embodiment of the present application is shown.

[0019] Figure 3 The encryption and decryption principle diagram of the weight parameters of the sensitive layer in the embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] In order to make the personnel in the technical field better understand the technical solutions of the present application, the technical solutions of the present application will be further described in detail below in combination with the drawings in the embodiment of the present application.

[0021] As shown in the figure, the TEE-based edge deep neural network model inference method of the present embodiment includes the following steps: Figure 1 ​S101, obtain the specified sensitive layer in the target deep neural network model, and the remaining layers are non-sensitive layers; S102, memory requirement derivation is performed on the sensitive layer operator to determine the memory requirement of the sensitive layer operator; S103, training of the target deep neural network model is completed; S104, the weight parameter information of the sensitive layer in the target deep neural network model after training is completed is encrypted; S105, the weight parameter information of the non-sensitive layer of the target deep neural network model, the encrypted weight parameter information of the sensitive layer, and the memory dependency information of the sensitive layer are serialized to generate a model file in a specified format (such as.pt format); S106, the model file is deployed and submitted to a target edge device with TEE and REE; S107, according to the memory dependency information of the sensitive layer, the sensitive layer task is divided into n sub-tasks based on the preset parallel optimization core number n on the target edge device; S108, the non-sensitive layer in the target deep neural network model is constructed in the REE, and the n sub-tasks generated by the sensitive layer are constructed in the TEE, and the construction of the target deep neural network model is completed; S109, the system interface of the target deep neural network model is called on the REE side of the target edge device to perform model inference; S110, the system interface of the target deep neural network model is called on the REE side of the target edge device to obtain the model inference result of the target deep neural network model.

[0022] Step S101 is used to obtain the specified sensitive layer in the target deep neural network model, and the remaining layers are non-sensitive layers. The specification of the sensitive layer in the target deep neural network model can be specified during the development of the model.

[0023] As Figure 2As shown, a lightweight operator library and a reasoning framework implemented in C are transplanted in the TEE environment in this embodiment. Since the TEE is only responsible for the inference of the sensitive layer, only the forward propagation logic is provided. The operator library handles the actual operator computation, while the reasoning framework receives the call from the REE end, manages the model construction, operator invocation in the inference process, and result return. A hybrid C / C++ operator library is implemented in the REE environment in this embodiment, which is consistent with the inference behavior of the TEE end, while supporting back propagation to enable sensitive layer training. The C++ component is integrated into the TorchScript module of PyTorch to support model serialization, static graph construction, and provide a unified high-level calling interface. Memory analysis is used to infer the memory dependency of the sensitive layer operator through the static graph provided by TorchScript, so as to realize the operator management strategy based on pre-allocation. The semantic segmenter further applies the semantic segmentation-based parallel optimization scheme to the sensitive layer operator according to the memory dependency. Step S102 is used to obtain the memory dependency information of all sensitive layers by analyzing the structure information of the target deep neural network model, so as to complete resource allocation before inference. Taking five commonly used operators as an example: convolution, batch normalization, pooling, activation, and full connection. They constitute the core computing modules of most deep neural network models, cover the main functional stages from feature extraction to classification decision, and cover various memory demand modes that operators may involve in the inference process. The memory demand of the five types of operators in the forward inference stage is analyzed in this embodiment, and the memory dependent on the operator is divided into two categories: static resident memory: used to save structural parameters, that is, weights, biases, mean, and variance, the size of which can be determined in the operator declaration stage. Dynamic temporary memory: including memory for saving intermediate activation values or workspace for assisting operator execution, the size of which needs to be derived in combination with the structural parameters of the operator and the dimension information of the input tensor, wherein the intermediate activation value is equivalent to the output of each operator. Table 1 shows the dependency of the five types of operators on static resident memory and dynamic temporary memory. This result provides a direct basis for the dynamic temporary memory derivation process in the subsequent memory pre-allocation strategy.

[0024] Table 1: Dependency of five types of operators on static resident memory and dynamic temporary memory

[0025] In Table 1, Conv represents convolution, BN represents batch normalization, Pool represents pooling, Act represents activation, FC represents full connection, represents that the memory requirement in this part must exist, represents that the memory requirement in this part may exist, and represents that the memory requirement in this part does not exist. In step S102, the memory requirement of the sensitive layer operator is derived to determine the memory requirement of the sensitive layer operator, including determining the static resident memory required by the sensitive layer operator according to the structural parameters declared by the sensitive layer operator, the structural parameters including part or all of the weight, the bias, the mean and the variance, deriving the dynamic temporary memory required by all sensitive layer operators according to the calculation graph of the target deep neural network model and the dimension information of the input tensor, the dynamic temporary memory including the occupied memory of the intermediate activation value and the workspace, To obtain the dynamic temporary memory dependency information of all sensitive layers, first, the entire model is converted into a static computation graph through the torch.jit.script interface provided by TorchScript, and the computation graph expresses the dependency relationship and execution order of all operators in the form of a directed acyclic graph. Then, the computation graph is traversed in combination with a predefined input sample. All memory dependency information of the sensitive layers (including the dynamic temporary memory information derived by the algorithm and the static resident memory information specified by the structural parameters in the model definition stage) obtained by the cloud system before the model deployment will be persisted and stored as a binary intermediate representation through the serialization mechanism of TorchScript, so as to avoid the additional overhead introduced by repeated dynamic temporary memory derivation in the model deployment stage. Before the model inference, the client program will complete the initialization of all sensitive layers in the TEE according to this information, and pre-allocate memory resources. The sensitive layer will not exist in the memory operation in the inference process. It can be observed that the intermediate activation value of each layer will be saved during the model execution process to support the back propagation in the training stage or be referenced by subsequent operators. However, in the TEE environment that only performs forward inference, this mechanism will cause the waste of secure memory. To solve this problem, the inference framework in the TEE will create two memory areas based on the memory dependency information of the sensitive layer to alternately save the input and output data of each layer. The size of each memory area is equal to the maximum value of the intermediate activation value, and in addition, considering that residual connections are widely used in deep neural networks, the shortcut path across layers needs to share part of the intermediate activation value. In this embodiment, an interface for saving the intermediate activation value is designed for each sensitive layer operator to ensure that the cross-layer dependency relationship is correctly maintained. The above optimization can significantly reduce the secure memory overhead of the intermediate activation value of the sensitive layer. Specifically, in this embodiment, the dynamic temporary memory required by all sensitive layer operators is derived according to the calculation graph of the target deep neural network model and the dimension information of the input tensor, including: S201, obtain the directed acyclic graph DAG=(V,E) and input tensor set X of the target deep neural network model, where V is the set of nodes, E is the set of edges, nodes are the computation operations of neural network operators, and edges are the data transmission between computation operations; S202, perform topological sorting on the directed acyclic graph (DAG) and generate queues according to the dependency order between nodes; S203, determine whether there are any unprocessed nodes in the topologically sorted queue; if so, obtain an unprocessed node as the current node v; if there are no unprocessed nodes, jump to step S209. S204, determine if the in-degree of the current node v is 0; if so, select the corresponding input tensor x from the input tensor set X; otherwise, aggregate the output tensors of all its predecessor nodes as the input tensor x of the current node v. S205, determine whether the current node v belongs to the sensitive layer. If it does not belong to the sensitive layer, jump to step S208; otherwise, jump to step S206. S206, Obtain the dimension information x of the input tensor x. d Based on the dimensional information x of the input tensor x d The memory size s required to determine the intermediate activation value of the current node v v ; S207, determine whether the current node v depends on the workspace. If it does not depend on the workspace, then set the memory size w required by the workspace of the entry node v. v Set to 0; otherwise, based on the dimension information x of the input tensor x. d Determine the memory size w required for the workspace of entry node v. v ; S208, Perform forward propagation operation on the current node v, calculate the output based on the input tensor x, and store the result for use by its successor nodes, then jump to step S203; S209, Returns the dynamic temporary memory dependency information of all accumulated sensitive layer nodes, End.

[0026] In step S206 of this embodiment, the dimension information x of the input tensor x is used. d The memory size s required to determine the intermediate activation value of the current node v v Includes: dimensional information x based on the input tensor x d Based on the derivation rules of the output size of the operator corresponding to the current node v, the shape of the output tensor of the current node v is determined, and the memory size s required for the output tensor of the current node v, i.e., the intermediate activation value, is calculated in combination with the data type of the output tensor. v In step S207, the dimension information x is based on the input tensor x. ddetermining the memory size w required for the workspace of the entry node v v comprising: determining the memory size w required for the workspace of the entry node v based on the dimension information x of the input tensor d , in combination with the computing characteristics of the operator corresponding to the entry node v, determining the temporary buffer space size required for supporting the intermediate calculation in the execution process to support the accelerated computing operation as the workspace memory size w v , the accelerated computing operation including im2col conversion.

[0027] The step S104 of the embodiment includes: generating a temporary elliptic curve key pair composed of a temporary public key PK_temp and a temporary private key SK_temp, performing ECDH key agreement using the temporary public key PK_temp and the fixed elliptic curve public key PK_tee provided in the TEE environment of the target edge device to form a shared key K_shared and a symmetric key derived therefrom; locally encrypting the weight parameters of the sensitive layer in the trained target deep neural network model using the symmetric key derived from the shared key K_shared; and when decrypting the encryption result of the weight parameters of the sensitive layer, the TEE environment of the target edge device uses its fixed elliptic curve private key SK_tee and the received temporary public key PK_temp to perform ECDH key agreement again to obtain the same shared key K_shared and the symmetric key derived therefrom, and decrypt the encryption result of the weight parameters of the sensitive layer using the derived symmetric key, and finally restore the original weight parameters of the sensitive layer in the TEE environment of the target edge device.

[0028] Step S107 is used to perform semantic segmentation-based parallel optimization, that is, on the target edge device, sensitive layer tasks are segmented according to the memory dependency information of the sensitive layer, and based on the pre-set number of parallel optimization cores n, the sensitive layer tasks are divided into n sub-tasks. Given that existing TEEs lack native support for multi-threaded parallel execution, this embodiment proposes a method for designing and controlling parallel execution logic on the REE side, thereby bypassing the limitations of the TEE execution model. This method utilizes the reentrancy characteristic of the security application TA: in OP-TEE, a single security application TA instance allows multiple TEEC_Sessions to access it concurrently. That is, multiple CA threads from the REE can interact with the same security application TA simultaneously through their respective independent Sessions (i.e., TEEC_Session instances). When receiving concurrent requests, the security application TA maintains an independent context environment for each Session, including input parameters, state variables, and execution flow, ensuring that each Session does not interfere with the others. This mechanism ensures that the TA does not introduce state conflicts or data race problems when processing multiple tasks concurrently. Furthermore, the OP-TEE scheduler can distribute concurrent requests from multiple sessions to different processor cores for execution. Combined with the reentrancy of the security application TA, the method in this embodiment achieves true cross-core parallel processing capabilities within the TEE. Specifically, step S107 in this embodiment includes: S301, Obtain the sensitive layer operator linked list consisting of all sensitive layers in the target deep neural network model. , and the preset number of parallel optimization cores n; S302, check the validity of the preset number of parallel optimization cores n: n must be greater than 1 and less than or equal to the minimum value of the number of cores on the physical device and the output channels of the sensitive layer operator. If n does not meet the conditions, report an error and jump to step S308; otherwise, jump to step S303. S303, based on the parallel optimization core number n, constructs n empty sub-linked lists in the REE on the target edge device. Each sub-linked list This represents a subtask after being segmented, and serves as the basis for constructing the sensitive layer of the target deep neural network model in the TEE; S304, Determine the sensitive layer operator list If there are any unprocessed operators, then retrieve the unprocessed operators as the current operator and jump to step S305; otherwise, jump to step S308. S305, Determine whether there are weight parameters that need to be segmented in the channel dimension of the output tensor of the current operator. If there are, jump to step S306; otherwise, jump to step S307. S306, split the weight parameters of the current operator based on the output channels to generate the split sub-operators which are inserted into each sub-chain table in turn S307, insert a copy of the current operator into each sub-chain table S308, return the final output, which contains n sensitive layer sub-lists generated by n sub-chain tables S308, return the final output, which contains n sensitive layer sub-lists generated by n sub-chain tables

[0029] Traditional cycle-level parallel schemes are usually limited to local solutions within operators, lacking a global perspective of the entire model. In the TEE scenario, such schemes often result in frequent world switching operations to coordinate intermediate data and boundary processing. To overcome the above problems, the embodiment proposes a semantic-aware operator partitioning strategy: for the computing tasks of multiple sensitive layer operators, uniformly split along the semantically consistent dimensions (e.g., output channels), and assign continuous sub-task sequences to each thread. This strategy improves the consistency of cross-layer execution and significantly improves the overall parallel efficiency within TEE.

[0030] In the embodiment, step S108 includes: S401, deserialize the neural network model file into the REE memory on the REE side of the target edge device using the TorchScript interface in PyTorch, where the non-sensitive layers of the model will be delegated to the PyTorch underlying runtime management and initialization during the deserialization process; S402, based on the number of parallel optimization cores n passed through the legality check, create n sub-threads in the REE of the target edge device, each corresponding to a sensitive layer sub-list generated by task partitioning, and each sub-thread will run on a different core; S403, each sub-thread creates an independent session for communication with the TEE and calls SMC to embed the core where it is located into the TEE, and sequentially sends all sensitive operators in the sub-list after task partitioning to the TEE, including structural parameters and dynamic temporary memory requirement information; the structural parameters include part or all of the weights, biases, means, and variances; the dynamic temporary memory requirement information includes intermediate activation value memory size and workspace memory size; S404, the inference framework in the TEE of the target edge device accepts the parameters for constructing the sensitive layers of the model from the REE side and completes the initialization operation of the sensitive layer operators.

[0031] The system interface of the target deep neural network model is called on the REE side of the target edge device in step S109 of the embodiment to perform model inference, which includes: ​​S501, obtain an input tensor as an input of the target deep neural network model at the REE side, and initialize an empty sensitive layer operator execution sequence at the REE side, and set the values of two flags Flag_E and Flag_D as False to respectively indicate whether the intermediate result tensor needs to be encrypted or decrypted during the inference process; S502, determine whether there is an operator to be executed in the target deep neural network model, if there is an operator to be executed, obtain the next operator to be executed as the current to-be-executed operator based on the model architecture of the target deep neural network model; if not, jump to step S518; otherwise, jump to step S503; S503, determine whether the current sensitive layer operator execution sequence is empty, if empty, jump to step S504; if not empty, jump to step S507; S504, determine whether the current to-be-executed operator is a non-sensitive layer operator, if the to-be-executed operator is a non-sensitive layer operator, jump to step S505; if the current to-be-executed operator is a sensitive layer operator, jump to step S506; S505, call the PyTorch underlying runtime to perform inference calculation on the to-be-executed non-sensitive operator, return and save the intermediate result for constructing the input of subsequent operator inference, reset the values of the two flags Flag_E and Flag_D as False, and jump to step S502; S506, add the sensitive layer operator to the sensitive layer operator execution sequence, and jump to step S502; S507, determine whether the current to-be-executed operator satisfies the conditions of belonging to a sensitive layer operator and not having a full connection characteristic for the input tensor, if both conditions are met, jump to step S506; otherwise, jump to step S508; S508, determine whether the current to-be-executed operator satisfies the conditions of belonging to a sensitive layer operator and having a full connection characteristic for the input tensor, if both conditions are met, set the value of the flag Flag_E as True; S509, send the sensitive layer operator execution sequence, the input tensor, and the two flags Flag_E and Flag_D to each sub-thread; S510, each sub-thread calls SMC in parallel on different cores and embeds an independent TEE, and inputs parameters including the sensitive layer operator execution sequence, the input tensor, and the two flags Flag_E and Flag_D; S511, the inference framework inside the TEE on each core determines whether the value of the flag Flag_D is True, if True, decrypts the input tensor using a pre-set symmetric key; S512, the inference framework in the TEE on each core completes the parallel inference of the continuous sensitive layers in the execution sequence and obtains an output tensor, and judges whether the value of the flag bit Flag_E is True, and if True, encrypts the output tensor using a pre-set symmetric key to generate a new output tensor; S513, the TEE on each core returns the output tensor obtained by the sensitive layer inference to the corresponding sub-thread on the REE side, and each sub-thread aggregates the output tensors in the channel dimension to generate a new output tensor as an intermediate result for constructing the input of the subsequent operator inference; S514, clearing the sensitive layer operator execution sequence; S515, judging whether there are still operators to be executed, and if not, jumping to step S519; S516, judging whether the current operator to be executed is a non-sensitive layer operator, and if so, jumping to step S505; otherwise, jumping to step S517; S517, setting the value of the flag bit Flag_D to Flag_E, and resetting the value of the flag bit Flag_E to False and jumping to step S506; S518, judging whether the current sensitive layer operator execution sequence is empty, and if not, jumping to step S509; otherwise, jumping to step S519; S519, obtaining the latest output tensor of the target deep neural network model as the final inference result.

[0032] Let the continuous sensitive layer operators be represented as , where n≥1. All operators can be divided into sub-tasks , which will be allocated to threads on the REE side for execution respectively. Each thread communicates with the same reentrant safe application TA through an independent session, where . Define to represent the set of sub-tasks allocated to the session, , then: , that is, each is composed of the sub-tasks in all sensitive layer operators, forming a sequence of sub-tasks that can be continuously executed in the TEE, corresponding to the session. Each sub-task corresponds to a sub-computational expression obtained by dividing a certain operator in the semantic dimension (such as the output channel), and its abstract computational form is: . Where represents the The computation function of each operator, It is the original input tensor A subset along a certain semantic dimension (such as channels or output neurons). This is the output result corresponding to the subtask. This represents the local operator parameters used for this subtask. For the entire set of subtasks... The system communicates with the reentrant security application TA. Each computation function is computed concurrently in an independent session. All computational functions The computational tasks are data-independent, thus allowing them to be executed in parallel across different sessions without handling cross-operator dependencies. Furthermore, based on the sub-computation formulas for different types of operators, differences in input data dependencies can be observed: for operators with local dependencies (such as normalization, pooling, and activation), the input and output of subtasks are strictly aligned along the channel dimension, satisfying the following relationship: Conversely, for convolutional layers and fully connected layers, the first... Subtasks of the operators The input tensor, depending on the outputs of all subtasks of the previous operator, should satisfy: .in, This represents a concatenation function that joins the outputs of all subtasks along a specific dimension. For example... Figure 3 As shown, TEE for Session_1, TEE for Session_2, and TEE for Session_P represent P (P=3) independent sessions of the security application TA, considering the set of subtasks: If there exists a continuous subsequence within it. satisfy Furthermore, since all these subtasks originate from operators with local dependencies, then for any , Its execution does not depend on other task sets. The intermediate activation value can be used to execute consecutively within the same session. However, if A subtask in For convolutional or fully connected layers, their input tensors depend on the subtask outputs of the previous operator across all sessions: At this point, the intermediate activation values ​​of each subtask in the TEE must be returned to the REE through their respective sessions and concatenated to construct the ... The required complete input tensor.

[0033] In summary, the TEE-based edge deep neural network model inference method of the embodiment faces a PyTorch target deep neural network model, uses TEE to provide efficient operator-level confidential protection, and at the same time ensures that the confidential computing module is seamlessly integrated into the mainstream machine learning workflow. Specifically, the following four core design goals are included: (1) make TEE execute model inference: (2) provide PyTorch-compatible operator invocation semantics for operators executed in TEE; (3) pre-allocate secure memory for sensitive layers before inference; and (4) optimize inference on sensitive layers in TEE through parallelization. The embodiment includes memory requirement derivation for sensitive layer operators to determine the memory requirements of the sensitive layer operators, and by constructing a static memory allocation plan for the sensitive layers, the additional runtime overhead and memory fragmentation risk brought by PyTorch just-in-time mapping for the TEE environment can be avoided. The method includes dividing the sensitive layer task according to the memory dependency information of the sensitive layer on the target edge device, dividing the sensitive layer task into n sub-tasks based on the preset number of parallel optimization cores n, and adaptively dividing the calculation task of the sensitive layer operators according to the mapping relationship between the operator input and output and the calculation characteristics, and distributing to multiple independent sessions for parallel execution, thereby improving the execution efficiency of the sensitive layer and relieving the performance pressure of TEE. The design of the method is compatible with the inference framework PyTorch, can use TEE to provide efficient operator-level confidential protection, and at the same time ensures that the confidential computing module is seamlessly integrated into the mainstream machine learning workflow. Based on the ARM TrustZone hardware security extension technology, the embodiment has the advantages of high execution efficiency, smaller secure memory occupation, simple model script construction, and stronger universality while ensuring the confidentiality and integrity of the model sensitive layer data.

[0034] In addition, the embodiment also provides a TEE-based edge deep neural network model inference system, which includes a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the TEE-based edge deep neural network model inference method. The embodiment also provides a computer readable storage medium having a computer program or instructions stored therein, which are programmed or configured to execute the TEE-based edge deep neural network model inference method by a processor. The embodiment also provides a computer program product including a computer program or instructions, which are programmed or configured to execute the TEE-based edge deep neural network model inference method by a processor.

[0035] Those skilled in the art will appreciate that the technology of the present application can be embodied in a method, system, or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer usable program code. The specification and drawings are, accordingly, to be regarded as illustrative and not restrictive. The present application is described in terms of flowcharts and / or block diagrams in which each block indicates a set of elements carried out normally by a computer program. Those skilled in the art will appreciate that each block of the flowcharts and / or block diagrams, and combinations thereof, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowcharts and / or block diagrams block or blocks. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowcharts and / or block diagrams block or blocks. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowcharts and / or block diagrams block or blocks. According to a further aspect of the present application, there is provided an apparatus comprising: means for performing the steps of the methods described herein; and means for causing the steps to be performed. Figure 1 one or more of the flowcharts and / or block diagrams. Figure 1 one or more of the flowcharts and / or block diagrams. Figure 1 one or more of the flowcharts and / or block diagrams. Figure 1 one or more of the flowcharts and / or block diagrams. Figure 1 one or more of the flowcharts and / or block diagrams. Figure 1 one or more of the flowcharts and / or block diagrams.

[0036] The above description is only preferred embodiments of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any improvement and modification made by those skilled in the art without departing from the principle of the present application should be considered as falling within the protection scope of the present application.

Claims

1. A method for edge-based deep neural network model inference based on TEE, characterized in that, Includes the following steps: S101: Obtain the specified sensitive layer in the target deep neural network model, and treat the remaining layers as non-sensitive layers; S102, Determine the memory requirements of the sensitive layer operators by deriving the memory requirements of the sensitive layer operators; S103, complete the training of the target deep neural network model; S104, encrypt the weight parameter information of the sensitive layer in the target deep neural network model after training; S105, serialize the weight parameter information of the non-sensitive layer, the encrypted weight parameter information of the sensitive layer, and the memory dependency information of the sensitive layer of the target deep neural network model to generate a model file in a specified format; S106, Deploy and submit the model file to the target edge device with TEE and REE; S107, On the target edge device, the sensitive layer task is divided according to the memory dependency information of the sensitive layer. Based on the preset number of parallel optimization cores n, the sensitive layer task is divided into n sub-tasks. S108, build the non-sensitive layers in the target deep neural network model in the REE, and build the n sub-tasks generated by the sensitive layer segmentation in the TEE, thus completing the construction of the target deep neural network model; S109, On the REE side of the target edge device, call the system interface of the target deep neural network model to perform model inference; S110, on the REE side of the target edge device, call the system interface of the target deep neural network model to obtain the model inference result of the target deep neural network model.

2. The edge-based deep neural network model inference method based on TEE according to claim 1, characterized in that, In step S102, when deriving the memory requirements of sensitive layer operators to determine their memory needs, this includes determining the static resident memory required by the sensitive layer operators based on the structural parameters declared by the operators. These structural parameters include some or all of the weights, biases, mean, and variance. The dynamic temporary storage memory required by all sensitive layer operators is then derived based on the computation graph of the target deep neural network model and the dimensionality information of the input tensor. This dynamic temporary storage memory includes the memory occupied by intermediate activation values ​​and the workspace. The derivation of the dynamic temporary storage memory required by all sensitive layer operators based on the computation graph of the target deep neural network model and the dimensionality information of the input tensor includes: S201, obtain the directed acyclic graph DAG=(V,E) and input tensor set X of the target deep neural network model, where V is the set of nodes, E is the set of edges, nodes are the computation operations of neural network operators, and edges are the data transmission between computation operations; S202, perform topological sorting on the directed acyclic graph (DAG) and generate queues according to the dependency order between nodes; S203, determine whether there are any unprocessed nodes in the topologically sorted queue; if so, obtain an unprocessed node as the current node v; if there are no unprocessed nodes, jump to step S209. S204, determine whether the in-degree of the current node v is 0; if so, select the corresponding input tensor x from the input tensor set X; otherwise, aggregate the output tensors of all its predecessor nodes as the input tensor x of the current node v. S205, determine whether the current node v belongs to the sensitive layer. If it does not belong to the sensitive layer, jump to step S208; otherwise, jump to step S206. S206, Obtain the dimension information x of the input tensor x. d Based on the dimensional information x of the input tensor x d The memory size s required to determine the intermediate activation value of the current node v v ; S207, determine whether the current node v depends on the workspace. If it does not depend on the workspace, then set the memory size w required by the workspace of the entry node v. v Set to 0; otherwise, based on the dimension information x of the input tensor x. d Determine the memory size w required for the workspace of entry node v. v ; S208, Perform forward propagation operation on the current node v, calculate the output based on the input tensor x, and store the result for use by its successor nodes, then jump to step S203; S209, Returns the dynamic temporary memory dependency information of all accumulated sensitive layer nodes, End.

3. The edge-based deep neural network model inference method based on TEE according to claim 2, characterized in that, In step S206, the dimension information x is based on the input tensor x. d The memory size s required to determine the intermediate activation value of the current node v v Includes: dimensional information x based on the input tensor x d Based on the derivation rules of the output size of the operator corresponding to the current node v, the shape of the output tensor of the current node v is determined, and the memory size s required for the output tensor of the current node v, i.e., the intermediate activation value, is calculated in combination with the data type of the output tensor. v In step S207, the dimension information x is based on the input tensor x. d Determine the memory size w required for the workspace of entry node v. v Includes: dimensional information x based on the input tensor d Based on the computational characteristics of the operator corresponding to the entry node v, the size of the temporary buffer space required to support intermediate computations during execution is determined as the workspace memory size w. v The accelerated computing operation includes the im2col conversion.

4. The edge-based deep neural network model inference method based on TEE according to claim 1, characterized in that, Step S104, which encrypts the weight parameter information of the sensitive layer in the target deep neural network model after training, includes: generating a temporary elliptic curve key pair consisting of a temporary public key PK_temp and a temporary private key SK_temp; performing ECDH key negotiation using the temporary public key PK_temp and the fixed elliptic curve public key PK_tee provided in the TEE environment of the target edge device to obtain a shared key K_shared and its derived symmetric key; using the symmetric key derived from the shared key K_shared to locally encrypt the weight parameters of the sensitive layer in the target deep neural network model after training; and when decrypting the encryption result of the weight parameters of the sensitive layer, the TEE environment of the target edge device uses its fixed elliptic curve private key SK_tee and the received temporary public key PK_temp to perform ECDH key negotiation again to obtain the same shared key K_shared and its derived symmetric key, and uses the derived symmetric key to decrypt the encryption result of the weight parameters of the sensitive layer, finally recovering the original weight parameters of the sensitive layer in the TEE environment of the target edge device.

5. The edge-based deep neural network model inference method based on TEE according to claim 1, characterized in that, Step S107 includes: S301, Obtain the sensitive layer operator linked list consisting of all sensitive layers in the target deep neural network model. , and the preset number of parallel optimization cores n; S302, check the validity of the preset number of parallel optimization cores n: n must be greater than 1 and less than or equal to the minimum value of the number of cores on the physical device and the output channels of the sensitive layer operator. If n does not meet the conditions, report an error and jump to step S308; otherwise, jump to step S303. S303, based on the parallel optimization core number n, constructs n empty sub-linked lists in the REE on the target edge device. Each sub-linked list This represents a subtask after being segmented, and serves as the basis for constructing the sensitive layer of the target deep neural network model in the TEE; S304, Determine the sensitive layer operator list If there are any unprocessed operators, then retrieve the unprocessed operators as the current operator and jump to step S305; otherwise, jump to step S308. S305, Determine whether there are weight parameters that need to be segmented in the channel dimension of the output tensor of the current operator. If there are, jump to step S306; otherwise, jump to step S307. S306, the weight parameters of the current operator are divided based on the output channel, and the resulting sub-operators are inserted sequentially into the respective sub-linked lists. middle; S307, Copy the current operator to each sub-list. Insert one copy into the middle; S308 returns the final output, which contains n sub-linked lists. The process ends after generating a list of n sensitive sub-layers.

6. The edge-based deep neural network model inference method based on TEE according to claim 1, characterized in that, Step S108 includes: S401, On the REE side of the target edge device, the neural network model file is deserialized into the REE memory using the TorchScript interface in PyTorch. The non-sensitive layers of the model will be delegated to the PyTorch underlying runtime for management and initialization during the deserialization process. S402, based on the number of parallel optimization cores n that pass the legality check, creates n sub-threads in the REE of the target edge device. Each sub-thread manages a sensitive layer sub-list generated by the task partitioning. Each sub-thread will run on a different core. S403, each sub-thread creates an independent session for communicating with the TEE and calls SMC to embed its core into the TEE, and sequentially sends the parameters of all sensitive operators in the sub-list after task segmentation to the TEE, including structural parameters and dynamic temporary storage memory requirements; the structural parameters include some or all of the weights, biases, means, and variances; the dynamic temporary storage memory requirements include the memory size of intermediate activation values ​​and the memory size of the workspace; S404, the inference framework in the TEE of the target edge device accepts the parameters from the REE side for building the sensitive layer of the model and completes the initialization operation of the sensitive layer operator.

7. The edge-based deep neural network model inference method based on TEE according to claim 1, characterized in that, Step S109, which involves calling the system interface of the target deep neural network model on the REE side of the target edge device to perform model inference, includes: S501, obtain the input tensor on the REE side as the input of the target deep neural network model, and initialize an empty sensitive layer operator execution sequence on the REE side, and initialize the values ​​of two flag bits Flag_E and Flag_D to False to indicate whether the intermediate result tensor needs to be encrypted or decrypted during the inference process, respectively. S502, determine whether there are any operators to be executed in the target deep neural network model. If there are any operators to be executed, obtain the next operator to be executed based on the model architecture of the target deep neural network model as the current operator to be executed; if there are none, jump to step S518; otherwise, jump to step S503. S503, determine whether the current sensitive layer operator execution sequence is empty. If it is empty, proceed to step S504; if it is not empty, proceed to step S507. S504, determine whether the operator to be executed is a non-sensitive layer operator. If the operator to be executed is a non-sensitive layer operator, jump to step S505; if the operator to be executed is a sensitive layer operator, jump to step S506. S505, call the PyTorch underlying runtime to perform inference calculations on the non-sensitive operators to be executed, return and save the intermediate results for use as input when constructing subsequent operator inference, reset the values ​​of the two flag bits Flag_E and Flag_D to False, and jump to step S502; S506, add the sensitive layer operator to the sensitive layer operator execution sequence, and jump to step S502; S507, determine whether the operator to be executed meets the conditions of being a sensitive layer operator and not having fully connected properties for the input tensor. If both conditions are met, jump to step S506; otherwise, jump to step S508. S508, determine whether the operator to be executed meets the conditions of being a sensitive layer operator and having fully connected characteristics for the input tensor. If both conditions are met, set the value of the flag bit Flag_E to True. S509, send the execution sequence of the fault-sensitive layer operator, the input tensor, and two flag bits Flag_E and Flag_D to each sub-thread; S510, each sub-thread calls SMC in parallel on different cores and embeds its own independent TEE, and passes in parameters including the execution sequence of the fault-sensitive layer operator, the input tensor and two flag bits Flag_E and Flag_D; S511, the inference framework inside the TEE on each core determines whether the value of the flag bit Flag_D is True. If it is True, the input tensor is decrypted using a pre-set symmetric key. S512, the inference framework within the TEE on each core completes parallel inference of the consecutive sensitive layers in the execution sequence, obtains the output tensor, and determines whether the value of the flag bit Flag_E is True. If it is True, the output tensor is encrypted using the pre-set symmetric key to generate a new output tensor. S513, the TEE on each core returns the output tensor obtained by the sensitive layer inference to the corresponding sub-thread on the REE side. Each sub-thread aggregates its own output tensor in the channel dimension to generate a new output tensor as an intermediate result, which is used to construct the input for subsequent operator inference. S514, clear the execution sequence of sensitive layer operators; S515, determine whether there are still operators to be executed; if not, jump to step S519. S516, determine whether the operator to be executed is a non-sensitive layer operator. If yes, jump to step S505; otherwise, jump to step S517. S517, set the value of flag_D to Flag_E, reset the value of flag_E to False and jump to step S506; S518, Determine whether the current sensitive layer operator execution sequence is empty. If it is not empty, proceed to step S509; otherwise, proceed to step S519. S519: Obtain the latest updated output tensor of the target deep neural network model as the final inference result.

8. An edge-based deep neural network model inference system based on TEE, comprising interconnected microprocessors and memory, characterized in that, The microprocessor is programmed or configured to execute the TEE-based edge deep neural network model inference method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the edge-based deep neural network model inference method according to any one of claims 1 to 7 via a processor.

10. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the edge-based deep neural network model inference method according to any one of claims 1 to 7 via a processor.

Images