System for optimizing deep neural network end-side collaborative reasoning

Abstract

The invention discloses a system for optimizing deep neural network end-edge collaborative reasoning, which comprises a sub-model cutting module and an end-edge splitting decision module, and is characterized in that the end-edge splitting decision module comprises a modeling assembly, a selection assembly and a fine tuning assembly, and the sub-model cutting module and the fine tuning assembly cooperate to adjust a splitting deployment decision scheme. The modeling component measures the privacy of the middle output of each layer to obtain a privacy measurement result, and combines the privacy measurement result with a preset reasoning time delay requirement to realize modeling of DNN end-connected collaborative reasoning; and the selection component is used for calling an existing hierarchical end-connected cooperative reasoning strategy, optimizing the privacy and the reasoning time delay, and determining an end-edge splitting scheme. According to the method, DNN end-edge collaborative reasoning modeling fusing privacy measurement and reasoning time delay is designed, so that an end-edge collaborative splitting scheme with optimal privacy and time delay can be quickly found through an end-edge collaborative splitting strategy.

Description

Technical Field

[0001] The present invention relates to the technical field of network edge - side collaborative processing, and particularly to a system for optimizing the edge - side collaborative inference of a deep neural network. Background Art

[0002] In existing networks, the wide deployment of various sensors has significantly increased the data on the edge side, which in turn has led to a substantial increase in the number of computing tasks. Due to the limitations of network bandwidth, storage space, and computing power in the traditional cloud computing model, it is very difficult to process such a large amount of data and tasks in real - time.

[0003] Currently, some technologies have also attempted to divide a deep neural network (DNN) into multiple parts and deploy them separately on edge devices (such as intelligent cameras) and edge nodes (such as edge servers) to achieve collaborative inference. This has improved the inference efficiency of the DNN model in the sensor network. Although collaborative inference effectively utilizes the computing power of edge devices and edge nodes, it also poses potential risks to data privacy. Adversaries such as malicious edge nodes can reconstruct the original input based on the intermediate data transmitted from the edge device to the edge node, which is called a model inversion attack. Subsequently, the reconstructed input can be used to perform additional analysis on sensitive user details (such as age and race), bringing great risks to the security of users' privacy data.

[0004] Moreover, due to the huge difference in computing power between edge nodes and edge devices, existing edge - side collaborative inference strategies often do not allocate computing tasks to edge devices, resulting in the direct upload of the original input to the edge node. As a result, adversaries such as malicious edge nodes can directly analyze sensitive user details from the unprocessed input data, leading to significant data security risks.

[0005] To solve the above problems, existing technologies have also attempted to use adversarial learning techniques to protect data privacy in edge - side collaborative inference. It trains two entities simultaneously - an edge - side task model and a proxy attack model. The goal is to maintain the accuracy of the task model while reducing the accuracy of the attack model. Therefore, for each edge - side task model deployed across different solutions (for example, partitioned at different layers), a unique adversarial learning process is required to resist a certain attack, resulting in a significant increase in its deployment cost. Summary of the Invention

[0006] The purpose of the present invention is to provide a system for optimizing the edge - side collaborative inference of a deep neural network to solve the technical problem in the prior art that the deployment cost is too high due to protecting data privacy.

[0007] The purpose of the present invention is achieved through the following technical solutions:

[0008] A system for optimizing edge - device - assisted collaborative inference of deep neural networks, including a sub - model cutting module and an edge - device splitting decision module. The sub - model cutting module is used to search and traverse the layers of a deep neural network (DNN) model, and split the model into block - level sub - models according to a predetermined inference latency requirement. A part of the split layers is deployed to the edge device, and the remaining layers are deployed to the edge node. The edge - device splitting decision module includes a modeling component, a selection component, and a fine - tuning component, where:

[0009] The sub - model cutting module collaborates with the fine - tuning component to adjust the split - deployment decision scheme. The fine - tuning component adopts a fine - tuning method based on the depth of the DNN layer to ensure that the candidate split scheme maintains sufficient depth of the edge - side model while meeting the predetermined inference latency requirement.

[0010] The modeling component measures the privacy of the intermediate output of each layer, obtains privacy measurement parameters, and combines the privacy measurement parameters with the predetermined inference latency requirement to achieve the modeling of DNN edge - device collaborative inference.

[0011] The selection component is used to call existing hierarchical edge - device collaborative inference strategies to determine the edge - device splitting scheme that optimizes the privacy and the inference latency.

[0012] The sub - model cutting module further includes a pair of latency predictors trained using a linear regression model for each type of DNN layer. The predictors are used to estimate the running latency of each layer in the model on the edge device and the edge node.

[0013] The inputs of the latency predictors include:

[0014] In the convolutional layer, max - pooling layer, and average - pooling layer, the number of channels, the number of filters, the scale of the input feature map, the scale of the kernel, the stride, and the padding parameters are used as the inputs of the latency predictors.

[0015] In the fully - connected layer, the number of input neurons and the number of output neurons are used as the inputs of the latency predictors.

[0016] In the dropout layer and the activation layer with the ReLU function, the number of input elements is used as the input of the latency predictors.

[0017] The sub - model cutting module is also used to perform the process of searching for the block - level sub - model D sub from the DNN model D. This process includes:

[0018] Dividing the model D into a series of DNN blocks: B1, B2, …, B d , assuming that all DNN blocks are initially deployed on the edge node, and the initial inference latency is lat trans (L0)+∑ 1≤i≤fin latedge (L i )。Subsequently, gradually transfer the DNN block from the edge node to the end device, and iteratively update the inference latency until the inference latency just exceeds the threshold th lat , at which time the DNN block on the end device forms D sub , as the block-level sub-model D from the DNN model D sub .

[0019] After the sub-model cutting module cuts and determines the block-level sub-model D sub , when deploying D on the end device sub and deploying the remaining layers of D on the edge node, if the inference latency exceeds the threshold th lat , then it is necessary to implement fine-tuning processing based on the depth of the DNN layer through the fine-tuning component. The fine-tuning processing includes:

[0020] For each layer of the model D, define its depth, and determine the candidate end-edge splitting scheme as P D = D end , D edge . If the inference latency of running D under P D exceeds the threshold th lay , then relocate the layer with the highest depth in D end to D edge , until the inference latency meets the predetermined inference latency requirement.

[0021] The predetermined inference latency requirement is achieved through the introduced inference latency constraint, and the inference latency constraint is defined as follows:

[0022]

[0023] s.t.lat(D, D S ) ≤ th lat

[0024] where Norm(lat(D, P D )) ∈ (0, 1] represents the normalized inference latency, Norm(priv(D, P D )) ∈ (0, 1] represents the normalized privacy measure of running the model D under the splitting scheme P D , α represents the user-defined trade-off factor, s.t.lat(D, P D ) is the latency under the splitting scheme P D , and th lat is the latency threshold.

[0025] The process by which the modeling component obtains the privacy measure parameters includes:

[0026] Express the computation of each DNN layer in model D as the product of a one - dimensional input vector and a two - dimensional parameter matrix;

[0027] Introduce a privacy measure for the intermediate data in the DNN inference process. The privacy measure is used to quantify the challenges faced by an adversary in reconstructing the original input from this data. Specifically, use the rank of the two - dimensional parameter matrix to evaluate the degree to which the output of this layer retains the original input data as the privacy measure parameter. Among them, a high retention degree indicates high privacy, and vice versa indicates poor data privacy.

[0028] The privacy measure parameter includes:

[0029] The privacy measure parameter is the privacy measure of the DNN splitting scheme. The privacy measure of this splitting scheme is regarded as the weighted sum of the privacy measures of the intermediate outputs across multiple DNN layers, and the privacy measures of the intermediate outputs across multiple DNN layers split by the DNN splitting scheme are normalized by the privacy coefficient.

[0030] The DNN splitting scheme P D The calculation formula of the privacy measure includes:

[0031]

[0032] Wherein, is the privacy coefficient of edge (L u , L v );

[0033] The calculation method of the privacy coefficient λ includes:

[0034]

[0035]

[0036] Wherein, the input of L i is composed of the output of , and the output of L i forms the input of θ layers including L j , i < j.

[0037] Based on the DNN model D under the edge - side splitting scheme P D , the system further includes: normalizing the inference latency lat(D, P D ) and the privacy measure priv(D, P D ) which have different value ranges. Among them, the normalization of lat(D, P D ) is achieved based on the upper bound of lat(D, P D ), and the normalization value range of priv(D, P D ) is in (0, 1];

[0038] Based on this normalization method, the modeling process of the modeling component includes:

[0039] If the model D is a chained DNN structure, each splitting scheme of D will correspond to a certain layer ; and executed on the end device, while executed on the edge node, then the trade-off between the normalized inference latency and the normalized privacy measure for running the model D under the splitting P D is:

[0040]

[0041] where m D is the upper bound of the total latency, lat end (L) is the end-side inference latency of L, lat trans (L) is the transmission latency of the output of L, lat edge (L) is the edge-side inference latency of L, and pri(L) is the privacy measure for the output of L;

[0042] If the model D is a directed acyclic graph structure, each splitting scheme of the model D corresponds to a corresponding set of DNN layers {V_end, V_edge}, where the layers in V_end are run on the end device and the layers in V_end are executed on the edge node, then the trade-off between the normalized inference latency and the privacy measure is as follows:

[0043]

[0044] In the directed acyclic graph G, each splitting corresponds to a v end -v edge cut, where the layers in V end / V edge and the vertex v end / v edge together form two subgraphs, and Norm(lat(D, P D )) + αNorm(priv(D, P D )) is equal to the sum of the edge weights in the corresponding v end -v edge cut, is the privacy coefficient.

[0045] Compared with the prior art, the system provided by the present invention for optimizing edge-cloud collaborative inference of deep neural networks can accurately characterize the difficulty for an attacker to reconstruct the original input by designing an efficient privacy measurement scheme for intermediate data in the DNN inference process. Moreover, by designing a DNN edge-cloud collaborative inference model that integrates privacy measurement and inference latency, it is possible to quickly find the optimal splitting scheme for privacy and latency in the edge-cloud collaborative splitting strategy. At the same time, a fast splitting scheme adjustment mechanism is also designed to ensure that the DNN model meets the latency constraint under the final edge-cloud splitting scheme. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0047] Figure 1 Schematic diagram of the edge-cloud collaborative inference random strategy provided for the embodiment of the present invention;

[0048] Figure 2 Schematic diagram of the system architecture applied to the embodiment of the present invention;

[0049] Figure 3 Schematic diagram of the symbol definition provided for the embodiment of the present invention;

[0050] Figure 4 Schematic diagram of the construction process of the DNN block provided for the embodiment of the present invention;

[0051] Figure 5 Schematic diagram of the modeling process of the chain-structured DNN edge-cloud collaborative inference provided for the embodiment of the present invention;

[0052] Figure 6 Schematic diagram of the modeling process of the directed acyclic graph-structured DNN edge-cloud collaborative inference provided for the embodiment of the present invention;

[0053] Figure 7 Schematic diagram of the actual deployment effect provided for the embodiment of the present invention;

[0054] Figure 8 Schematic diagram of the DNN block construction algorithm provided for the embodiment of the present invention;

[0055] Figure 9 Schematic diagram of the SecoInfer algorithm process provided for the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] Combined with the specific content of the present invention, the technical solutions in the embodiments of the present invention will be described clearly and completely; obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments, which does not constitute a limitation to the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0057] First, the terms that may be used in this article are explained as follows:

[0058] The term "and / or" means that either one of the two or both can be realized. For example, X and / or Y means that it includes both the case of "X" or "Y" and the three cases of "X and Y".

[0059] The description of terms such as "comprising", "including", "containing", "having" or other similar semantics should be interpreted as non-exclusive inclusion. For example: including a certain technical feature element (such as raw material, component, ingredient, carrier, dosage form, material, size, part, component, mechanism, device, step, process, method, reaction condition, processing condition, parameter, algorithm, signal, data, product or article, etc.), it should be interpreted as not only including the clearly listed certain technical feature element, but also including other technical feature elements well-known in the art that are not clearly listed.

[0060] The term "consisting of" means excluding any technical feature element that is not clearly listed. If this term is used in a claim, then this term will make the claim a closed type, making it not contain technical feature elements other than the clearly listed technical feature elements, except for related conventional impurities. If this term only appears in a sub-clause of a claim, then it only limits the elements clearly listed in that sub-clause, and the elements recorded in other sub-clauses are not excluded from the overall claim.

[0061] Unless otherwise clearly specified or limited, terms such as "installed", "connected", "joined", "fixed", etc. should be understood in a broad sense. For example: it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in this article can be understood according to specific situations.

[0062] When a concentration, temperature, pressure, size, or other parameter is expressed in the form of a numerical range, the numerical range should be understood as specifically disclosing all ranges formed by the pairing of any upper limit value, lower limit value, and preferred value within the numerical range, regardless of whether the range is explicitly recited; for example, if the numerical range "2 to 8" is recited, then this numerical range should be interpreted as including ranges such as "2 to 7", "2 to 6", "5 to 7", "3 to 4 and 6 to 7", "3 to 5 and 7", "2 and 5 to 7", etc. Unless otherwise specified, the numerical ranges recited herein include both their end values and all integers and fractions within the numerical range.

[0063] The terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. It is only for the convenience of description and simplification of the description, and does not explicitly or implicitly imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to this article.

[0064] To solve the problems existing in the prior art, it is necessary to design an effective DNN edge-cloud collaborative inference strategy. This strategy must ensure the usability of the DNN model (i.e., ensure that the inference latency is small enough) and the privacy of the intermediate transmitted data, making it challenging to reconstruct the original input from the intermediate data.

[0065] As Figure 1 shown, it shows a comparison between the existing collaborative inference strategy that focuses on minimizing the inference latency and the strategy that simultaneously optimizes data privacy and inference latency to be provided in the embodiments of the present invention.

[0066] Specifically, the embodiment of the present invention proposes a SecoInfer (edge-end collaborative inference) implementation solution, that is, a system for optimizing privacy and latency in deep neural network edge-end collaborative inference, which is a hierarchical DNN edge-end collaborative inference framework and can be used as a component of existing edge-end collaborative inference strategies to achieve joint optimization of privacy and latency while complying with latency constraints. In SecoInfer, to quantify the difficulty faced by an attacker in reconstructing the original input, the computation of each DNN layer is also transformed into the form of multiplying an input vector by a parameter matrix, and the rank is used to measure the privacy of the intermediate data. To combine the privacy measurement of the intermediate data with the inference latency of each layer in the DNN, a modeling technique for DNN edge-end splitting is also designed, so that the privacy measurement and inference latency of each splitting scheme can be directly calculated from the elements of the list or the cuts of the directed acyclic graph. To ensure that the splitting scheme complies with the latency constraints, an efficient adjustment algorithm is also designed for the splitting scheme. On the one hand, the search space of feasible solutions is reduced by block-level sub-model cutting, and on the other hand, the final splitting scheme based on the depth fine-tuning of each DNN layer meets the latency constraints.

[0067] As Figure 2 shown, Figure 2 Figure 1 shows the decision-making process of the splitting scheme that simultaneously optimizes privacy and latency for a DNN that needs to be split and deployed across end devices and edge nodes to complete the edge-end collaborative inference process in the SecoInfer implementation solution provided by the present invention. A system for optimizing privacy and latency in deep neural network edge-end collaborative inference (i.e., the SocoInfer implementation solution) provided by the embodiment of the present invention includes a sub-model cutting module and an edge-end splitting decision module, and the latter can be further divided into a modeling component, a selection component, and a fine-tuning component, where:

[0068] The sub-model cutting module and the fine-tuning component cooperate to adjust the splitting and deployment solution to ensure that it is consistent with the latency constraints and meets the latency constraint requirements;

[0069] The modeling component first measures the privacy of the intermediate output of each layer to obtain privacy measurement parameters, and then combines the privacy measurement parameters with the inference latency to complete the modeling of DNN edge-end collaborative inference;

[0070] The selection component can call existing hierarchical edge-end collaborative inference strategies to search for candidate splitting schemes. For example, it can call DADS (Dynamic Adaptive DNN Surgery) and QDMP (Quick Deep Model Partition) to determine the best splitting scheme that simultaneously optimizes data privacy and inference latency.

[0071] For better understanding, the following will describe in detail the specific implementation methods of the embodiments of the present invention in conjunction with the accompanying drawings. First, the definitions of all symbols involved in the subsequent detailed description are as follows Figure 3 as shown.

[0072] In the implementation process of the embodiments of the present invention, it is necessary to measure the privacy of the intermediate output of each layer to ensure that the edge-side splitting scheme can take into account the privacy protection of intermediate data while meeting the latency requirements. To this end, it is first necessary to understand the attacker model, and then it is possible to make it as difficult as possible for an attacker to invert the original data from the obtained intermediate data based on the attacker model.

[0073] Since the edge node only provides computing services to the end device, the edge node cannot interfere with the computing process Z = D end (X) on the end device, so it lacks knowledge of the input X and the internal intermediate data of D end . The edge node can only access the output Z of D end (X), which will be used as the input for the subsequent process Y = D edge (Z). The attacker's goal is to reconstruct the original input X from the intermediate data Z. Based on this, two types of attackers performing model inversion attacks on malicious edge nodes are studied in the present invention. Each type of attacker has different capabilities, and the two attacker models include:

[0074] (1) The white-box attacker has knowledge of the end-side model, including its structure and parameters. Its goal is to identify the sample X ★ that best mimics the original input X. Therefore, the optimization goal of the white-box attacker can be defined as follows:

[0075] ED(X ★ , X) = D end (X ★ ) - D end (X)|| 2

[0076]

[0077]

[0078] ED(X ★ , X) uses the Euclidean distance to measure the similarity between the intermediate data D end (X ★ ) and D end (X); TV(X ★ ) is used to measure X ★Smoothness, where a larger β leads to a smoother reconstruction of the input, and μ is a parameter that balances the influence of ED(·) and TV(·). Since white-box attackers know the structure and parameters of the edge-side model, they can use gradient descent for input reconstruction.

[0079] (2) Due to the lack of knowledge about the edge-side model, black-box attackers cannot directly use gradient descent on D end (X) to reconstruct the original input. However, black-box attackers have the ability to generate a set of inputs {X1, X2, …, X m} and query the corresponding intermediate data {D end (X1), D end (X2), …, D end (X m )}; subsequently, they can use {D end (X1), D end (X2), …, D end (X m )} as the feature set and {X1, X2, …, X m} as the label set to train an inverse network of D end :

[0080]

[0081] After learning the above two attacker models, the present invention can, during the edge-side splitting process, increase the difficulty of obtaining the original input data through the corresponding model based on the privacy measurement strategy, that is, effectively reduce the accuracy of the attacker's reconstruction of the original input through the stolen intermediate data and increase the difficulty of reconstructing the original data.

[0082] While considering privacy issues, the present invention also needs to consider whether the inference latency of the splitting scheme meets the requirements during the specific implementation process. For the DNN model D, the goal of the present invention is to select the best splitting scheme for collaborative inference between the edge device and the edge node to balance reducing the inference latency and reducing the accuracy of input reconstruction. However, excessive attention to the privacy of intermediate data may lead to an overly large edge-side model. Given that the computing power of edge devices is much lower than that of edge nodes, this may result in an overly large inference latency, thus affecting the real-time performance of the DNN model. Therefore, it is necessary to introduce an inference latency constraint to ensure that the corresponding inference latency is within a reasonable range; specifically, the best splitting of the corresponding model D and the related latency constraint definition can be as follows:

[0083]

[0084] s.t.lat(D,Pp)≤th lat

[0085] where, Norm(lat(D, P D )) ∈ (0, 1] represents the normalized inference latency, and Norm(priv(D, P D )) ∈ (0, 1] represents the normalized privacy measure for running model D under the splitting scheme P D , and α represents the user-defined trade-off factor.

[0086] Based on the above description, it can be seen that in the system of edge-cloud collaborative inference of deep neural networks, it is necessary to balance privacy requirements and latency requirements to achieve privacy and latency optimization in the process of edge-cloud collaborative inference of deep neural networks.

[0087] In a system for optimizing edge-cloud collaborative inference of deep neural networks provided by the present invention, specifically, block-level sub-model cutting can be implemented through a sub-model cutting module.

[0088] The implementation method of block-level sub-model cutting will be described in detail below.

[0089] The sub-model cutting module in the present invention is mainly used to search and traverse the layers of the deep neural network DNN model, and perform block-level sub-model splitting according to the predetermined inference latency requirement. After splitting, a part of the layers are deployed to the edge device, and the remaining layers are deployed to the edge node.

[0090] Specifically, the sub-model cutting module aims to efficiently search for the block-level sub-model D sub from the DNN model D. This process only needs to traverse a part of the layers of D. When D sub is deployed to the edge device and the remaining layers are deployed to the edge node, the overall inference latency of the model only slightly exceeds the latency threshold th lat . During the cutting process, moving any layer from the edge node to the edge device will result in an increase in the inference latency.

[0091] Specifically, in the present invention, a linear regression model can be used to train a pair of latency predictors for each type of DNN layer. The predictors are used to estimate the running latency of each layer in the model D on the edge device and the edge node.

[0092] Considering that the running latency of different DNN layers is mainly related to the amount of computation, the configurable parameters of each DNN layer can be used as the input of the corresponding latency predictor. Among them, the configurable parameters for convolutional, max pooling, and average pooling layers include the number of channels, the number of filters, the scale of the input feature map, the scale of the kernel, the stride, and the padding; for the fully connected layer, its input vector is multiplied by the weight matrix to obtain a vector, so the number of input neurons and output neurons can be used as the input of the latency predictor; for the dropout layer and the activation layer with the ReLU function, the input and output have a one-to-one mapping relationship, so the number of input elements can be used as the input of the latency predictor.

[0093] Based on the definitions of the fork layer and the merge layer, the present invention gives the definition of the DNN block, and the process of constructing the DNN block is as Figure 4 shown, and the respective definitions of the DNN block involved are as follows:

[0094] Definition 1: In the DNN model, if the output of a certain layer serves as the input of multiple subsequent layers, then this layer is a fork layer.

[0095] Definition 2: In the DNN model, if the input of a certain layer is formed by combining the outputs of multiple previous layers, then this layer is a merge layer.

[0096] Definition 3; In the DNN model, there is a series of alternating fork layers and merge layers; then it is defined that all layers from the fork layer to the nearest merge layer form a DNN block, and each remaining layer constitutes an independent DNN block.

[0097] The present invention divides the model D into a series of DNN blocks: B1, B2, …, B d . Suppose that at the beginning, all DNN blocks are deployed on the edge node, and the initial inference latency is lat trans (L0)+∑ 1≤i≤fin lat edge (L i ). Subsequently, these DNN blocks are gradually transferred from the edge node to the end device, and the inference latency is iteratively updated. This process continues until the inference latency just exceeds the latency threshold th lat , at which time the DNN blocks on the end device form D sub .

[0098] Through the above process, sub-model cutting at the block level can be achieved.

[0099] Next, the privacy measurement and processing process involved in the implementation process of the present invention will be described in detail.

[0100] First, represent the computation of each DNN layer in model D as the product of a one-dimensional input vector and a two-dimensional parameter matrix. Theorem 1 details how to represent the computations of various types of DNN layers, including convolutional layers, max pooling layers, average pooling layers, fully connected layers, dropout layers, and activation layers (using the ReLU function).

[0101] After that, introduce the privacy measure of the intermediate data during the DNN inference process, aiming to quantify the challenges faced by the adversary in reconstructing the original input from this data. Specifically, for each layer in model D, use the rank of the above two-dimensional parameter matrix to evaluate the degree to which the output of this layer retains the original input data; a higher retention degree indicates higher privacy, and vice versa indicates poorer data privacy. Definition 4 specifically describes the privacy measure of the intermediate output of each DNN layer.

[0102] Given that DNN models usually have a directed acyclic graph structure, when these models are split and deployed, the transmitted data usually consists of the outputs of multiple layers. Therefore, using Definition 4, introduce the privacy measure of the DNN splitting scheme in Definition 5. The privacy measure of the splitting scheme can be regarded as a weighted sum of the privacy measures of the intermediate outputs across multiple DNN layers. In addition, propose a privacy coefficient in Definition 6 to normalize the privacy measure of the DNN splitting in Definition 5.

[0103] The following will describe Theorem 1 in detail:

[0104] Theorem 1: For a DNN layer L with input X and output Y, flatten X and Y into one-dimensional vectors x and y, then L can be represented as y = Ax, where A is a two-dimensional matrix.

[0105] Proof: The process of proving Theorem 1 on different types of DNN layers is as follows:

[0106] For a convolutional layer with input dimension (n, n, c in ), stride s, padding p, and convolutional kernel with dimension (k, k, c in ), the calculation of the output elements is as follows:

[0107]

[0108] (2) For a max pooling layer with input dimension (n, n, c), kernel dimension (k, k), stride s, and padding p, each output element is the maximum value among the corresponding k 2 elements in the input tensor; let:

[0109]

[0110] Then the calculation of the output elements is as follows:

[0111]

[0112] Among them, is an indicator function that determines whether (a, b) is equal to (a m , b m )

[0113] (3) For an average pooling layer with an input dimension of (n, n, c), a kernel dimension of (k, k), a stride of s, and a padding of p, the calculation of the output element is as follows:

[0114]

[0115] (4) For a fully connected layer with n input neurons and m output neurons, the output element can be calculated as follows:

[0116]

[0117] Among them, ω {i,j} represents the weight between the i-th input neuron and the j-th output neuron;

[0118] (5) For a dropout layer with an input size of (d1, d2, …, d e ), where each input element may be discarded with a probability of p, let K be the index set of the retained input elements, with a size of approximately pd1…d e , then the output element can be calculated as follows:

[0119]

[0120] Among them, I K (i1, …, i e ) is an indicator function that determines whether (i1, …, i e ) belongs to K

[0121] (6) For an activation layer using the ReLU function, if the input size is (d1, d2, …, d e ), the output element can be calculated as follows:

[0122]

[0123] Among them, is an indicator function that determines whether is a positive real number;

[0124] Each element of Y can be combined with some elements of X to form equations, and these equations together form a system of linear equations. Therefore, after flattening X and Y into x and y, each DNN layer can be represented as y = Ax, where A is the coefficient matrix of this system of linear equations.

[0125] The following will provide a detailed description of Definitions 4, 5, and 6 respectively:

[0126] Definition 4: Assume that for L i (1 ≤ i ≤ fin), the input and output are X i , Y i respectively. After flattening them into one-dimensional vectors x i , y i , L i can be expressed as y i = A i x i . Define the privacy measure priv(·) of the output of L i as follows:

[0127]

[0128] Assume that the input of L i is composed of the outputs of γ layers . Then:

[0129]

[0130] priv(L i ) = h(ρ(L i ))), where h(·): (0, 1] → (0, 1] is a smooth function designed to prevent ρ(L i ) from decreasing too rapidly as i increases.

[0131] Definition 5: If the output of L i forms the input of L j , then i < j and there is an edge from L i to L j . These edges form the set E. Quantify the privacy of the splitting scheme P D through the privacy measure of the transmitted data as:

[0132]

[0133] where (L u , L v ) ∈ E, L u ∈ D end , L v ∈ D edge , is the privacy coefficient of the edge (L u , L v ).

[0134] Definition 6: If the output of L i forms the input of L jFor the input, i < j and there exists an edge from L i to L j , and these edges form the set E. Define the privacy coefficient of the edge (L u , L v ) as follows:

[0135] (1)

[0136] (2) If the input of L i consists of the output of , and the output of L i forms the input of θ layers including L j , then:

[0137]

[0138] DNN Collaborative Inference Modeling Scheme

[0139] The present invention designs a novel modeling implementation scheme for DNN edge-cloud collaborative inference, specifically by combining the privacy measurement of intermediate transmission data with the inference latency of each DNN layer. For each splitting scheme, the modeling implementation scheme provided by the present invention allows directly calculating the trade-off between privacy measurement and inference latency according to the list elements or the cuts of the directed acyclic graph, that is, performing optimization objective processing.

[0140] In addition, the modeling implementation scheme provided by the present invention enables SecoInfer to operate as a component of existing DNN layer splitting strategies that only focus on inference latency (such as Neurosurgeon, DADS, etc.), enabling existing corresponding components to make splitting decisions that optimize both inference latency and data privacy.

[0141] For the DNN model D under the edge-cloud splitting scheme P D , the inference latency lat(D, P D ) and the privacy measurement priv(D, P D ) have different value ranges, so they need to be normalized. Calculating the exact range of lat(D, P D ) involves traversing all splits of D, which is equivalent to traversing all cuts in the directed acyclic graph, and this process will generate a large amount of additional time overhead. Therefore, the upper bound of lat(D, P D ) can be established using Theorem 2 below, and this upper bound can be calculated quickly, thus allowing approximate normalization of lat(D, P D ) to save computing resources. Regarding the normalization of priv(D, P D ), the value range of priv(D, P D ) can be determined to be in (0, 1] through Theorem 3 and Theorem 4.

[0142] Theorem 2, Theorem 3, and Theorem 4 will be described below, where:

[0143] Theorem 2: For any splitting scheme P of model D D , the total inference latency is less than:

[0144]

[0145] where fin is the total number of layers in the deep neural network, n p is the maximum number of branches, and lat end (L i ) is the edge-side inference latency of L i , and lat trans (L i ) is the transmission latency of the output of L i .

[0146] The proof process for Theorem 2 includes: Considering that the computing power of the edge node is significantly greater than that of the edge device, the computing latency of running D under each splitting scheme must be less than the following formula:

[0147]

[0148] Since at most the outputs of n p layers need to be transmitted from the edge device to the edge node, the transmission latency must be less than the following formula:

[0149] n p max 0≤i≤fin lat trans (L i )

[0150] Therefore, under any splitting scheme, the inference latency of model D is less than the following formula:

[0151]

[0152] Theorem 3: If the output of L i forms the input of L j , then i < j and there are edges from L i to L j , and these edges form the set E. For any splitting scheme P of model D D = {D end , D edge}, there is:

[0153]

[0154] where (L u , L v ) ∈ E, Lu ∈D end ,L v ∈D edge 。

[0155] The proof process of Theorem 3 includes: Assume that the input of L i consists of the output of , and the output of L i constitutes the input of . If L i is deployed on the edge device, is deployed on the edge node, will be used to calculate the privacy measure. If L i is deployed on the edge node, is deployed on the edge device, will be used to calculate the privacy measure.

[0156] According to Definition 6, Therefore, no matter where L i is deployed, the total privacy coefficient remains unchanged. Thus,

[0157] Theorem 4: For any split P D of model D, the range of the privacy measure is (0, 1].

[0158] The proof process of Theorem 4 includes: According to Definition 4, for any layer L i (1 ≤ i ≤ fin) of model D, the privacy measure satisfies 0 < priv(L i ) ≤ 1. Combining Definition 5, for any split P D , Finally, based on Theorem 3,

[0159] Modeling of the chain-structured DNN

[0160] If model D is a chain-structured DNN as shown in Figure 5 , then each split scheme of D will correspond to a certain layer At this time, is executed on the edge device, while is executed on the edge node. Referring to Figure 5 shown, using the above Theorem 2 and Theorem 4, the trade-off between the normalized inference latency and the normalized privacy measure of running model D under split P D can be directly calculated:

[0161]

[0162] where, m Dis the upper bound of the total delay defined in Theorem 2, lat end (L) is the end-side inference delay of L, lat trans (L) is the transmission delay of the output of L, lat edge (L) is the edge-side inference delay of L, and priv(L) is the privacy measure of the output of L.

[0163] Modeling of Directed Acyclic Graph Structure DNN

[0164] If the model D has a directed acyclic graph structure, then as Figure 6 shown, create a directed acyclic graph G to represent its end-edge collaborative inference as follows:

[0165] (1) Each layer of D corresponds to a vertex in G.

[0166] (2) For each pair of layers L i , L j (1 ≤ i < j ≤ fin), if the output of L i forms the input of L j , then there is an edge from L i to L j . The weight of the edge (L i , L j ) is defined as

[0167] (3) The special vertices v end and v edge represent the end device and the edge node respectively. For each DNN layer L i , the weight of the edge (v end , L i ) is defined as is defined as

[0168] (4) Since L0 represents that the input data is directly uploaded to the edge node, L0 must be located at the end device. Therefore, set the weight of the edge (v end , L0) to +∞, the weight of (v edge , L0) to 0, and the weight of (L0, L1) to

[0169] (5) If the output of L i forms the input of multiple layers , introduce a shaded vertex L′ i to prevent redundant calculation of the transmission delay. The weight of (L i , L′ i ) is defined as The weight of

[0170] Thus, each splitting scheme of model D should pair a set of two DNN layer sets {V_end, V_edge}, where the layers in V_end run on the end device, and the layers in V_edge execute on the edge node. By applying Theorem 2 and Theorem 4 above, the trade-off between the normalized inference latency and the privacy metric can be directly calculated as follows:

[0171]

[0172] where m D is the upper bound of the total latency defined in Theorem 2, lat end (L) is the end-side inference latency of L, lat trans (L) is the transmission latency of the output of L, lat edge (L) is the edge-side inference latency of L, and priv(L) is the privacy metric for the output of L. is the privacy coefficient defined in Definition 6.

[0173] In addition, in the directed acyclic graph G, each splitting corresponds to a v end -v edge cut, where the layers in V end / V edge and the vertex v end / v edge together form two subgraphs. In this case, Norm(lat(D, P D )) + αNorm(priv(D, P D )) is equal to the sum of the edge weights in the corresponding v end -v edge cut.

[0174] Fine-tuning of candidate end-edge splitting schemes

[0175] The sub-model cutting module in the embodiments of the present invention can efficiently cut the sub-model D sub . When D is deployed on the end device sub and the remaining layers of D are deployed on the edge node, the overall inference latency may slightly exceed the threshold th lat . Therefore, selecting the best end-edge splitting scheme on D sub may not fully meet the latency constraint. Therefore, the present invention also introduces a fine-tuning method based on the depth of the DNN layers to ensure that the candidate splitting scheme complies with the latency constraint conditions of the system while maintaining a sufficient depth of the end-side model.

[0176] Specifically, for each layer of model D, its depth is defined in Definition 7 below. For the candidate end-edge splitting scheme P D = D end , D edge, if the inference latency of D when running under P D exceeds the threshold th lat , then relocate the layer with the highest depth in D end until the inference latency meets the latency constraint. The implementation of the present invention can ensure that the original input passes through a sufficient number of DNN layers on the edge device on average. When the transmitted intermediate data is obtained through more calculations, it is often more challenging for an adversary to reconstruct the original input. edge

[0177] Definition 7: It defines the depth of each layer in the DNN model D as follows:

[0178] (1) L1.depth = 1

[0179] (2) If the input of L i consists of the output of

[0180]

[0181] The implementation of the embodiments of the present invention provides a collaborative splitting strategy for jointly optimizing latency and privacy for the first time, which is beneficial to the development of secure edge-cloud collaboration technologies. At the same time, the present invention also provides a measurement mechanism for the difficulty of inverting the original input from the intermediate data during the DNN inference process, which can guide the evaluation of data privacy in the edge-cloud DNN inference process and has practical application prospects. Moreover, the present invention can also be used as a component of existing edge-cloud collaborative inference strategies to promote the joint optimization of privacy and latency, and has a faster execution efficiency, which is conducive to the wide deployment and application of the solution.

[0182] In the specific implementation process of the present invention, the pseudo-code of the DNN block construction algorithm and the pseudo-code of the entire SecoInfer framework can be as Figure 8 and Figure 9 shown.

[0183] As Figure 7 shown, the embodiments of the present invention are actually deployed, that is, the SecoInfer framework is implemented on real hardware. In this framework, the Raspberry Pi 4B platform is used as the edge device, equipped with a 4-core ARM Cortex-A72 @ 1.5 GHz processor and 4G LPDDR4-3200 SDRAM, and the edge node uses a server equipped with a 6-core Intel Core i5-11500 @ 2.70 GHz processor and an NVIDIA GeForce RTX 3060 GPU. The edge device and the edge node communicate with each other through a router with a 100 Mbps bandwidth.

[0184] ​​The Raspberry Pi 4B platform retains the original input image and transfers the intermediate data to the server after the end-side model has completed the calculation. As a malicious edge node, after receiving the transmitted intermediate data, the server performs the calculation of the edge-side model and at the same time attempts to reconstruct the original image using the received data. However, due to the optimized privacy processing scheme adopted in the end-edge collaborative inference system provided by the present invention (i.e., the intermediate data is obtained through more calculations), the difficulty for the malicious edge node to reconstruct the original image based on the intermediate data is greatly increased, thereby effectively protecting the privacy of the data.

[0185] As described above, the above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims. The information disclosed in the background art part of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or any form of suggestion that this information constitutes the prior art known to those skilled in the art.

Claims

1. A system for optimizing edge-side collaborative inference of deep neural networks, characterized in that, It includes a sub-model cutting module and an edge-end splitting decision module. The sub-model cutting module is used to search and traverse the layers of a deep neural network (DNN) model, and split the block-level sub-model according to a predetermined inference latency requirement. After splitting, a part of the layers are deployed to the edge device, and the remaining layers are deployed to the edge node. The edge-end splitting decision module includes a modeling component, a selection component, and a fine-tuning component, where: The sub-model cutting module collaborates with the fine-tuning component to adjust the splitting and deployment decision scheme. The fine-tuning component adopts a fine-tuning method based on the depth of the DNN layer to ensure that the candidate splitting scheme maintains a sufficient depth of the edge-side model while complying with the predetermined inference latency requirement. The modeling component measures the privacy of the intermediate output of each layer, obtains the privacy measurement parameter, and combines the privacy measurement parameter with the predetermined inference latency requirement to implement the modeling of DNN edge-end collaborative inference. The selection component is used to call the existing hierarchical edge-end collaborative inference strategy to determine the edge-end splitting scheme for optimizing the privacy and the inference latency.

2. The system according to claim 1, characterized in that The sub-model cutting module further includes a pair of latency predictors trained for each type of DNN layer using a linear regression model. The predictors are used to estimate the running latency of each layer in the model on the edge device and the edge node.

3. The system according to claim 2, wherein The input of the latency predictor includes: In the convolutional layer, max pooling layer, and average pooling layer, the number of channels, the number of filters, the scale of the input feature map, the scale of the kernel, the stride, and the padding parameter are used as the input of the latency predictor. In the fully connected layer, the number of input neurons and the number of output neurons are used as the input of the latency predictor. In the dropout layer and the activation layer with the ReLU function, the number of input elements is used as the input of the latency predictor.

4. The system according to claim 1, wherein The sub-model cutting module is further configured to perform a process of searching for a block-level sub-model D from the DNN model D sub The process includes: Divide model D into a series of DNN blocks: B1, B2, …, B d , assuming that all DNN blocks are initially deployed on the edge node, and the initial inference latency is lat trans (L0)+∑ 1≤i≤fin lat edge (L i ). Subsequently, gradually transfer the DNN blocks from the edge node to the end device, iteratively update the inference latency until the inference latency just exceeds the threshold th lat , at which time the DNN blocks on the end device form D sub , as the block-level submodel D from the DNN model D sub .

5. The system according to claim 4, characterized in that After the sub-model cutting module cuts and determines the block-level sub-model D sub and deploys D on the end device sub and deploys the remaining layers of D on the edge node, if the inference latency exceeds the threshold th lat , then it is necessary to implement fine-tuning processing based on the depth of the DNN layer through the fine-tuning component, and the fine-tuning processing includes: For each layer of model D, define its depth and determine the candidate end-edge splitting scheme as P D = D end , D edge , if the inference latency of running D under P D exceeds the threshold th lat , then relocate the layer with the highest depth in D end to D edge until the inference latency meets the aforementioned predetermined inference latency requirement.

6. The system according to claim 5, wherein The predetermined inference latency requirement is achieved through the introduced inference latency constraint, and the inference latency constraint is defined as follows: s.t.lat(D,P D )≤th lat Among them, Norm(lat(D,P D )) ∈ (0,1] represents the normalized inference latency, and Norm(priv(D,P D )) ∈ (0,1] represents the normalized privacy measure for running model D under the splitting scheme P D . α represents the user-defined trade-off factor, s.t. lat(D,P D ) is the latency under the splitting scheme P D , and th lat is the latency threshold.

7. The system according to any one of claims 1 to 6, characterized in that The process by which the modeling component obtains the privacy measurement parameter includes: Express the calculation of each DNN layer in model D as the product of a one-dimensional input vector and a two-dimensional parameter matrix. The privacy measurement of the intermediate data in the DNN inference process is introduced. The privacy measurement is used to quantify the challenge faced by the adversary in reconstructing the original input from this data. Specifically, the rank of the two-dimensional parameter matrix is used to evaluate the degree to which the output of this layer retains the original input data as the privacy measurement parameter. Among them, a high retention degree indicates high privacy, and vice versa indicates poor data privacy.

8. The system according to claim 7, wherein The privacy measurement parameter includes: The privacy measurement parameter is the privacy measurement of the DNN splitting scheme. The privacy measurement of this splitting scheme is regarded as the weighted sum of the privacy measurements of the intermediate outputs across multiple DNN layers, and the privacy measurements of the intermediate outputs across multiple DNN layers split by the DNN splitting scheme are normalized by the privacy coefficient.

9. The system according to claim 8, characterized in that, The DNN splitting scheme P D The calculation formula for privacy measurement It includes: Among them, (L u , L v ) ∈ E, L u ∈ D end , L v ∈ D edge , is the privacy coefficient of the edge (L u , L v ); The calculation method of the privacy coefficient λ includes: where L i 's input consists of 's output, and L i 's output forms the input of θ layers including L j , where i < j.

10. The system according to claim 7, wherein Based on the edge - side splitting scheme P D For the DNN model D under it, the system further includes: normalizing the inference latency lat(D, P D ) and the privacy measure priv(D, P D ) which have different value ranges, wherein the normalization of lat(D, P D ) is achieved based on the upper bound of lat(D, P D ), and the value range of the normalization of the said priv(D, P D ) is in (0, 1]; Based on this normalization method, the modeling process of the modeling component includes: If the model D is a chained DNN, each splitting scheme of D will correspond to a certain layer and is executed on the end device, while is executed on the edge node, then the trade-off between the normalized inference latency and the normalized privacy measure for running the model D under the splitting P D is: where m D is the upper bound of the total delay, lat end (L) is the end-side inference delay of L, lat trans (L) is the transmission delay of the output of L, lat edge (L) is the edge-side inference delay of L, and priv(L) is the privacy measure of the output of L; If the model D is a directed acyclic graph structure, each splitting scheme of the model D corresponds to a corresponding set of DNN layers {V_{end}, V_{edge}}, where the layers in V_{end} run on the end device and the layers in V_{end} are executed on the edge node. Then the trade-off between the normalized inference latency and the privacy metric is as follows: In a directed acyclic graph G, each split corresponds to a v end -v edge cut, where the layers in V end / V edge and the vertex v end / v edge together form two subgraphs, and Norm(lat(D,P D )) + αNorm(priv(D,P D )) is equal to the sum of the edge weights in the corresponding v end -v edge cut, where α is the privacy coefficient.

Images