Model deployment method, device, electronic device and computer readable medium
By generating an initial computation graph and performing graph optimization and obfuscation, and combining node configuration information to generate deployment strategies, the problem of efficient deployment of machine learning models on servers and edge devices is solved, achieving efficient utilization of computing resources and secure protection of model data.
Patent Information
- Application Number
- CN202410815814.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-24
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-06-24
AI Technical Summary
In existing technologies, when machine learning models are deployed on the server side, the server's operating pressure increases as the model complexity and number of parameters increase, making it impossible to effectively utilize edge computing resources. Furthermore, the model data protection is insufficient, and the risk of leakage is high.
By generating an initial computation graph, performing graph optimization and computation graph obfuscation, and combining node configuration information to generate deployment strategies, dynamically matching deployment methods, optimizing model structure and protecting model data, efficient deployment on servers and edge devices is achieved.
It achieves efficient utilization of computing resources, reduces server operating pressure, protects model data, and ensures the normal operation and secure deployment of the model on different nodes.
Smart Images

Figure CN118394717B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a model deployment method, device, electronic device, and computer-readable medium. Background Art
[0002] Model deployment is a technology that deploys trained machine learning models to execution nodes. Currently, when deploying models, the commonly used method is to encapsulate the machine learning model and deploy it on the server to achieve the purpose of model deployment.
[0003] However, when the above method is adopted, the following technical problems often occur:
[0004] First, as the model complexity and the number of model parameters increase, using only the server as the carrier for model deployment will increase the operating pressure on the server as the number of visits increases, and at the same time fail to effectively utilize the computing resources on the edge.
[0005] Second, since the computing resources at the edge are limited, if the full model is deployed at the edge, it will exceed the data processing capabilities of the edge, thus affecting the normal operation of the model.
[0006] Third, the direct encapsulation method lacks effective model data protection, resulting in a high risk of model data leakage.
[0007] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention
[0008] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.
[0009] Some embodiments of the present disclosure propose a model deployment method, an apparatus, an electronic device, and a computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.
[0010] In a first aspect, some embodiments of the present disclosure provide a model deployment method, the method comprising: obtaining a target model file, wherein the target model file is a model description file corresponding to a machine learning model to be deployed; generating an initial computation graph according to the target model file, wherein the initial computation graph is a directed graph, and the initial computation graph comprises: an operator node set and a relationship edge set, wherein the relationship edge is used to describe the dependency relationship and tensor state between the operator nodes; determining the model deployment type according to a node configuration information set of the node to be deployed; performing graph optimization on the initial computation graph according to the model deployment type and the operator description information corresponding to the operator node to generate an optimized computation graph; performing computation graph obfuscation on the optimized computation graph to generate an obfuscated computation graph; generating a model file to be deployed according to the obfuscated computation graph; generating deployment strategy information according to the node configuration information set and the model deployment type; and deploying the model file to be deployed on the node to be deployed according to the deployment strategy information.
[0011] In a second aspect, some embodiments of the present disclosure provide a model deployment device, the device comprising: an acquisition unit, configured to acquire a target model file, wherein the target model file is a model description file corresponding to a machine learning model to be deployed; a first generation unit, configured to generate an initial computation graph according to the target model file, wherein the initial computation graph is a directed graph, and the initial computation graph comprises: an operator node set and a relationship edge set, wherein the relationship edge is used to describe the dependency relationship and tensor state between the operator nodes; a determination unit, configured to determine the model deployment type according to a node configuration information set of the node to be deployed; The graph optimization unit is configured to perform graph optimization on the above-mentioned initial computation graph according to the above-mentioned model deployment type and the operator description information corresponding to the operator node, so as to generate an optimized computation graph; the computation graph obfuscation unit is configured to perform computation graph obfuscation on the above-mentioned optimized computation graph, so as to generate an obfuscated computation graph; the second generation unit is configured to generate a model file to be deployed according to the above-mentioned obfuscated computation graph; the third generation unit is configured to generate deployment strategy information according to the above-mentioned node configuration information set and the above-mentioned model deployment type; the deployment unit is configured to deploy the above-mentioned model file to be deployed on the above-mentioned node to be deployed according to the above-mentioned deployment strategy information.
[0012] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.
[0013] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.
[0014] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the model deployment method of some embodiments of the present disclosure, efficient utilization of computing resources is achieved. In practice, as the complexity of the model increases and the number of model parameters increases, the method of using only the server as the carrier of model deployment will increase the operating pressure of the server side as the number of visits increases, and the computing resources of the edge side cannot be effectively utilized. Based on this, the model deployment method of some embodiments of the present disclosure, first, obtains the target model file, wherein the target model file is the model description file corresponding to the machine learning model to be deployed. In this way, a model description for the machine learning model to be deployed is obtained. Secondly, according to the target model file, an initial calculation graph is generated, wherein the initial calculation graph is a directed graph, and the initial calculation graph includes: an operator node set and a relationship edge set, and the relationship edge is used to describe the dependency relationship and tensor state between the operator nodes. By generating a calculation graph, the dependency relationship between the operator nodes can be dynamically described in the form of a graph. Then, according to the node configuration information set of the node to be deployed, the model deployment type is determined. By combining the node configuration of the node to be deployed, a deployment mode that matches the node configuration of the node to be deployed is dynamically determined. Further, according to the operator description information corresponding to the above-mentioned model deployment type and the operator node, the above-mentioned initial calculation graph is optimized to generate an optimized calculation graph. The model structure is simplified through graph optimization to improve the deployment capability of the model on nodes with low computing resources. In addition, the above-mentioned optimized calculation graph is obfuscated to generate an obfuscated calculation graph. The leakage of the model structure is avoided through graph obfuscation. In addition, according to the above-mentioned obfuscated calculation graph, a model file to be deployed is generated. Then, according to the above-mentioned node configuration information set and the above-mentioned model deployment type, deployment strategy information is generated. In this way, a deployment strategy matching the node configuration is generated to ensure the successful deployment of the model and the efficient use of the node's computing resources. Finally, according to the above-mentioned deployment strategy information, the above-mentioned model file to be deployed is deployed on the above-mentioned node to be deployed. In this way, the model deployment method of the model on different nodes is improved, and the computing resources of different nodes are efficiently utilized. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.
[0016] Figure 1is a flow chart of some embodiments of the model deployment method according to the present disclosure;
[0017] Figure 2 is a schematic diagram of the structure of some embodiments of the model deployment device according to the present disclosure;
[0018] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION
[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0020] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.
[0021] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0024] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.
[0025] refer to Figure 1 , shows a process 100 of some embodiments of the model deployment method according to the present disclosure. The model deployment method comprises the following steps:
[0026] Step 101, obtaining a target model file.
[0027] In some embodiments, the execution subject of the model deployment method (for example, a computing device) can obtain the target model file through a wired connection or a wireless connection. Among them, the above-mentioned target model file can be a model description file corresponding to a machine learning model that has been trained and is to be deployed. For example, taking the image recognition scenario as an example, the machine learning model to be deployed can be a YOLO (You Only Look Once) model. Therefore, the target model file can be a model description file of the YOLO model. Specifically, the target model file may include: a model parameter description file and a model structure description file. The model parameter description file can characterize the model parameters of the machine learning model to be deployed after the model training is completed. The model structure description file can characterize the model structure of the machine learning model to be deployed after the model training is completed.
[0028] It should be noted that the above-mentioned computing device can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the computing device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.
[0029] Step 102: Generate an initial calculation graph according to the target model file.
[0030] In some embodiments, the above-mentioned execution subject can generate an initial calculation graph based on the target model file. Among them, the initial calculation graph is a graph used to describe the operators involved in the machine learning model to be deployed and the dependencies between operators. The initial calculation graph is a directed graph. The initial calculation graph includes: an operator node set and a relationship edge set. The relationship edge is used to describe the dependency relationship and tensor state between operator nodes. Specifically, the relationship edge can represent the dependency order of two operator nodes with a dependency relationship, and the calculation state (tensor state) of the tensor involved in the two operator nodes with a dependency relationship. In practice, first, the above-mentioned execution subject can load the target model file, and then generate the initial calculation graph by calling the calculation graph generation function according to the machine learning framework corresponding to the machine learning model to be deployed. For example, taking the TensorFlow machine learning framework as an example, the calculation graph generation function can be a make-dot() function.
[0031] In some optional implementations of some embodiments, the execution subject generates an initial calculation graph according to the target model file, which may include the following steps:
[0032] The first step is to parse the above model definition file to generate an operator description information set.
[0033] Among them, the target model file includes: a model definition file. Among them, the model definition file can be a file for defining the model structure of the machine learning model to be deployed. The operator description information in the above-mentioned operator description information set includes: an operator identifier, an operator type, an operator input description information, an operator output description information, a pre-associated operator identifier, and a post-associated operator identifier. The operator identifier can be a unique identifier of the operator. The operator type can include but is not limited to: an arithmetic operator, a matrix operator, an activation operator, a normalization operator, a convolution operator, a recurrent neural network operator, a loss function operator, a logical operator, a conditional operator, a tensor shape control operator, an embedding operator, a random operator, and an optimization operator. The operator input description information represents the tensor of the input operator. The operator output description information represents the tensor after being processed by the operator. The pre-associated operator identifier represents the node identifier of the operator node pre-associated with the operator node. The post-associated operator identifier represents the node identifier of the operator node post-associated with the operator node.
[0034] The second step is to filter out the operator description information that meets the filtering conditions from the above operator description information set as the target operator description information.
[0035] The above screening condition is: the previous associated operator identifier included in the operator description information is an empty identifier.
[0036] The third step is to generate an operator node of the root node type according to the operator identifier and operator type included in the target operator description information.
[0037] As an example, the operator type may be a Conv2D roll operator type. The operator identifier may be "S1". Then the obtained operator node may be "Operator identifier: S1; Operator type: Conv2D roll operator type".
[0038] The fourth step is to use the operator node of the above root node type as the starting node, and determine the node dependency according to the operator description information including the previous operator identifier, operator type, previous associated operator identifier and subsequent associated operator identifier to generate a candidate computation graph.
[0039] Among them, the above candidate computation graph is a computation graph of the tensor state corresponding to the unupdated relationship edge.
[0040] In practice, the above-mentioned execution entity can take out the operator description information from the operator description information set in turn, and generate operator nodes and connect the operator nodes corresponding to the previous associated operator identifier according to the operator identifier, operator type, previous associated operator identifier and next associated operator identifier included in the taken out operator description information to obtain the above-mentioned candidate calculation graph.
[0041] The fifth step is to update the state of the tensor corresponding to the relationship edge in the above candidate calculation graph according to the operator input description information and the operator output description information included in the operator description information in the above operator description information set to generate the above initial calculation graph.
[0042] In practice, when the initial computation graph is initially generated, the state of the tensors corresponding to the relationship edges defaults to the tensor being unused.
[0043] Step 103: Determine the model deployment type according to the node configuration information set of the node to be deployed.
[0044] In some embodiments, the above-mentioned execution subject can determine the model deployment type according to the node configuration information set of the node to be deployed. Among them, the node to be deployed can be a node of the machine learning model to be deployed. In practice, the node to be deployed can be a virtual node. The node to be deployed can also be a hardware node. Specifically, when the node to be deployed is a hardware node, the node to be deployed can be composed of a server side and / or an edge side. When the node to be deployed is a virtual node, it can be a virtual node composed of virtual computing resources deployed on the server side and / or the edge side. Specifically, the server side can be composed of at least one server, which can be implemented as a distributed architecture. The edge side can be a computing resource other than the server side. For example, the edge side can be an edge server or a camera. The node configuration information represents the hardware configuration of the machine learning model to be deployed for different computing resources of the node to be deployed. For example, the node configuration information may include: node configuration information A and node configuration information B. Among them, the node configuration information A can represent the hardware configuration information and node demand of the unit node required when the node to be deployed adopts the computing resources of the server side. Node configuration information B can characterize the hardware configuration information and node demand of the unit node required when the node to be deployed adopts the computing resources of the server. The model deployment type characterizes the deployment method of the machine learning model. In practice, the model deployment types include: a first model deployment type, a second model deployment type, and a third model deployment type. The first model deployment type characterizes the model deployment using only the computing resources of the server. The second model deployment type characterizes the model deployment using the computing resources of the server and the computing resources of the edge. The third model deployment type characterizes the model deployment using only the computing resources of the edge. In practice, the above-mentioned execution entity can determine the model deployment type based on the model matching degree between the node configuration information and the machine learning model to be deployed.
[0045] Optionally, the node configuration information in the node configuration information set includes: node resource requirement information and node type. Node types include: edge virtual node type and non-edge virtual node type. The node resource requirement represents the amount of hardware resources corresponding to the unit node for supporting the operation of the machine learning model. The edge virtual node type corresponds to the edge computing node. The non-edge virtual node corresponds to the server computing node.
[0046] In some optional implementations of some embodiments, the execution subject determines the model deployment type according to the node configuration information set of the node to be deployed, which may include the following steps:
[0047] In the first step, the node configuration information including the node type of the edge virtual node type is screened out from the above node configuration information set as the first node configuration information.
[0048] In the second step, the node configuration information including the node type of a non-edge virtual node type is screened out from the above node configuration information set as the second node configuration information.
[0049] The third step is to determine the first hardware resource requirement according to the node resource requirement information included in the first node configuration information.
[0050] In practice, the execution entity may use the product of the hardware configuration information of the unit node included in the node resource requirement information and the node requirement as the first hardware resource requirement.
[0051] The fourth step is to determine the second hardware resource requirement according to the node resource requirement information included in the second node configuration information.
[0052] In practice, the execution entity may use the product of the hardware configuration information of the unit node included in the node resource requirement information and the node requirement as the second hardware resource requirement.
[0053] The fifth step is to determine the demand ratio of the first hardware resource demand to the second hardware resource demand.
[0054] Step 6: In response to the above demand ratio being greater than 2, the first model deployment type is determined as the above model deployment type.
[0055] In the seventh step, in response to the demand ratio being greater than 1 and the demand ratio being less than or equal to 2, the second model deployment type is determined as the model deployment type.
[0056] In the eighth step, in response to the demand ratio being greater than 0 and less than or equal to 1, the third model deployment type is determined as the model deployment type.
[0057] Step 104: Optimize the initial computation graph according to the model deployment type and the operator description information corresponding to the operator node to generate an optimized computation graph.
[0058] In some embodiments, the execution subject may optimize the initial computation graph according to the model deployment type and the operator description information corresponding to the operator node to generate an optimized computation graph. In practice, the execution subject may compress the initial computation graph according to the model deployment type and the operator description information corresponding to the operator node to generate an optimized computation graph.
[0059] In some optional implementations of some embodiments, the execution subject optimizes the initial computation graph according to the model deployment type and the operator description information corresponding to the operator node to generate an optimized computation graph, which may include the following steps:
[0060] In response to the model deployment type being the second model deployment type or the third model deployment type, the following graph optimization steps are performed:
[0061] The first step is to determine the operator node corresponding to the operator description information whose corresponding operator type is the target operator type in the above initial calculation graph as the target operator node, and obtain the target operator node set.
[0062] The target operator type may be a Crop operator type, a Reshape operator type, a Transpose operator type, a BinaryOp operator type, a Slice operator type, or a StrideSlice operator type. In practice, the execution subject may determine the operator node corresponding to the operator description information of the target operator type in the initial calculation graph by traversing and searching, as the target operator node.
[0063] In the second step, for each target operator node in the above target operator node set, perform the following operator rearrangement steps:
[0064] The first sub-step is to take the target operator node as the starting node and perform a reverse graph traversal on the initial calculation graph to determine the path to be rearranged corresponding to the target operator node.
[0065] Among them, the above-mentioned path to be rearranged is a local graph path with a path length of a preset path length and the above-mentioned target operator node as the path end node. In practice, 2≤preset path length≤5. In practice, the preset path length represents the number of operator nodes contained in the path to be rearranged. Specifically, when traversing the reverse graph, when traversing to an operator node of the target operator type and the path length ≤5, stop traversing. When traversing in reverse, when the path length is greater than 5, stop traversing.
[0066] The second sub-step is to update the target operator node to the path start end node of the path to be rearranged, so as to obtain the rearranged path.
[0067] In practice, first, the execution subject may construct a virtual queue. Then, the operator nodes included in the path to be rearranged are pushed into the virtual queue. Next, the target operator node is taken out from the tail of the virtual queue and pushed into the head of the queue to generate a rearrangement path.
[0068] The third sub-step is to update the initial computation graph according to the rearrangement path.
[0069] In practice, the execution entity may update the to-be-rearranged path in the initial computation graph to the rearranged path.
[0070] The fourth sub-step is to perform operator replacement on the updated initial computation graph to generate a computation graph after operator replacement.
[0071] In practice, the above execution subject can replace the operator in the updated initial calculation graph according to the operator that is consistent with the operator calculation logic but more conducive to model deployment, so as to generate the calculation graph after the operator replacement. For example, the BatchNorm operator node in the updated initial calculation graph is updated to the Scale operator node.
[0072] The fifth sub-step is to perform local operator fusion on the above operator replacement calculation graph to obtain the above optimized calculation graph.
[0073] In practice, different operators can be expressed by corresponding computational expressions. Therefore, operators can be fused by merging computational expressions to achieve local operator fusion. For example, local operator fusion can be performed by merging the Convolution operator node with the BatchNorm operator.
[0074] The above-mentioned content of "in some optional implementation methods of some embodiments", as an inventive point of the present disclosure, solves the second technical problem mentioned in the background technology, namely, "due to the limited computing resources at the edge, if the full model is deployed at the edge, it will exceed the data processing capacity of the edge, thereby affecting the normal operation of the model". Based on this, first, the present disclosure responds to the above-mentioned model deployment type being the above-mentioned second model deployment type or the above-mentioned third model deployment type, and executes the following graph optimization steps, that is, when the model needs to be deployed at the virtual node corresponding to the edge, the graph optimization is performed. Specifically, the first step is to determine the operator node corresponding to the operator description information of the target operator type in the above-mentioned initial calculation graph as the target operator node, and obtain a target operator node set. Specifically, the target operator node may be an operator node involving tensor clipping. In the second step, for each target operator node in the above target operator node set, the following operator rearrangement steps are performed: with the above target operator node as the starting node, the above initial calculation graph is reversely traversed to determine the path to be rearranged corresponding to the above target operator node, wherein the above path to be rearranged is a local graph path with a path length of the preset path length and the above target operator node as the path end node. By determining the path to be rearranged, the path that needs to adjust the order of the operator nodes is determined. The above target operator node is updated to the path starting end node of the above path to be rearranged to obtain the rearranged path; according to the above rearranged path, the above initial calculation graph is updated. Through local path adjustment, the operator nodes involved in tensor clipping are placed in front, so that tensor clipping is performed first, and then tensor operation is performed, thereby reducing the amount of calculation and alleviating the calculation pressure of the virtual nodes corresponding to the edge end. In the third step, the operator is replaced on the initial calculation graph after the graph update to generate the calculation graph after operator replacement. By replacing operators and ensuring the calculation accuracy, operators that are more friendly to model deployment are replaced to improve the compatibility of deployment. The fourth step is to perform local operator fusion in the calculation graph after the above operator replacement to obtain the above optimized calculation graph. In this way, the model structure and calculation amount can be further compressed through operator fusion from the perspective of calculation logic. In this way, the model can be effectively compressed to ensure that the computing power of the edge can meet the computing power requirements of the model, thereby ensuring the normal operation of the model.
[0075] Step 105: Obfuscate the optimized computation graph to generate an obfuscated computation graph.
[0076] In some embodiments, the execution subject may perform computational graph obfuscation on the optimized computational graph to generate an obfuscated computational graph. In practice, the execution subject may perform computational graph obfuscation on the optimized computational graph by replacing the operator identifiers to generate an obfuscated computational graph.
[0077] In some optional implementations of some embodiments, the execution subject performs computation graph obfuscation on the optimized computation graph to generate an obfuscated computation graph, which may include the following steps:
[0078] In the first step, for each operator node in the operator node set included in the above obfuscated computation graph, perform the following node insertion steps:
[0079] The first sub-step is to determine the node in-degree value and the node out-degree value corresponding to the above operator node.
[0080] The second sub-step is to generate the node insertion probability according to the node in-degree value and node out-degree value corresponding to the above operator node.
[0081] In practice, the value range of node insertion probability is [0,1]. The node insertion probability is positively correlated with the sum of the node in-degree value and the node out-degree value, that is, as the sum of the node in-degree value and the node out-degree value increases, the node insertion probability increases. Specifically, the above execution entity can use the hyperbolic tangent function as the probability formula to calculate the node insertion probability. Among them, the sum of the node in-degree value and the node out-degree value ≥ 0.
[0082] In the second step, according to the node insertion probability corresponding to the operator node, the operator nodes that meet the screening conditions are screened out from the operator node set included in the above obfuscated calculation graph as candidate operator nodes, and a candidate operator node set is obtained.
[0083] The above screening condition is: the node insertion probability corresponding to the operator node is the first K largest node insertion probabilities. In practice, the value range of K can be [3,5].
[0084] In the third step, for each candidate operator node in the above candidate operator node set, an invalid operator node is randomly inserted between the operator nodes connected to the above candidate operator node through a relationship edge.
[0085] Among them, an invalid operator node may be an operator node that does not contain specific tensor operation logic.
[0086] In the fourth step, the operator identifier corresponding to each operator node in the operator node set included in the obtained calculation graph after the node is inserted is obfuscated to obtain the calculation graph after the identifier is obfuscated.
[0087] In practice, the above-mentioned execution entity can use a hash algorithm to confuse the operator identifier.
[0088] The fifth step is to decompose the computational graph after the above-mentioned logo obfuscation to obtain a set of sub-computational graphs.
[0089] In practice, the execution entity may segment the computation graph after the identifier is obfuscated to obtain multiple sub-computation graphs as the sub-computation graph set.
[0090] In the sixth step, each sub-computation graph in the above sub-computation graph set is obfuscated to generate an obfuscated sub-computation graph, thereby obtaining an obfuscated sub-computation graph set.
[0091] In practice, the execution entity may use the first to third steps to confuse the sub-computation graph to generate an obfuscated sub-computation graph.
[0092] In the seventh step, the above obfuscated sub-computation graph set is combined to obtain the above obfuscated computation graph.
[0093] The above-mentioned "in some optional implementation methods of some embodiments" as an inventive point of the present disclosure solves the third technical problem mentioned in the background technology, namely, "the direct encapsulation method lacks effective model data protection, resulting in a high risk of model data leakage". Based on this, the present disclosure, first, for each operator node in the operator node set included in the above-mentioned obfuscated calculation graph, performs the following node insertion steps: determine the node in-degree value and node out-degree value corresponding to the above-mentioned operator node; generate a node insertion probability according to the node in-degree value and node out-degree value corresponding to the above-mentioned operator node. Then, according to the node insertion probability corresponding to the operator node, the operator nodes that meet the screening conditions are screened out from the operator node set included in the above-mentioned obfuscated calculation graph as candidate operator nodes to obtain a candidate operator node set. Further, for each candidate operator node in the above-mentioned candidate operator node set, an invalid operator node is randomly inserted between the operator nodes connected to the above-mentioned candidate operator node and through relationship edges. That is, the insertion position of the invalid operator node is dynamically determined by the node insertion probability. In addition, the operator identifier corresponding to each operator node in the operator node set included in the obtained node insertion calculation graph is obfuscated to obtain the calculation graph after identifier obfuscation. The readability of the model is reduced by identifier obfuscation, and the difficulty of cracking the model is increased. In addition, the calculation graph after identifier obfuscation is decomposed to obtain a sub-calculation graph set. Then, each sub-calculation graph in the above sub-calculation graph set is obfuscated to generate an obfuscated sub-calculation graph to obtain an obfuscated sub-calculation graph set. In this way, further graph obfuscation is performed from a more fine-grained graph perspective, that is, two consecutive graph obfuscations, which further increase the difficulty of cracking. Finally, the above obfuscated sub-calculation graph set is combined to obtain the above obfuscated calculation graph. In this way, the model data can be effectively protected and the risk of model data leakage can be reduced.
[0094] Step 106: Generate a model file to be deployed based on the obfuscated calculation graph.
[0095] In some embodiments, the execution subject may generate a model file to be deployed according to the obfuscated computation graph. In practice, taking the Pytarch model framework as an example, the execution subject may generate a model file to be deployed corresponding to the obfuscated computation graph through the torch.save() function.
[0096] Step 107: Generate deployment strategy information according to the node configuration information set and the model deployment type.
[0097] In some embodiments, the execution subject may generate deployment strategy information according to the node configuration information set and the model deployment type. The deployment strategy information represents the deployment method of the model file to be deployed. In practice, the execution subject may generate deployment strategy information according to the node configuration information set and the model deployment type by using template matching.
[0098] In some optional implementations of some embodiments, the execution subject generates deployment strategy information according to the node configuration information set and the model deployment type, which may include the following steps:
[0099] The first step is to generate, for each node configuration information in the node configuration information set, a node configuration feature corresponding to the node configuration information.
[0100] In practice, the above-mentioned execution entity can encode the node configuration information through a word embedding model, such as a Bert model, to obtain a node configuration feature corresponding to the above-mentioned node configuration information.
[0101] The second step is to generate deployment strategy constraint information based on the above model deployment type.
[0102] In practice, the above-mentioned execution entity may select pre-configured constraint information that matches the above-mentioned model deployment type as the deployment strategy constraint information.
[0103] The third step is to generate the above deployment strategy information based on the obtained node configuration feature set and the above deployment strategy constraint information.
[0104] In practice, the execution subject may use the generative adversarial network model as a strategy generation model, and use the node configuration feature set and the deployment strategy constraint information as input to obtain deployment strategy information. For example, the deployment strategy information may include: deployment mode, deployment configuration and other information.
[0105] Step 108: deploy the model file to be deployed on the node to be deployed according to the deployment strategy information.
[0106] In some embodiments, the execution subject may deploy the model file to be deployed on the node to be deployed according to the deployment strategy information. In practice, the execution subject may deploy the model file to be deployed on the node to be deployed by OTA distribution.
[0107] In some optional implementations of some embodiments, the execution subject deploys the model file to be deployed on the node to be deployed according to the deployment strategy information, which may include the following steps:
[0108] According to the deployment strategy information, the model files to be deployed are dynamically distributed to deploy the model files to be deployed on the nodes to be deployed.
[0109] In practice, especially when the nodes to be deployed include: virtual nodes corresponding to the server side and virtual nodes corresponding to the edge side, there are differences in deployment timing and method. Therefore, a dynamic distribution method can be adopted, such as distributing the model files to be deployed to the nodes to be deployed in combination with the network status, so as to reduce the problem of high packet loss rate caused by network congestion.
[0110] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the model deployment method of some embodiments of the present disclosure, efficient utilization of computing resources is achieved. In practice, as the complexity of the model increases and the number of model parameters increases, the method of using only the server as the carrier of model deployment will increase the operating pressure of the server side as the number of visits increases, and the computing resources of the edge side cannot be effectively utilized. Based on this, the model deployment method of some embodiments of the present disclosure, first, obtains the target model file, wherein the target model file is the model description file corresponding to the machine learning model to be deployed. In this way, a model description for the machine learning model to be deployed is obtained. Secondly, according to the target model file, an initial calculation graph is generated, wherein the initial calculation graph is a directed graph, and the initial calculation graph includes: an operator node set and a relationship edge set, and the relationship edge is used to describe the dependency relationship and tensor state between the operator nodes. By generating a calculation graph, the dependency relationship between the operator nodes can be dynamically described in the form of a graph. Then, according to the node configuration information set of the node to be deployed, the model deployment type is determined. By combining the node configuration of the node to be deployed, a deployment mode that matches the node configuration of the node to be deployed is dynamically determined. Further, according to the operator description information corresponding to the above-mentioned model deployment type and the operator node, the above-mentioned initial calculation graph is optimized to generate an optimized calculation graph. The model structure is simplified through graph optimization to improve the deployment capability of the model on nodes with low computing resources. In addition, the above-mentioned optimized calculation graph is obfuscated to generate an obfuscated calculation graph. The leakage of the model structure is avoided through graph obfuscation. In addition, according to the above-mentioned obfuscated calculation graph, a model file to be deployed is generated. Then, according to the above-mentioned node configuration information set and the above-mentioned model deployment type, deployment strategy information is generated. In this way, a deployment strategy matching the node configuration is generated to ensure the successful deployment of the model and the efficient use of the node's computing resources. Finally, according to the above-mentioned deployment strategy information, the above-mentioned model file to be deployed is deployed on the above-mentioned node to be deployed. In this way, the model deployment method of the model on different nodes is improved, and the computing resources of different nodes are efficiently utilized.
[0111] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a model deployment device, and these device embodiments are Figure 1 Corresponding to the method embodiments shown, the model deployment device can be specifically applied to various electronic devices.
[0112] like Figure 2As shown, the model deployment device 200 of some embodiments includes: an acquisition unit 201, a first generation unit 202, a determination unit 203, a graph optimization unit 204, a computational graph obfuscation unit 205, a second generation unit 206, a third generation unit 207 and a deployment unit 208. The acquisition unit 201 is configured to acquire a target model file, wherein the target model file is a model description file corresponding to a machine learning model to be deployed; the first generation unit 202 is configured to generate an initial computational graph according to the target model file, wherein the initial computational graph is a directed graph, and the initial computational graph includes: an operator node set and a relationship edge set, wherein the relationship edge is used to describe the dependency relationship and tensor state between the operator nodes; the second generation unit 203 is configured to determine the model deployment type according to the node configuration information set of the node to be deployed; the graph optimization unit 204 is configured to generate an initial computational graph according to the above target model file; ... is configured to generate an initial computational graph according to the above target model file; the initial computational graph is a directed graph, and the initial computational graph is configured The model deployment type and the operator description information corresponding to the operator node are used to optimize the above-mentioned initial calculation graph to generate an optimized calculation graph; the calculation graph obfuscation unit 205 is configured to perform calculation graph obfuscation on the above-mentioned optimized calculation graph to generate an obfuscated calculation graph; the second generation unit 206 is configured to generate a to-be-deployed model file according to the above-mentioned obfuscated calculation graph; the third generation unit 207 is configured to generate deployment strategy information according to the above-mentioned node configuration information set and the above-mentioned model deployment type; the deployment unit 208 is configured to deploy the above-mentioned to-be-deployed model file on the above-mentioned to-be-deployed node according to the above-mentioned deployment strategy information.
[0113] It is understandable that the units recorded in the model deployment device 200 and the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the model deployment device 200 and the units contained therein, and will not be repeated here.
[0114] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, a computing device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0115] like Figure 3As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory 302 or a program loaded from a storage device 308 into a random access memory 303. Various programs and data required for the operation of the electronic device 300 are also stored in the random access memory 303. The processing device 301, the read-only memory 302, and the random access memory 303 are connected to each other via a bus 304. An input / output interface 305 is also connected to the bus 304.
[0116] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.
[0117] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the read-only memory 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.
[0118] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0119] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0120] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a target model file, wherein the target model file is a model description file corresponding to the machine learning model to be deployed; generates an initial calculation graph according to the target model file, wherein the initial calculation graph is a directed graph, and the initial calculation graph includes: an operator node set and a relationship edge set, and the relationship edge is used to describe the dependency relationship and tensor state between the operator nodes; determines the model deployment type according to the node configuration information set of the node to be deployed; optimizes the initial calculation graph according to the model deployment type and the operator description information corresponding to the operator node to generate an optimized calculation graph; performs calculation graph obfuscation on the optimized calculation graph to generate an obfuscated calculation graph; generates a model file to be deployed according to the obfuscated calculation graph; generates deployment strategy information according to the node configuration information set and the model deployment type; and deploys the model file to be deployed on the node to be deployed according to the deployment strategy information.
[0121] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0122] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0123] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including an acquisition unit, a first generation unit, a determination unit, a graph optimization unit, a computational graph obfuscation unit, a second generation unit, a third generation unit, and a deployment unit. Among them, the names of these units do not constitute a limitation on the unit itself in certain cases. For example, the graph optimization unit may also be described as "a unit that optimizes the above-mentioned initial computational graph according to the above-mentioned model deployment type and the operator description information corresponding to the operator node to generate an optimized computational graph".
[0124] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0125] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.
Claims
1. A model deployment method, include: Obtain a target model file, wherein the target model file is a model description file corresponding to the machine learning model to be deployed; Generate an initial computation graph according to the target model file, wherein the initial computation graph is a directed graph, and the initial computation graph includes: an operator node set and a relationship edge set, and the relationship edge is used to describe the dependency relationship and tensor state between the operator nodes, and the target model file includes: a model definition file; Determine the model deployment type according to the node configuration information set of the node to be deployed; Optimizing the initial computation graph according to the model deployment type and the operator description information corresponding to the operator node to generate an optimized computation graph; Performing computation graph obfuscation on the optimized computation graph to generate an obfuscated computation graph; Generate a model file to be deployed according to the obfuscated calculation graph; Generate deployment strategy information according to the node configuration information set and the model deployment type; According to the deployment strategy information, the model file to be deployed is deployed on the node to be deployed, wherein: The step of generating an initial calculation graph according to the target model file includes: Parsing the model definition file to generate an operator description information set, wherein the operator description information in the operator description information set includes: an operator identifier, an operator type, an operator input description information, an operator output description information, a pre-association operator identifier, and a post-association operator identifier; Filtering out operator description information satisfying a filtering condition from the operator description information set as target operator description information, wherein the filtering condition is: a previous associated operator identifier included in the operator description information is an empty identifier; Generate an operator node of a root node type according to the operator identifier and operator type included in the target operator description information; Taking the operator node of the root node type as the starting node, determining the node dependency relationship according to the previous operator identifier, operator type, previous associated operator identifier, and subsequent associated operator identifier included in the operator description information, so as to generate a candidate computation graph, wherein the candidate computation graph is a computation graph of the tensor state corresponding to the unupdated relationship edge; According to the operator input description information and the operator output description information included in the operator description information in the operator description information set, the state of the tensors corresponding to the relationship edges in the candidate calculation graph is updated to generate the initial calculation graph.
2. The method according to claim 1, in, The generating of deployment strategy information according to the node configuration information set and the model deployment type includes: For each node configuration information in the node configuration information set, generating a node configuration feature corresponding to the node configuration information; Generate deployment strategy constraint information according to the model deployment type; The deployment strategy information is generated according to the obtained node configuration feature set and the deployment strategy constraint information.
3. The method according to claim 2, in, The deploying the to-be-deployed model file on the to-be-deployed node according to the deployment strategy information includes: According to the deployment strategy information, dynamic file distribution is performed on the model file to be deployed, so as to deploy the model file to be deployed on the node to be deployed.
4. The method according to claim 3, in, The node configuration information in the node configuration information set includes: node resource requirement information and node type, the node type includes: edge virtual node type and non-edge virtual node type; and The step of determining the model deployment type according to the node configuration information set of the node to be deployed includes: Filter out node configuration information including a node type of an edge virtual node type from the node configuration information set as first node configuration information; Filter out node configuration information including a node type that is a non-edge virtual node type from the node configuration information set as second node configuration information; Determine a first hardware resource requirement according to the node resource requirement information included in the first node configuration information; Determining a second hardware resource requirement according to the node resource requirement information included in the second node configuration information; Determining a demand ratio of the first hardware resource demand to the second hardware resource demand; In response to the demand ratio being greater than 2, determining the first model deployment type as the model deployment type; In response to the demand ratio being greater than 1 and the demand ratio being less than or equal to 2, determining the second model deployment type as the model deployment type; In response to the demand ratio being greater than 0 and the demand ratio being less than or equal to 1, a third model deployment type is determined as the model deployment type.
5. A model deployment device, include: An acquisition unit is configured to acquire a target model file, wherein the target model file is a model description file corresponding to a machine learning model to be deployed; A first generating unit is configured to generate an initial computation graph according to the target model file, wherein the initial computation graph is a directed graph, the initial computation graph includes: an operator node set and a relationship edge set, the relationship edge is used to describe the dependency relationship and tensor state between the operator nodes, and the target model file includes: a model definition file; A determination unit, configured to determine a model deployment type according to a set of node configuration information of a node to be deployed; A graph optimization unit is configured to perform graph optimization on the initial computation graph according to the model deployment type and the operator description information corresponding to the operator node to generate an optimized computation graph; A computation graph obfuscation unit, configured to perform computation graph obfuscation on the optimized computation graph to generate an obfuscated computation graph; A second generating unit is configured to generate a model file to be deployed according to the obfuscated calculation graph; A third generating unit is configured to generate deployment strategy information according to the node configuration information set and the model deployment type; A deployment unit is configured to deploy the to-be-deployed model file on the to-be-deployed node according to the deployment strategy information, wherein: The step of generating an initial calculation graph according to the target model file includes: Parsing the model definition file to generate an operator description information set, wherein the operator description information in the operator description information set includes: an operator identifier, an operator type, an operator input description information, an operator output description information, a pre-association operator identifier, and a post-association operator identifier; Filtering out operator description information satisfying a filtering condition from the operator description information set as target operator description information, wherein the filtering condition is: a previous associated operator identifier included in the operator description information is an empty identifier; Generate an operator node of a root node type according to the operator identifier and operator type included in the target operator description information; Taking the operator node of the root node type as the starting node, determining the node dependency relationship according to the previous operator identifier, operator type, previous associated operator identifier, and subsequent associated operator identifier included in the operator description information, so as to generate a candidate computation graph, wherein the candidate computation graph is a computation graph of the tensor state corresponding to the unupdated relationship edge; According to the operator input description information and the operator output description information included in the operator description information in the operator description information set, the state of the tensors corresponding to the relationship edges in the candidate calculation graph is updated to generate the initial calculation graph.
6. An electronic device, include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.
7. A computer readable medium having a computer program stored thereon, in, When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.
