A method for processing neural network models and related equipment

Abstract

This invention discloses a neural network model processing method and related equipment. Since the static graph pre-configures a number of equivalent nodes based on preset node attribute reset rules and the node to be processed, after identifying a node as an equivalent node, the node attribute reset rules are applied to the equivalent node. After processing, the connection to the node to be processed is disconnected, allowing the processed equivalent node to replace the node to be processed in the static graph, thus obtaining a processing result that meets the processing requirements. Because the configuration information in the equivalent nodes can indicate the processing rules and the node to be processed, this method can directly and quickly determine the node to be processed and process both the equivalent node and the node to be processed, shortening the development cycle of the neural network model and greatly improving the model's performance.

Description

Technical Field

[0001] This invention relates to the field of model processing technology, and in particular to a method and related equipment for processing neural network models. Background Technology

[0002] Deep learning is a new research direction in the field of machine learning. It is a complex machine learning algorithm that can learn the inherent patterns and representational layers of sample data. The information gained during the learning process greatly helps in interpreting data such as text, images, or sound, and has achieved remarkable results in speech and image recognition. In the field of deep learning, a neural network model is typically trained according to requirements. This neural network model is designed by establishing an appropriate number of neurons and multi-layered computational hierarchies, and selecting suitable input and output layers. Using a large amount of sample data, through network learning and optimization, a functional relationship from input to output is established. This relationship can approximate the relationship between input and output in reality as closely as possible. If the functional relationship achieves this close approximation, the neural network model is successfully trained and can theoretically be applied to certain scenarios, such as speech recognition and image recognition.

[0003] However, before using a trained model, it needs to be deployed online, that is, deployed to a specific application environment or business scenario. If problems occur during deployment, the trained model needs to be processed according to the needs of the application environment or business, such as pruning, modifying, or splitting, and the processed model will be used as the final model for online service.

[0004] The efficiency of this processing and the accuracy of the results directly affect the development cycle and effectiveness of the model. Therefore, how to quickly and accurately process neural network models has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention disclose a neural network model processing method and related equipment to solve the problems of low processing efficiency and accuracy of neural network models in the prior art.

[0006] A neural network model processing method, comprising:

[0007] A reference static graph of the neural network model to be processed is obtained. The reference static graph is obtained by pre-configuring an equal number of nodes based on the static graph of the neural network to be processed. The equal number of nodes are configured according to a preset node attribute reset rule and the node to be processed. The equal number of nodes are connected to the node to be processed, and the connection order is determined according to the node attribute reset rule.

[0008] The nodes in the reference static graph are read sequentially according to a preset order;

[0009] If the node read is the same as the node, set the attributes of the node according to the node attribute reset rule, and disconnect the connection with the node to be processed.

[0010] Once all nodes in the reference static graph have been read and processed, the topology graph consisting of the input nodes, output nodes, and intermediate nodes between the input nodes and output nodes in the reference static graph is determined and used as the processed static graph.

[0011] Based on the processed static graph, the processed neural network model is obtained.

[0012] Preferably, the rules for resetting node attributes include any one or more combinations of resetting input nodes, resetting output nodes, and resetting intermediate nodes.

[0013] Preferably, setting the attributes of the equal number of nodes according to the reset node attribute rules includes:

[0014] If the rule for resetting node attributes is to reset the input node, then the attribute of the equal number of nodes is set as the input node;

[0015] If the rule for resetting node attributes is to reset the output node, then the attribute of the equal number of nodes is set as the output node;

[0016] If the rule for resetting node attributes is to reset intermediate nodes, then the attribute of the equal number of nodes is set to intermediate nodes.

[0017] Preferably, the process of determining the connection order between the equal number of nodes and the node to be processed according to the reset node attribute rule includes:

[0018] If the rule for resetting node attributes is to reset the input node, then the equivalent input node after compiling the equivalent node code is the successor node of the node to be processed.

[0019] If the rule for resetting node attributes is to reset the output node, then the equivalent output node after compiling the equivalent node code is the predecessor node of the node to be processed.

[0020] If the rule for resetting node attributes is to reset intermediate nodes, then the equivalent intermediate nodes after compiling the code of the equivalent nodes are the successor nodes of the node to be processed.

[0021] Preferably, the process of obtaining a reference static graph by pre-configuring an equal number of nodes in the static graph of the neural network to be processed includes:

[0022] Determine the reset node attribute rules corresponding to the processing requirements, and the nodes to be processed corresponding to each reset node attribute rule;

[0023] Generate an equivalent number of node codes for the nodes to be processed, corresponding to the rules for resetting node attributes;

[0024] According to the preset addition rules, the same amount of node code is added to the original code of the neural network to be processed as reference code;

[0025] Compile the reference code to obtain a static diagram, which is then used as a reference static diagram.

[0026] Preferably, the equal number of node codes includes: a function name and a function input, wherein the function name is used to characterize the reset node attribute rule, and the function input includes the node to be processed.

[0027] Preferably, when the reset node attribute rule is to reset intermediate nodes, the equal intermediate node code includes: equal pre-replacement and trimming code and equal post-replacement and trimming code. The function name of the equal pre-replacement and trimming code represents the reset node attribute rule, and the function input includes the node to be processed, the pre-replacement node of the node to be processed, and a first equal node identifier. The function name of the equal post-replacement and trimming code represents the reset node attribute rule, and the function input includes the node to be processed, the post-replacement node of the node to be processed, and a second equal node identifier, wherein the first equal node identifier is the same as the second equal node identifier.

[0028] Preferably, if the node attribute reset rule is to reset the intermediate node, determining the connection order between the equal number of nodes and the node to be processed according to the node attribute reset rule includes the equal number of nodes including: the equal number of pre-replacement nodes corresponding to the equal number of pre-replacement clipping code, and the equal number of post-replacement clipping code corresponding to the equal number of post-replacement clipping code, and the node to be processed is located between the equal number of pre-replacement code nodes and the equal number of post-replacement clipping codes.

[0029] Preferably, the process of setting the attributes of the equal number of nodes according to the reset node attribute rules and disconnecting the connection with the node to be processed when the read node is the same number of nodes includes:

[0030] If the read node is an equal amount of pre-replacement clipping node, determine the corresponding equal amount of post-replacement clipping node;

[0031] Merge the equal amount of pre-replacement clipping nodes and the equal amount of post-replacement clipping nodes, and set the merged node as an equal amount of intermediate nodes;

[0032] Disconnect the connection between the equal number of intermediate nodes and the node to be processed.

[0033] A neural network model processing device, comprising:

[0034] The reference static graph acquisition module is used to acquire a reference static graph of the neural network model to be processed. The reference static graph is obtained by pre-configuring an equal number of nodes based on the static graph of the neural network to be processed. The equal number of nodes are configured according to a preset node attribute reset rule and the node to be processed. The equal number of nodes are connected to the node to be processed, and the connection order is determined according to the node attribute reset rule.

[0035] The node reading module is used to read the nodes in the reference static diagram sequentially according to a preset order;

[0036] The node processing module is used to, when the read node is the same number of nodes, set the attributes of the same number of nodes according to the reset node attribute rules, and disconnect the connection with the node to be processed.

[0037] The topology graph generation module is used to determine, after all the equal number of nodes in the reference static graph have been read and processed, the topology graph formed by the input nodes, output nodes and intermediate nodes between the input nodes and output nodes in the reference static graph, and use it as the processed static graph.

[0038] The model determination module is used to obtain the processed neural network model based on the processed static graph.

[0039] A storage medium comprising a stored program,

[0040] Specifically, during program execution, the device containing the storage medium is controlled to perform the neural network model processing method described above.

[0041] A processor for running programs,

[0042] The program executes the neural network model processing method described above during runtime.

[0043] The neural network model processing method disclosed in the embodiments of this invention pre-configures a number of equivalent nodes in the static graph based on preset node attribute reset rules and the node to be processed. After identifying a node as an equivalent node, the node attribute reset rules are applied to the equivalent node. After processing, the connection to the node to be processed is disconnected, allowing the processed equivalent node to replace the node to be processed in the static graph, thus obtaining a processing result that meets the processing requirements. Since the configuration information in the equivalent nodes indicates the processing rules and the node to be processed, this method can directly and quickly determine the node to be processed and process both the equivalent node and the node to be processed, shortening the development cycle of the neural network model and greatly improving the model's performance. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0045] Figure 1 This is a schematic diagram illustrating the process of converting Python code into a static diagram, as provided in an embodiment of the present invention.

[0046] Figure 2 A flowchart illustrating the processing of a neural network model provided in an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of a reference static diagram provided for an embodiment of the present invention;

[0048] Figure 4 A schematic diagram of yet another reference static diagram provided for an embodiment of the present invention;

[0049] Figure 5 A schematic diagram illustrating the process of obtaining a reference static diagram by pre-configuring an equal number of nodes, as provided in an embodiment of the present invention;

[0050] Figure 6 A schematic diagram of a static image provided for an embodiment of the present invention;

[0051] Figure 7 Provided for embodiments of the present invention Figure 6 An example of adding an equal number of nodes to the original code of the static graph shown;

[0052] Figure 8 for Figure 7 The diagram shown is a reference static image obtained after compiling the code.

[0053] Figure 9 for Figure 8 The image shown is a schematic diagram of the static image after processing.

[0054] Figure 10 for Figure 8 The reference static diagram is a schematic diagram of another processed static diagram.

[0055] Figure 11 This is a schematic diagram of yet another static diagram provided for an embodiment of the present invention;

[0056] Figure 12 Provided for embodiments of the present invention Figure 11 An example of adding an equal number of nodes to the original code of the static graph shown;

[0057] Figure 13 for Figure 12 The diagram shown is a reference static image obtained after compiling the code.

[0058] Figure 14 for Figure 13 The diagram shown is a schematic representation of the effect of processing the static image.

[0059] Figure 15 for Figure 13 The reference static image is shown as a schematic diagram after processing.

[0060] Figure 16 This is a schematic diagram of the structure of a neural network processing device disclosed in an embodiment of the present invention;

[0061] Figure 17 This is a schematic diagram of the reference static image generation module disclosed in an embodiment of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] The training and deployment processes of neural network models are usually independent. Therefore, after the neural network model is trained, there may be scenarios where the neural network needs to be processed. For example, when deploying the trained neural network model to a specific application environment or business scenario, it is necessary to process the trained neural network model according to the specific business scenario requirements. For example, if the trained model contains multiple sub-models, but the actual business scenario only needs to use one of the sub-models, then the trained model needs to be pruned, keeping only one sub-model for online service.

[0064] Another scenario is that if a trained model is too large to be supported by a single server, the model will be split up. A part of the model will be separated and deployed on another server. During online inference, the model will be split into two parts for online service, and new operations will be required to combine the two parts, so that the model can run completely.

[0065] Alternatively, if the trained model has a high computational cost, the intermediate nodes of the model may be pruned to reduce the computational cost and simplify the model architecture.

[0066] Deep learning models typically have model code, usually written in Python. While Python code is easy to write, its execution efficiency is relatively low. To improve efficiency, the Python model code is processed by deep learning frameworks, such as TensorFlow, to generate a static model computation graph, hereinafter referred to as a static graph. This static graph is then used for training or online inference. Figure 1 The diagram illustrates the process of converting Python code into a static graph. Nodes X1, X2, and X3 are the model's input nodes; A, B, and C are the model's parameter inputs; and Y1 and Y2 are the model's output nodes. Intermediate nodes include the addition node `add` and the multiplication nodes `matmul_1` and `matmul`. Model processing typically involves processing the input nodes, output nodes, and intermediate nodes.

[0067] The process of pruning or splitting a trained model, as described above, can be summarized as model processing. When faced with processing requirements for the model, a static model computation graph can usually be used to determine the nodes corresponding to the processing requirements. For example, to reset the model's input or output nodes, nodes need to be reselected and set as new input or output nodes in the static graph. Nodes before the input node need to be deleted, and nodes after the output node need to be deleted. If an intermediate node is deleted, the connection relationships of the remaining nodes that are connected to that intermediate node need to be modified. For example, two nodes that were originally connected through this intermediate node can be directly connected.

[0068] These operations can all be achieved by modifying the static graph, which in turn modifies the model code, ultimately completing the model processing.

[0069] However, using static graphs to process the model often leads to some problems, such as the processed neural network model failing to function properly.

[0070] The inventors discovered that these problems stem from incorrect node selection during splitting or trimming operations. While static diagrams can visually represent the topological connections between nodes in the model, the sheer number and complexity of the nodes make selecting the appropriate node for processing difficult, hindering accurate and rapid identification. Incorrect node selection leads to unusable or malfunctioning models after processing. Resolving these issues requires re-executing the node selection and processing process, resulting in cumbersome deployment steps and impacting development cycles and usability.

[0071] To address the current problems in processing trained models (hereinafter referred to as the neural network model to be processed), the inventors conducted in-depth analysis. To accurately locate and process the nodes corresponding to the processing requirements, information can be added to the static graph to indicate these nodes. This allows for accurate identification of the nodes to be processed during static graph analysis. For example, in the code of the trained model, the nodes requiring processing can be identified, and modifications can be made to their code. These modifications might include adding fields to represent these nodes as new input or output nodes, or changing the names of their parent and child nodes to alter their connections in the static graph. When the modified model code is converted to a static graph, these fields can be used to identify the nodes to be processed, thus achieving accurate location and rapid processing of the nodes to be processed.

[0072] However, this method requires modifying the model's code, which can easily lead to errors if not handled carefully. Moreover, the model's code is very complex, and such modifications can have unpredictable impacts on the model's performance.

[0073] In view of this, the inventors continued their research and proposed the following solution:

[0074] When faced with the need to process a neural network model, after determining the node attribute reset rules and the node to be processed corresponding to the processing requirement, an equivalent node connected to the node to be processed is generated. The equivalent node and the node to be processed coexist in the topology of the reference static graph. The equivalent node is configured according to the predetermined node attribute reset rules and the node to be processed. When processing the neural network model in the reference static graph, the nodes in the reference static graph are read. If an equivalent node is found, its attributes are reset according to the node attribute reset rules, and its connection to the node to be processed is disconnected. This allows the processed equivalent node to replace the node to be processed in the topology, thus achieving the processing of the neural network model. This method can quickly and accurately determine the node to be processed through equivalent nodes and process the neural network model by resetting the attributes of the equivalent nodes. Furthermore, the operation of resetting node attributes applies to the equivalent node rather than the original nodes in the model, so it does not involve modifying the attributes of the original nodes in the model, improving the stability of the system.

[0075] This application discloses a neural network model processing method, the process of which is as follows: Figure 2 As shown, it includes:

[0076] Step S201: Obtain a reference static graph of the neural network model to be processed.

[0077] In this embodiment, after determining the neural network model to be processed, it is not processed directly, but rather using the static graph of the neural network model. In this embodiment, the original static graph of the neural network to be processed is not obtained, but rather a reference static graph. The reference static graph is obtained by pre-configuring an equal number of nodes based on the static graph of the neural network to be processed. That is, in addition to containing all the nodes in the original static graph, the reference static graph also contains a pre-configured equal number of nodes.

[0078] In this embodiment, the equivalent nodes are obtained based on predetermined node attribute reset rules and the configuration of the nodes to be processed. Their function is to replace the nodes to be processed, applying the necessary processing to the nodes according to processing requirements to the equivalent nodes, thus achieving an equivalent replacement of the nodes to be processed.

[0079] Resetting node attribute rules are rules that redefine the attributes of nodes. Node attributes include input nodes, output nodes, and intermediate nodes. When facing the need to process a neural network model, whether it's pruning branches or replacing nodes, it can be converted into an operation of setting the attributes of nodes in the model. Therefore, in this embodiment, resetting node attribute rules are rules that redefine the attributes of nodes. Resetting node attribute rules include any one or more combinations of resetting input nodes, resetting output nodes, and resetting intermediate nodes. The specific rules included in resetting node attribute rules are related to the processing requirements of the neural network to be processed.

[0080] Nodes to be processed are those corresponding to processing requirements. In other words, after understanding the processing requirements in advance, it is determined which nodes need to be processed to satisfy those requirements; these nodes are the nodes to be processed. An equal number of nodes are connected to the nodes to be processed, and the connection order is determined according to the rules for resetting node attributes.

[0081] like Figure 3 The diagram shows a reference static graph of the neural network to be processed. Nodes X1, A, sub, matmul, and Y1 are nodes from the original static graph of the neural network. The two nodes labeled with equal input and equal output are examples of two equal-input nodes. In this example, the node to be processed corresponding to the equal-input node is X1, and the node to be processed corresponding to the equal-output node is Y1.

[0082] Since the equal number of nodes is obtained based on the rules for resetting node attributes and the configuration of the nodes to be processed, the configuration information of the equal number of nodes contains information that can indicate the nodes to be processed and the rules for resetting node attributes.

[0083] Step S202: Read the nodes in the reference static diagram in sequence according to the preset order.

[0084] In this embodiment, the preset order in this step can be either breadth-first search or depth-first search to read nodes sequentially. If breadth-first search is used, the preset reading order is to start from an input node, sequentially read the nodes connected to this input node (i.e., adjacent nodes), and then sequentially read the adjacent nodes of each adjacent node.

[0085] If a depth-first search is used, it starts from the input node, reads upwards sequentially until the output node, then backtracks to the previous node, searches for a new branch, and reads upwards again, reading the nodes in the reference static graph sequentially.

[0086] Step S203: If the read node is an equal number of nodes, set the attributes of the equal number of nodes according to the node attribute reset rules, and disconnect the connection with the node to be processed.

[0087] The configuration information of each node is read sequentially. Referring to the information carried by input nodes, output nodes, and intermediate nodes in the static graph, this includes node attributes, node connection relationships, etc. Node attributes include whether a node is an input node, output node, or intermediate node. The node connection relationship refers to the node's predecessor and successor nodes in the static graph's topology. The configuration information for equal-quantity nodes includes information indicating processing rules. Therefore, when an equal-quantity node is read, the node attribute reset rule is obtained. Based on this processing rule, the equal-quantity node's attributes are set. For example, if the node attribute reset rule is to reset the input node, then the equal-quantity node's attribute is set to input node.

[0088] If the rule for resetting node attributes is to reset the output node, then the attribute of the equal number of nodes is set to the output node.

[0089] If the rule for resetting node attributes is to reset the intermediate node, then the attribute of the equal number of nodes is set to the intermediate node.

[0090] A node that has been disconnected can be directly deleted, along with any branches that are only connected to that node. This is because once the node is deleted, the branch can no longer exist in the topology graph and should also be deleted. Figure 4 As shown in the diagram, if the node to be processed is node sub, after setting the equal-quantity node as the equal-quantity input node and disconnecting the connection with the node to be processed sub, if node sub is deleted, the branch in the dashed box with sub as the endpoint will no longer be able to connect to other nodes in the topology graph. In this case, node X1 and node A in the branch should be deleted together.

[0091] In some cases, the node to be processed after disconnection can remain in the reference static graph, while the topology structure composed of the nodes corresponding to the processing requirements is directly extracted from the reference static graph as the processed result. In this embodiment, the node to be processed and other nodes in the topology structure are disconnected by disconnection, so that subsequent processing can be carried out according to specific requirements.

[0092] It can be seen that although the object of this application is nodes, the processing of nodes can directly affect the branches including the nodes because there are topological relationships between the nodes in the topology graph. Based on this, the processing of the neural network model to be processed in this application can be realized.

[0093] Step S204: After all the nodes in the reference static graph have been read and processed, determine the topology graph formed by the input nodes, output nodes, and intermediate nodes between the input and output nodes in the reference static graph, and use it as the processed static graph.

[0094] In the previous step, disconnecting the node to be processed from the equivalent number of nodes means that the node to be processed is removed from the topology graph, resulting in a change in the topological relationships of the nodes in the topology graph. At the same time, since the input, output, or intermediate nodes are reset, the topological relationships of the remaining nodes in the topology graph will also change. Therefore, here, the intermediate nodes are recalculated based on the input and output nodes to form a new topology graph, which serves as the processed static graph.

[0095] Step S205: Obtain the processed neural network model based on the processed static graph.

[0096] After obtaining the processed static graph, the code of the processed neural network is obtained by decompiling, and then the processed neural network model is obtained.

[0097] Figure 1 The neural network model processing method disclosed in the illustrated embodiment pre-configures a number of equivalent nodes in the static graph based on preset node attribute reset rules and the node to be processed. After identifying a node as an equivalent node, the node attribute reset rules are applied to the equivalent node. Upon completion of processing, the connection to the node to be processed is disconnected, allowing the processed equivalent node to replace the node to be processed in the static graph, thus obtaining a processing result that meets the processing requirements. Since the configuration information in the equivalent nodes indicates the processing rules and the node to be processed, this method can directly and quickly determine the node to be processed and process both the equivalent node and the node to be processed, shortening the model development cycle and significantly improving the model's effectiveness.

[0098] The above embodiments illustrate the process and effects of processing neural network models. It can be seen that the role of equal-quantity nodes is crucial in this scheme. The specific implementation process is obtained by pre-configuring equal-quantity nodes based on the static graph of the neural network to be processed. This can be achieved by adding equal-quantity node code to the original code of the neural network model, and then compiling the neural network model code after adding the equal-quantity node code to obtain the static graph, which will then contain equal-quantity nodes.

[0099] Similarly, to avoid affecting the normal operation of the neural network model, the equivalent amount of node code added to its original code is pass-through code. For example, interface program code with pass-through functionality only serves as a node for data transmission and does not process the data. Even if a neural network model with the equivalent amount of node code is trained, it will not affect the performance of the neural network model.

[0100] In the above embodiments, the process of pre-configuring an equal number of nodes to obtain a reference static graph can be implemented using a pre-compiled tool, which can be... Figure 2 In the illustrated embodiment, the tool used to perform the processing of the reference static diagram can also be a standalone tool, and this is not limited here. Regardless of which tool is used, the principles and procedures followed are basically the same.

[0101] The following is combined Figure 5 The process of obtaining a reference static graph by pre-configuring an equal number of nodes is described in detail, including:

[0102] Step S501: Determine the reset node attribute rules corresponding to the processing requirements, and the nodes to be processed corresponding to each reset node attribute rule.

[0103] After the tool starts, it first determines the rules for resetting node attributes and the corresponding nodes to be processed. This determination process can be either receiving information containing the rules for resetting node attributes and the corresponding nodes to be processed, and directly retrieving the rules and nodes from the information; or receiving information representing processing requirements, analyzing the requirements according to predetermined analysis rules, and then obtaining the rules for resetting node attributes and the corresponding nodes to be processed. The specific implementation process is not limited.

[0104] like Figure 6 The diagram shows a static graph containing three input nodes X1, X2, and X3, two output nodes Y1 and Y2, and several intermediate nodes between the input and output nodes. If the processing requirement is to preserve the topology formed by the nodes within the dashed box, analyzing the preserved topology reveals that the input and output nodes have changed compared to the original static graph. Therefore, to ensure the preserved nodes form a complete topology, it is necessary to define input and output nodes for them.

[0105] Therefore, the processing rules corresponding to the processing requirements include: resetting the input nodes, resetting the output nodes, and resetting multiple input nodes and multiple output nodes.

[0106] Corresponding to Figure 6 The topology shown in the dashed box requires resetting two input nodes and one output node in this embodiment. The node attribute reset rules include: resetting the first input node, resetting the second input node, and resetting the output node.

[0107] Of course, other processing requirements may also include resetting intermediate nodes, and there may be multiple intermediate nodes that can be reset.

[0108] according to Figure 6 The processing requirements shown can directly determine which nodes in the static graph will be used as input nodes and which will be used as output nodes. In this embodiment, node add is used as the first input node and node X2 is used as the second input node, so the nodes to be processed are node add and node X2. Node Y1 is used as the output node, so the node to be processed is node Y1.

[0109] In this embodiment, the node to be processed corresponding to the rule of the first input node is reset to node add, the node to be processed corresponding to the rule of the second input node is reset to node X2, and the node to be processed corresponding to the rule of the output node is reset to Y1.

[0110] Step S502: Generate the same number of node codes for the nodes to be processed corresponding to the rules for resetting node attributes.

[0111] The code for an equal number of nodes includes: a function name and function input. The function name is used to represent the rules for resetting node attributes, and the function input includes the node to be processed.

[0112] by Figure 6 In the example shown, resetting the node attribute rule to reset the first input node is taken as an example. The equivalent node code is `tf.equivalent_input(node)`, which means that node `node` is used as the equivalent input node, that is, the node to be processed is set as the equivalent input node. In this embodiment, the node to be processed is `add`, so its corresponding equivalent input node code is `equivalent input(add)`. The node generated by this equivalent input code is used as the equivalent node of the node to be processed. That is, in this application, if the processing requirement needs to set `add` as the first input node, then an equivalent input code is generated in this step. The node generated by this equivalent input code is the equivalent node of node `add`, and in subsequent processing, it is used to replace `add` as the first input node. Because the function input is the node to be processed, the configuration information of the node generated by this equivalent input code contains information indicating the node to be processed, while the function name indicates information indicating the processing rule.

[0113] When the node attribute reset rule is to reset the second input node, the corresponding equivalent node code is similar to the example above. It is also an equivalent input node code. You only need to change the node to be processed in the function input to X2. Then the equivalent input code is tf.equivalent_input(X2).

[0114] When the node attribute reset rule is to reset the output node, its corresponding equivalent node code can be the equivalent output node code tf.equivalent_output(Y1). The node generated by this code is the equivalent node of Y1.

[0115] When resetting node attributes by resetting intermediate nodes, let's assume the node to be processed is... Figure 6 The node matmul_1 connected to the node add shown in the diagram has the equivalent node code tf.equivalent_replace(matmul_1). This code generates a node that is an equivalent intermediate node with the same function as matmul. It can be regarded as an equivalent intermediate node and can directly replace the matmul node.

[0116] The equivalent node code in this embodiment uses a Python interface program as an example. In practical applications, the equivalent node code can have various implementation forms.

[0117] To ensure the stability of the neural network model to be processed, the code for the same number of nodes in this embodiment is all passed through in an equal amount, and the nodes generated are also passed through in an equal amount, so they will not affect the data processing of the neural network to be processed.

[0118] Step S503: According to the preset addition rules, add an equal amount of node code to the original code of the neural network to be processed as reference code.

[0119] The default addition principle is to satisfy the condition that the connection order of equal number of nodes and nodes to be processed corresponds to the node attribute reset rule of equal number of nodes.

[0120] The generated equivalent node code needs to be added to the original code and compiled together with the original code to ultimately appear as nodes in the static graph. When adding, the equivalent node code can be placed at a specific location in the original code, such as the beginning or end of the code. Alternatively, it can be added after the code corresponding to the node to be processed, so that the node to be processed and its corresponding equivalent node can be compiled consecutively during compilation. Regardless of the method, the connection order of the equivalent nodes and the node to be processed must correspond to the rules for resetting the node attributes of the equivalent nodes.

[0121] The correspondence between the connection order of equal nodes and nodes to be processed and the processing rules for equal nodes is as follows:

[0122] If the rule for resetting node attributes is to reset the input node, then the equivalent input node after compiling the equivalent node code is the successor node of the node to be processed.

[0123] If the rule for resetting node attributes is to reset the output node, then the equivalent output node code after compiling the equivalent node code is the predecessor node of the node to be processed.

[0124] If the rule for resetting node attributes is to reset intermediate nodes, then the equivalent intermediate node code after compiling the equivalent node code is the successor node of the node to be processed.

[0125] The connection order of equal nodes and nodes to be processed is set according to the above rules in order to ensure that, in the subsequent processing of nodes in the static graph, after the node to be processed is disconnected, the equal nodes can still remain in the topology and replace the node to be processed as new input, output or intermediate nodes.

[0126] Still with Figure 6 As shown in the example, in order to preserve the topology within the dashed box, the corresponding equivalent code is generated and added to... Figure 4 The result of the corresponding original code is as follows Figure 7 As shown. The code within the wireframe represents the code for the added, equal number of nodes.

[0127] Step S504: Compile the reference code to obtain a static diagram, and use it as a reference static diagram.

[0128] If a reference static graph contains nodes that correspond to the code of an equal node, these nodes are defined as equal nodes.

[0129] Will Figure 7 After compiling the code, the resulting reference static diagram is as follows: Figure 8 As shown, the added equal-quantity node codes are already displayed in the reference static graph. For example, the node with equal-quantity input is connected between node add and node matmul_1, the node with equal-quantity input is connected between node X2 and node sub, and the node with equal-quantity output is connected between node matmul and Y1.

[0130] In the process of obtaining a reference static graph by pre-configuring equal-quantity nodes disclosed in this embodiment, after determining the node to be processed, equal-quantity node code corresponding to the node to be processed and the node attribute reset rules is generated and added to the original code so that an equal-quantity node can be generated after the code is compiled. Its position is set to be connected to the node to be processed, and the connection order corresponds to the node attribute reset rules. The equal-quantity node and the node to be processed coexist in the topology of the reference static graph. Since the equal-quantity code is generated based on the node to be processed and the node attribute reset rules, its corresponding equal-quantity node configuration information can be used to indicate the node to be processed and the node attribute reset rules. Therefore, in the subsequent processing of the reference static graph, this information can be read to determine how to process the node to be processed. Furthermore, the processing operations originally intended for the node to be processed are applied to the equal-quantity node. This operation does not affect the performance of the neural network model being processed, providing a reliable and stable foundation for the processing of the neural network model.

[0131] This embodiment only uses the above-mentioned rules for resetting node attributes corresponding to processing needs as an example. In specific application scenarios, all of the above rules for resetting node attributes may be included simultaneously, and the number of input nodes, output nodes, and intermediate nodes that are reset may also be multiple. However, regardless of the specific number of rules for resetting node attributes, they are all categorized under the three rules of resetting input nodes, resetting output nodes, and resetting intermediate nodes. The combination of these three types of rules can cover the current needs for processing neural network models.

[0132] by Figure 8 The process of processing the neural network model shown in the static diagram is illustrated below.

[0133] First, a reference static graph is obtained. Nodes in the reference static graph are read sequentially from input nodes to output nodes. When the first equal node is read, the node attribute reset rule for the equal node and the node to be processed are obtained. The node attribute reset rule is to reset the input node. The node to be processed is "add". Therefore, the equal node is set as an equal input node, and the connection between the equal input node and the node to be processed "add" is disconnected. This operation also means that the branch ending at "add" is broken.

[0134] Similarly, all other equal nodes are read and processed. Once all equal nodes have been processed, and referring to the static graph where new input and output nodes have already been determined, the intermediate nodes between the input and output nodes are determined to form a new topology graph, which serves as the processed static graph.

[0135] like Figure 9As shown, the new topology is the part within the solid line box, and the part within the dashed line box represents the nodes that were deleted in this process.

[0136] exist Figure 5 In the illustrated embodiment, when generating the code for the equivalent number of nodes, since it has the same function as the node to be processed, the identifier of the node to be processed can be used as the identifier of the equivalent number of nodes.

[0137] Furthermore, to more clearly display the equivalent nodes in the reference static graph, a new identifier can be set for them when generating the equivalent node code. This can be achieved simply by modifying the equivalent node code. For example, the equivalent input node code could be `tf.equivalent_input(add, name="x1")`, which adds the function input `x1` as the identifier for the equivalent input node in addition to the original function input `add`. The equivalent output code could be `tf.equivalent_output(Y1, name="y1")`, using `y1` as the identifier for the equivalent output node. Using this as an example, then... Figure 8 The reference static image shown is processed as follows: Figure 10 As shown.

[0138] The equivalent intermediate code can be tf.equivalent_replace(matmul, name="new"), with new as the identifier for the equivalent intermediate code.

[0139] exist Figure 5 The example shown illustrates the equivalent amount of intermediate code, which can identify which node is to be processed. However, considering that the connection relationship of intermediate nodes is also a critical factor in the process of resetting intermediate nodes, errors in this process can lead to mistakes in subsequent processing.

[0140] To address this issue, this application discloses an alternative method for setting an equal amount of intermediate code.

[0141] When resetting node attribute rules involves resetting intermediate nodes, the equivalent intermediate node code includes: equivalent pre-replacement and trimming code and equivalent post-replacement and trimming code. The function name of the equivalent pre-replacement and trimming code represents the node attribute reset rule, and the function input includes the node to be processed, the pre-replacement node of the node to be processed, and the first equivalent node identifier. The function name of the equivalent post-replacement and trimming code represents the node attribute reset rule, and the function input includes the node to be processed, the post-replacement node of the node to be processed, and the second equivalent node identifier. The first equivalent node identifier is the same as the second equivalent node identifier.

[0142] In this method, the equivalent intermediate code consists of two pairs of codes, each containing information indicating the predecessor and successor nodes of the node to be processed.

[0143] For example, the equivalent pre-replacement clipping code: tf.equivalent_pre(node,replace_to_new) is used to indicate the predecessor node of the node to be processed.

[0144] Where node represents the name of the predecessor node of the node to be processed, and replace_to_new is the identifier of the first node.

[0145] The code for equivalent post-replacement clipping is: tf.equivalent_post(node,replace_to_new,name) is used to indicate the predecessor node of the node to be processed.

[0146] Where node represents the name of the successor node to the node to be processed, and replace_to_new is the identifier of the second equivalent node.

[0147] In both code snippets, the first and second node identifiers are both `replace_to_new`. This ensures that the two code snippets appear in pairs: one represents the predecessor node of the node to be processed, and the other represents the successor node. This allows for accurate determination of the position and connection relationships of the node to be processed. Finally, the node to be processed is replaced with the equivalent `replace_to_new` node.

[0148] In this embodiment, the equivalent intermediate node also includes a pair of equivalent nodes: an equivalent preceding replacement pruning node that serves as the predecessor node of the node to be processed, and an equivalent following replacement pruning node that serves as the successor node of the node to be processed. That is, when determining the relationship between the node to be processed and the equivalent nodes according to the node attribute reset rule, the node to be processed is located between the equivalent preceding replacement pruning node and the equivalent following replacement pruning node.

[0149] by Figure 11 Taking the static diagram shown as an example, and combining it with the above-mentioned method of generating an equal amount of intermediate code, the process of processing the model to be processed is explained.

[0150] Figure 11The dotted lines indicate the nodes that need to be deleted. It's clear that the processing requirement is to delete the branch ending with `matmul_l`. In other words, the intermediate nodes of `add` and `matmul` need to be reset. The processing rule is to reset the intermediate nodes. The node to be processed is `matmul_1`, and the generated equivalent intermediate node code is the equivalent pre-replacement trimming code `tf.equivalent_pre(add, replace_to_new=“new_op”)` and the equivalent post-replacement trimming code `tf.equivalent_post(matmul_1, replace_to_new=…”)`.

[0151] The two code snippets generate equal number of nodes, which serve as the predecessor and successor nodes to indicate the node to be processed.

[0152] The diagram shows how to add these two code snippets after the original code of the neural network model to be processed, as shown below. Figure 12 As shown, the code for the added equal number of nodes is within the dashed box.

[0153] Although there are two code segments, the fact that the new nodes in the nodes have the same name means that these two code segments need to be used in pairs, as shown in the compiled reference static graph. Figure 13 As shown, the vertices of both triangles point to the same location, which is the location of the node to be processed.

[0154] During the processing of the reference static graph, each node is read starting from the input node X1. When an equal-quantity pre-replacement pruning node is read, the corresponding equal-quantity post-replacement pruning node can be determined through configuration information. Since two nodes will eventually be replaced by one node, the equal-quantity pre-replacement pruning node and the equal-quantity post-replacement pruning node are merged, and the merged node is set as the equal-quantity intermediate node new_op. After setting, the connection with the node to be processed, matmul_1, can be disconnected. Figure 14 As shown in the diagram. The node new_op remaining in the topology at this point replaces node matmul_1, forming the processed reference static graph. Finally, based on the processed reference static graph, the processed neural network model is obtained as follows: Figure 15 As shown.

[0155] In this embodiment, the node to be processed shown in the figure is a single node. Under other processing requirements, the node to be processed can also be multiple consecutive nodes. Therefore, when using the method in this embodiment, simply set the equal-quantity pre-replacement clipping node as the pre-node of the multiple nodes to be processed, and set the equal-quantity post-replacement clipping node as the post-node of the multiple nodes to be processed. Then, the clipping or replacement of multiple nodes to be processed can be achieved according to the above steps.

[0156] This application discloses a neural network model processing device, the structure of which is as follows: Figure 16 As shown, it includes:

[0157] The reference static graph acquisition module 1601 is used to acquire a reference static graph of the neural network model to be processed. The reference static graph is obtained by pre-configuring an equal number of nodes based on the static graph of the neural network to be processed. The equal number of nodes are configured according to a preset reset node attribute rule and the node to be processed. The equal number of nodes are connected to the node to be processed, and the connection order is determined according to the reset node attribute rule.

[0158] Node reading module 1602 is used to read the nodes in the reference static diagram sequentially according to a preset order;

[0159] The node processing module 1603 is used to, when the read node is the same number of nodes, set the attributes of the same number of nodes according to the reset node attribute rules, and disconnect the connection with the node to be processed.

[0160] The topology graph generation module 1604 is used to determine, after the equal number of nodes in the reference static graph have been read and processed, the topology graph formed by the input nodes, output nodes and intermediate nodes between the input nodes and output nodes in the reference static graph, and use it as the processed static graph.

[0161] The model determination module 1605 is used to obtain the processed neural network model based on the processed static graph.

[0162] The neural network model processing device disclosed in this embodiment receives a reference static graph pre-configured with an equal number of nodes based on preset node attribute reset rules and the nodes to be processed. After identifying a node as an equal node, the device applies the node attribute reset rules to the equal node. Upon completion of processing, it disconnects the node to be processed, allowing the processed equal node to replace the node to be processed in the static graph, thus obtaining a processing result that meets the processing requirements. Since the configuration information in the equal nodes indicates the processing rules and the nodes to be processed, the device can directly and quickly determine the nodes to be processed and process both the equal nodes and the nodes to be processed, shortening the model development cycle and greatly improving the model's usability.

[0163] Furthermore, this embodiment of the invention also discloses a reference static graph generation module, used to obtain a reference static graph by pre-configuring an equal number of nodes based on the static graph of the neural network to be processed. The structure of the reference static graph generation module is as follows: Figure 17 As shown, it includes:

[0164] The determining unit 1701 is used to determine the reset node attribute rules corresponding to the processing requirements, and the nodes to be processed corresponding to each reset node attribute rule.

[0165] Code generation unit 1702 is used to generate the same amount of node code for the node to be processed as the rule for resetting node attributes;

[0166] Code addition unit 1703 is used to add an equal amount of node code to the original code of the neural network to be processed according to a preset addition rule, as a reference code;

[0167] Code compilation unit 1704 is used to compile the reference code to obtain a static graph, which is then used as the reference static graph.

[0168] The reference static image generation module in this embodiment can be used as... Figure 16 Part of the neural network model processing device shown, with Figure 16 The components shown work together. They can also be independently installed in other processors, functioning as a separate unit. Figure 16 The neural network model processing device shown is used in conjunction with this device.

[0169] The specific workflow of the neural network model processing device and reference static graph generation module disclosed in this embodiment can be found in [reference]. Figure 2-15 The embodiments shown are not described in detail here.

[0170] This invention provides a storage medium storing a program that, when executed by a processor, implements a neural network model processing method.

[0171] This invention provides a processor for running a program, wherein the program executes a neural network model processing method during runtime.

[0172] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0173] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A neural network model processing method, characterized in that, The neural network model processing method is applied to deploying a trained neural network model to a specific application environment or a specific business scenario, and includes the following steps: obtaining a reference static graph of a neural network model to be processed, the reference static graph being obtained according to a static graph of the neural network to be processed and a preset equal node, the equal node being connected to a node to be processed according to a preset node attribute resetting rule and a node configuration to be processed, the connection order being determined according to the node attribute resetting rule; the node to be processed is a node corresponding to a processing requirement, and the processing requirement includes a requirement for splitting the neural network model to be processed when a server cannot implement the neural network model to be processed; reading the nodes in the reference static graph in a preset order; in the case that the read node is the equal node, setting the attribute of the equal node according to the node attribute resetting rule, and disconnecting the connection with the node to be processed; in the case that the equal nodes in the reference static graph are read and processed, determining a topology graph composed of an input node, an output node and intermediate nodes between the input node and the output node in the reference static graph, and taking the topology graph as a processed static graph; obtaining a processed neural network model according to the processed static graph.

2. The method of claim 1, wherein, The node attribute resetting rule includes any one or a combination of resetting the input node, resetting the output node and resetting the intermediate node.

3. The method of claim 2, wherein, The process of setting the attribute of the equal node according to the node attribute resetting rule includes: if the node attribute resetting rule is to reset the input node, the attribute of the equal node is set as an input node; if the node attribute resetting rule is to reset the output node, the attribute of the equal node is set as an output node; if the node attribute resetting rule is to reset the intermediate node, the attribute of the equal node is set as an intermediate node.

4. The method of claim 3, wherein, The process of determining the connection order of the equal node and the node to be processed according to the node attribute resetting rule includes: if the node attribute resetting rule is to reset the input node, the equal input node after code compilation of the equal node is a post-node of the node to be processed; if the node attribute resetting rule is to reset the output node, the equal output node after code compilation of the equal node is a pre-node of the node to be processed; if the node attribute resetting rule is to reset the intermediate node, the equal intermediate node after code compilation of the equal node is a post-node of the node to be processed.

5. The method of claim 1, wherein, The process of obtaining the reference static graph by preconfiguring the equal node in the static graph of the neural network to be processed includes: determining a node attribute resetting rule corresponding to a processing requirement and a node to be processed corresponding to each node attribute resetting rule; generating equal node code of the node to be processed corresponding to the node attribute resetting rule; adding the equal node code to the original code of the neural network to be processed as reference code according to a preset addition rule; compiling the reference code to obtain a static graph, which is taken as a reference static graph.

6. The method of claim 5, wherein, The equivalent node code comprises a function name and function input, wherein the function name is used to represent the reset node attribute rule, and the function input comprises the to-be-processed node.

7. The method of claim 6, wherein, In the case where the reset node attribute rule is to reset an intermediate node, the equivalent intermediate node code comprises equivalent pre-replacement clipping code and equivalent post-replacement clipping code, wherein the function name of the equivalent pre-replacement clipping code is used to represent the reset node attribute rule, and the function input comprises the to-be-processed node, a pre-node of the to-be-processed node, and a first equivalent node identifier; the function name of the equivalent post-replacement clipping code is used to represent the reset node attribute rule, and the function input comprises the to-be-processed node, a post-node of the to-be-processed node, and a second equivalent node identifier, the first equivalent node identifier being the same as the second equivalent node identifier.

8. The method of claim 2, wherein, In the case where the reset node attribute rule is to reset an intermediate node, the connection order of the equivalent node and the to-be-processed node is determined according to the reset node attribute rule, and the equivalent node comprises an equivalent pre-replacement clipping node corresponding to the equivalent pre-replacement clipping code and an equivalent post-replacement clipping node corresponding to the equivalent post-replacement clipping code, and the to-be-processed node is located between the equivalent pre-replacement clipping node and the equivalent post-replacement clipping node.

9. The method of claim 8, wherein, In the case where the read node is the equivalent node, the process of setting the attribute of the equivalent node according to the reset node attribute rule and disconnecting the connection with the to-be-processed node comprises: In the case where the read node is an equivalent pre-replacement clipping node, determining an equivalent post-replacement clipping node corresponding thereto; Merging the equivalent pre-replacement clipping node and the equivalent post-replacement clipping node, and setting the merged node as an equivalent intermediate node; Disconnecting the equivalent intermediate node and the to-be-processed node.

10. A neural network model processing apparatus, comprising: The neural network model processing method is applied to deploying a trained neural network model to a specific application environment or a specific business scenario, and comprises: A reference static graph acquisition module is configured to acquire a reference static graph of a to-be-processed neural network model, the reference static graph being obtained according to a static graph of the to-be-processed neural network and pre-configured equivalent nodes, the equivalent nodes being configured according to a preset reset node attribute rule and a to-be-processed node, the equivalent nodes being connected with the to-be-processed node in a connection order determined according to the reset node attribute rule; the to-be-processed node is a node corresponding to a processing requirement, and the processing requirement comprises a requirement of splitting the to-be-processed neural network model when a server cannot implement the to-be-processed neural network model; A node reading module is configured to read nodes in the reference static graph in a preset order; A node processing module is configured to, in the case where a read node is an equivalent node, set an attribute of the equivalent node according to a reset node attribute rule and disconnect the connection with a to-be-processed node. A topology graph generation module is configured to, when the equivalent nodes in the reference static graph are read and processed, determine a topology graph composed of input nodes, output nodes and intermediate nodes between the input nodes and the output nodes in the reference static graph, and take the topology graph as a processed static graph. A model determination module is configured to obtain a processed neural network model according to the processed static graph.

11. A storage medium, characterized by The storage medium includes a stored program, When the program is executed, the device in which the storage medium is located performs the neural network model processing method in any one of claims 1-9.

12. A processor, comprising: The processor is configured to execute the program, When the program is executed, the device in which the storage medium is located performs the neural network model processing method in any one of claims 1-9.

Images