Image cutting method and device for open neural network switching model, and electronic equipment
By converting the implicit graph of ONNX into an explicit directed graph and using algorithms such as community detection for segmentation, the problem of inaccurate graph slicing in ONNX is solved, achieving highly accurate graph slicing and simplifying subsequent operations.
Patent Information
- Application Number
- CN202511574242.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-30
- Publication Date
- 2026-01-30
AI Technical Summary
In existing technologies, the ONNX slicing method has difficulty accurately handling subgraphs with multiple inputs and outputs or complex connections, resulting in inaccurate slicing.
The implicit graph of ONNX is converted into an explicit directed graph. Nodes and edges are constructed using an inverted dictionary. The graph is then divided into multiple connected subgraphs using algorithms such as community detection, cut vertex, and biconnected components, and finally converted into the target ONNX file.
It improves the accuracy and flexibility of ONNX graph slicing, ensuring that each subgraph is an independent computational unit, and simplifies subsequent graph analysis and deployment operations.
Smart Images
Figure CN121436073A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of deep learning, for example to a cutting method and device for an open neural network exchange model and electronic equipment. BACKGROUND
[0002] With the rapid development of automatic driving technology, the vehicle end model uses modules mixed with conv (Convolution) and transformer, and the model generally uses an end-to-end solution, including perception and control two parts. The development of technology and the requirements of business make the model overall larger and larger, which is manifested as more and more operators, such as reaching the level of 4w onnx (Open Neural Network Exchange) operators. However, for the onnx representation file of the multi-operator model, it is also difficult to open using the commonly used netron tool in the industry; at the same time, various bugs will appear when deploying in the commonly used tensorrt. Therefore, a cutting method is urgently needed to cut the onnx into small pieces.
[0003] In the related art, an onnx cutting method is disclosed, which uses the function onnx.utils.extract_model() in the python library, inputs the names of the input edges and output edges of the subgraph, and can realize onnx cutting. Among them, the names of the input edges and output edges need to be found by using netron to open the onnx and find the input edges and output edges of the subgraph.
[0004] In the process of implementing the embodiments of the present disclosure, it is found that at least the following problems exist in the related art: If the subgraph has multiple inputs and outputs, or the connection between the subgraph and the peripheral part is complex, it is difficult to find the input edges and output edges of the subgraph using netron, and it is difficult to realize accurate onnx cutting.
[0005] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY
[0006] In order to have a basic understanding of some aspects of the disclosed embodiments, the following is a simple summary. The summary is not a general review, nor is it intended to determine the key / important components or delineate the scope of protection of these embodiments, but as a prelude to the detailed description below.
[0007] The embodiments of the present disclosure provide a cutting method and device for an open neural network exchange model and electronic equipment to improve the accuracy of onnx cutting.
[0008] In some embodiments, the graph slicing method for an open neural network exchange model includes: converting an implicit graph in the open neural network exchange model into an explicit directed graph; slicing the directed graph into multiple connected subgraphs; and converting the multiple connected subgraphs into multiple target open neural network exchange model files, respectively.
[0009] Optionally, converting the implicit graph in the open neural network exchange model into an explicit directed graph includes: constructing a first inverted dictionary and a second inverted dictionary based on the nodes in the implicit graph; the first inverted dictionary is a dictionary from node input names to node names, and the second inverted dictionary is a dictionary from node output names to node names; converting the inputs and outputs in the implicit graph into pseudo-nodes of the directed graph; and converting the nodes and edges in the implicit graph into nodes and edges of the directed graph based on the first inverted dictionary and the second inverted dictionary.
[0010] Optionally, the nodes and edges in the implicit graph are converted into nodes and edges in the directed graph based on the first and second inverted dictionaries, including: traversing each node in the implicit graph and traversing the input edges and output edges of each node; finding the relevant nodes corresponding to each input edge and output edge based on the first and second inverted dictionaries, and determining the names of the relevant nodes for each input edge and output edge; constructing the nodes and edges of the directed graph based on the names of the relevant nodes for each input edge and output edge; and defining the attributes of the edges in the directed graph as the names of the edges in the corresponding implicit graph based on the correspondence between the edges in the implicit graph and the edges in the directed graph.
[0011] Optionally, the directed graph can be divided into multiple connected subgraphs, including: dividing the directed graph into a preset number of node sets based on a community detection algorithm, and obtaining a preset number of connected subgraphs based on the node sets; or, dividing the directed graph based on the cut vertices and biconnected components of the directed graph to obtain multiple connected subgraphs; or, forming multiple connected subgraphs based on all ancestor nodes of the target node; or, dividing the directed graph based on the plug-in type nodes in the directed graph to obtain multiple connected subgraphs; or, dividing the directed graph based on the marked nodes in the directed graph to obtain multiple connected subgraphs.
[0012] Optionally, the directed graph is divided according to the cut vertices and biconnected components to obtain multiple connected subgraphs, including: determining the cut vertices and biconnected components in the directed graph based on a preset function; after removing the cut vertices from the biconnected components, dividing the directed graph into connected subgraphs corresponding to the biconnected components after the cut vertices are removed, and the cut vertices; or, dividing the directed graph into connected subgraphs corresponding to the biconnected components, wherein adjacent biconnected components have the same cut vertices.
[0013] Optionally, the directed graph can be divided according to the nodes of the plug-in type in the directed graph to obtain multiple connected subgraphs, including: deleting all plug-in type nodes in the directed graph; and finding all connected components in the remaining directed graph to obtain multiple connected subgraphs.
[0014] Optionally, the connected subgraph is converted into a target open neural network exchange model file in the following manner: after removing pseudo-nodes from the connected subgraph, the nodes and edges of the target open neural network exchange model are determined based on the nodes and edges in the connected subgraph; the graph structure and model structure of the target open neural network exchange model are determined based on the metadata information of the implicit graph; and the nodes, edges, graph structure, and model structure of the target open neural network exchange model are combined and saved as a target open neural network exchange model file.
[0015] Optionally, determining the nodes and edges of the target open neural network exchange model based on the nodes and edges in the connected subgraph includes: determining the input and output edges of the connected subgraph based on the nodes and edges in the connected subgraph; determining the input and output in the target open neural network exchange model based on the input and output edges of the connected subgraph; determining the nodes in the target open neural network exchange model based on the node names in the connected subgraph; and determining the names of the edges in the target open neural network exchange model based on the attributes of the edges between nodes in the connected subgraph.
[0016] In some embodiments, the tiling apparatus for an open neural network exchange model includes a processor and a memory storing program instructions, the processor being configured to execute the tiling method for an open neural network exchange model as described above when the program instructions are executed.
[0017] In some embodiments, the electronic device includes: an electronic device body; and a slicing device for an open neural network exchange model, as described above, mounted on the electronic device body.
[0018] The image slicing method, apparatus, and electronic device for open neural network exchange models provided in this disclosure can achieve the following technical effects: In this embodiment, the implicit graph in ONNX is converted into an explicit directed graph, resulting in a clear and unambiguous graph structure and node connectivity. Each node and edge in the directed graph is clearly defined, facilitating graph analysis and partitioning operations. By accurately identifying the boundaries of each connected subgraph, the explicit directed graph can be partitioned into multiple connected subgraphs, ensuring that each connected subgraph is an independent and complete computational unit, thus improving the accuracy of the partitioning. Finally, the partitioned connected subgraphs are converted into multiple target open neural network exchange model files, achieving precise ONNX graph partitioning.
[0019] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description
[0020] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein: Figure 1 This is a schematic diagram of a slicing method for an open neural network exchange model provided in an embodiment of this disclosure; Figure 2 This is an implicit diagram of ONNX provided in an embodiment of the present disclosure; Figure 3 This is an explicit directed graph based on networkx provided in an embodiment of the present disclosure; Figure 4 This is a schematic diagram of dividing a directed graph into multiple connected subgraphs according to an embodiment of the present disclosure; Figure 5 This is another schematic diagram of dividing a directed graph into multiple connected subgraphs according to an embodiment of the present disclosure; Figure 6 This is another schematic diagram of dividing a directed graph into multiple connected subgraphs according to an embodiment of the present disclosure; Figure 7 This is another schematic diagram of dividing a directed graph into multiple connected subgraphs according to an embodiment of the present disclosure; Figure 8 This is another schematic diagram of dividing a directed graph into multiple connected subgraphs according to an embodiment of the present disclosure; Figure 9 This is a schematic diagram of a slicing device for an open neural network exchange model provided in an embodiment of this disclosure. Detailed Implementation
[0021] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.
[0022] The terms "first," "second," etc., used in the technical solutions described in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.
[0023] Unless otherwise stated, the term "multiple" means two or more.
[0024] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.
[0025] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.
[0026] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.
[0027] Combination Figure 1 As shown, this disclosure provides a graph slicing method for an open neural network exchange model. The execution entity of this graph slicing method can be a processor, and the graph slicing method includes: S101, the processor converts the implicit graph in the open neural network exchange model into an explicit directed graph.
[0028] S102, the processor divides the directed graph into multiple connected subgraphs.
[0029] S103, the processor converts multiple connected subgraphs into multiple target open neural network exchange model files respectively.
[0030] In this embodiment, the implicit graph in ONNX is converted into an explicit directed graph, resulting in a clear and unambiguous graph structure and node connectivity. Each node and edge in the directed graph is clearly defined, facilitating graph analysis and partitioning operations. By accurately identifying the boundaries of each connected subgraph, the explicit directed graph can be partitioned into multiple connected subgraphs, ensuring that each connected subgraph is an independent and complete computational unit, thus improving the accuracy of the partitioning. Finally, the partitioned connected subgraphs are converted into multiple target open neural network exchange model files, achieving precise ONNX graph partitioning.
[0031] Optionally, all operations related to the ONNX model in this embodiment can be implemented using Python's ONNX library. The ONNX library provides a rich API, making it convenient to operate and process the ONNX model.
[0032] Optionally, operations related to the ONNX model include: loading the ONNX model from a file; parsing the computation graph of the ONNX model and extracting node and edge information; performing graph slicing operations on the directed graph as needed, such as adding or deleting nodes; and saving the sliced connected subgraph as a new ONNX file.
[0033] Optionally, implicit graphs are the default representation for ONNX models. In an implicit graph, each node contains the names of its own input and output tensors, centered around the nodes, and edges are implicitly defined through the node's input and output tensors.
[0034] Alternatively, explicit directed graphs include Digraph (directed graph) based on NetworkX. The graph structure and connections between nodes in Digraph are clear and explicit, with each node and edge having a well-defined structure, facilitating graph operations and analysis.
[0035] Optionally, converting the implicit graph in the open neural network exchange model into an explicit directed graph includes: constructing a first inverted dictionary and a second inverted dictionary based on the nodes in the implicit graph; the first inverted dictionary is a dictionary from node input names to node names, and the second inverted dictionary is a dictionary from node output names to node names; converting the inputs and outputs in the implicit graph into pseudo-nodes of the directed graph; and converting the nodes and edges in the implicit graph into nodes and edges of the directed graph based on the first inverted dictionary and the second inverted dictionary.
[0036] In this embodiment, constructing a first and a second inverted dictionary enables rapid lookup of node inputs and outputs, thus accurately establishing connections between nodes during the transformation process. Converting an implicit graph into an explicit directed graph makes the graph structure clearer and more explicit, with each node and edge clearly defined, facilitating graph operations and analysis. Explicit graphs also reduce errors caused by ambiguous node relationships, improving the accuracy and reliability of graph operations.
[0037] Optionally, pseudo-nodes are used to represent the start and end points of a directed graph, making the structure of the directed graph more complete. For example... Figure 3 As shown, in the directed graph, two pseudo-nodes, input and output, are defined.
[0038] Optionally, the nodes and edges in the implicit graph are converted into nodes and edges in the directed graph based on the first and second inverted dictionaries, including: traversing each node in the implicit graph and traversing the input edges and output edges of each node; finding the relevant nodes corresponding to each input edge and output edge based on the first and second inverted dictionaries, and determining the names of the relevant nodes for each input edge and output edge; constructing the nodes and edges of the directed graph based on the names of the relevant nodes for each input edge and output edge; and defining the attributes of the edges in the directed graph as the names of the edges in the corresponding implicit graph based on the correspondence between the edges in the implicit graph and the edges in the directed graph.
[0039] In this embodiment, each node in the implicit graph contains input edges and output edges. Input edges represent data flowing to that node, and output edges represent data flowing out of that node. For example... Figure 2 As shown, for node op1, the input edge is e1, and the output edge is e2. For each input edge, the source node of the input edge is found using the first inverted dictionary. For each output edge, the target node of the output edge is found using the second inverted dictionary. The inverted dictionary allows for quick determination of the start and end points of each edge, thus clarifying the connection relationships between nodes. In a directed graph, each node corresponds to an operation node in the implicit graph. Based on the names of the related nodes of the input and output edges, edges in the directed graph are constructed, with each edge connecting two nodes, representing the direction of data flow. Furthermore, in a directed graph, each edge can have one or more attributes. Based on the correspondence between edges in the implicit graph and edges in the directed graph, the attributes of edges in the directed graph are defined as the names of the edges in the corresponding implicit graph, thus preserving the original information of the edges in the implicit graph for subsequent analysis and operations. For example... Figure 2 and Figure 3 As shown, the properties of the edge (op2, op4) in the directed graph are defined as the name e3 of the edge in the corresponding implicit graph.
[0040] Optionally, based on the correspondence between the edges in the implicit graph and the edges in the directed graph, the attributes of the edges in the directed graph are defined as the names of the edges in the corresponding implicit graph. This includes: when the output of a target operator in the implicit graph is given to multiple subsequent nodes, multiple edges are defined in the directed graph, with each edge having two nodes: the target operator and a subsequent node, and the attribute of each edge is defined as the name of the output.
[0041] In this embodiment, for cases where an edge in ONNX has multiple users, it needs to be equivalently transformed into multiple edges in the directed graph. For example... Figure 2As shown, in the implicit graph of ONNX, suppose node op1 is a node with a single input e1 and a single output e2. Its output e2 is used by nodes op2 and op3 simultaneously, which is represented by an edge between op1 and op2, and also an edge between op1 and op3. In ONNX, this edge only has the same name e2. However, in... Figure 3 In a directed graph, after converting the nodes in the implicit graph to nodes in the directed graph, since a directed graph requires two nodes to represent an edge, the same edge in the implicit graph becomes two edges in the directed graph. At this time, the name e2 of the edge in the implicit graph needs to be used as an attribute on the edges (op1, op2) and (op1, op3) in the directed graph.
[0042] exist Figure 2 The implicit graph shown includes four nodes: op1, op2, op3, and op4, and five edges: e1, e2, e3, e4, and e5. In the... Figure 2 Implicit graph transformation Figure 3 In the process of constructing a directed graph, firstly, an inverted dictionary needs to be constructed based on the nodes in the implicit graph. Then, pseudo-nodes input and output of the directed graph are constructed based on the input and output of the implicit graph. Next, based on the nodes in the implicit graph, corresponding nodes are constructed in the directed graph: op1, op2, op3, op4. And based on the input and output edges of each node in the implicit graph, corresponding edges are constructed in the directed graph: (input, op1), (op1, op2), (op1, op3), (op2, op4), (op3, op4), (op4, output). Finally, the attributes of the edges in the directed graph are defined according to the names of the edges in the implicit graph: edge (input, op1) has attribute e1, edge (op1, op2) has attribute e2, edge (op1, op3) has attribute e2, edge (op2, op4) has attribute e3, edge (op3, op4) has attribute e4, and edge (op4, output) has attribute e5. This ultimately achieves the goal of... Figure 2 Implicit graph conversion in Figure 3 A directed graph in [the context of a graph].
[0043] Optionally, the directed graph can be divided into multiple connected subgraphs, including: dividing the directed graph into a preset number of node sets based on a community detection algorithm, and obtaining a preset number of connected subgraphs based on the node sets; or, dividing the directed graph based on the cut vertices and biconnected components of the directed graph to obtain multiple connected subgraphs; or, forming multiple connected subgraphs based on all ancestor nodes of the target node; or, dividing the directed graph based on the plug-in type nodes in the directed graph to obtain multiple connected subgraphs; or, dividing the directed graph based on the marked nodes in the directed graph to obtain multiple connected subgraphs.
[0044] This embodiment employs multiple methods to segment the directed graph, enabling more flexible and accurate slicing operations based on different needs and scenarios, thus improving the flexibility of graph slicing. Different slicing methods are suitable for different application scenarios; for example, community detection is suitable for modular analysis, while cut vertex detection is suitable for critical path analysis. By identifying cut vertices, ancestor nodes, plug-in nodes, and marker nodes, the boundaries of the slice can be clearly defined, avoiding slicing errors caused by unclear boundaries and ensuring that each subgraph is an independent and complete computational unit. Furthermore, multiple slicing methods can be used in combination during the slicing process. For example, after the directed graph is segmented into a preset number of node sets based on the community detection algorithm, the segmented node sets can be further segmented based on cut vertices and biconnected components.
[0045] Optionally, based on a community detection algorithm, the directed graph is divided into a preset number of node sets, and a preset number of connected subgraphs are obtained based on the node sets, including: using the async_fluidc community detection algorithm built into networkx to divide the directed graph into a preset number of subgraphs.
[0046] In this embodiment, the asyn_fluidc algorithm is a fluid diffusion-based community detection algorithm implemented in the NetworkX library. It belongs to the asynchronous fluid community detection algorithm family and is used to identify community structures in complex networks. By inputting a directed graph and a preset number of connected subgraphs to be divided into, the asyn_fluidc algorithm returns the set of node names for each community. Each community's set of node names can form a connected subgraph. For example... Figure 4 As shown, after inputting a directed graph and specifying a preset number of connected subgraphs as 2, the asyn_fluidc algorithm can divide the directed graph into two parts. Each part contains a set of nodes corresponding to a connected subgraph. For example, the first set of nodes is {op1, op2, op3, op4}, and the second set of nodes is {op0, op5, op6, op7}. Using the asyn_fluidc algorithm, the number of connected subgraphs can be specified. For ONNX objects that Netron cannot open due to too many operators, they can be easily opened separately using Netron after partitioning.
[0047] Optionally, the directed graph is divided according to the cut vertices and biconnected components to obtain multiple connected subgraphs, including: determining the cut vertices and biconnected components in the directed graph based on a preset function; after removing the cut vertices from the biconnected components, dividing the directed graph into connected subgraphs corresponding to the biconnected components after the cut vertices are removed, and the cut vertices; or, dividing the directed graph into connected subgraphs corresponding to the biconnected components, wherein adjacent biconnected components have the same cut vertices.
[0048] In this embodiment, based on the block cut vertex tree, a directed graph can be represented as a special tree-like data structure of an undirected graph, and the graph is decomposed into biconnected components and cut vertices. In a specific embodiment, the `nx.articulation_points()` function can be used to find all cut vertices, and then `nx.biconnected_components()` can be used to find all biconnected components. Then, by removing the cut vertices from the biconnected components, multiple connected subgraphs and cut vertices can be obtained. For example... Figure 5 As shown, after inputting a directed graph, two biconnected components are obtained: {op0, op1, op2, op3, op4, op5, op6} and {op6, op7}, along with the cut vertex op6. At this point, the cut vertex can be removed from the biconnected components to obtain two connected subgraphs: {op0, op1, op2, op3, op4, op5} and {op7}. The cut vertex can also be considered as a single connected subgraph: {op6}. Alternatively, two connected subgraphs can be directly generated: {op0, op1, op2, op3, op4, op5, op6} and {op6, op7}, showing that the two connected subgraphs share the cut vertex op6. Furthermore, the biconnected components can be merged into a single node, which can be used to simplify the graph structure. Using a block cut vertex tree, the biconnected components can be merged into a single node while retaining the cut vertex. Opening the merged ONNX file using Netron allows you to view the logical relationships of the simplified graph and identify the cut vertex, thus confirming key operators. Alternatively, you can further use a dominance tree to view the dominance relationships between operators.
[0049] Optionally, multiple connected subgraphs are formed based on all ancestor nodes of the target node, including: after determining the target node, determining the parent node of the target node, and sequentially determining the previous parent node of all parent nodes to obtain all ancestor nodes of the target node; and treating all ancestor nodes of the target node as a connected subgraph.
[0050] In this embodiment, the `nx.ancestor()` function in `networkx` can be used to determine all ancestor nodes of the target node. For example... Figure 6 As shown, for the target node op4 in the directed graph, we can determine that the parent nodes of op4 are op2 and op3, the parent nodes of op2 and op3 are op1, and the parent node of op1 is op0. Thus, we can determine that the set of ancestor nodes of op4 is {op0, op1, op2, op3}.
[0051] Optionally, the target node includes the error node. By determining all ancestor nodes of the error node, nodes that do not need to be considered in the error checking can be excluded, and the dependencies of the error node can be accurately traced back based on the connected subgraph determined by the error node.
[0052] Optionally, the directed graph can be divided according to the nodes of the plug-in type in the directed graph to obtain multiple connected subgraphs, including: deleting all plug-in type nodes in the directed graph; and finding all connected components in the remaining directed graph to obtain multiple connected subgraphs.
[0053] In this embodiment, such as Figure 7 As shown, for the plugin-type nodes op4 and op5 in the directed graph, deleting op4 and op5 from the directed graph forms two connected subgraphs: {op0, op1, op2, op3} and {op6, op7}. By deleting the plugin-type nodes, the influence of plugin operators can be eliminated, making it easier to quickly run ONNX using ONNXRuntime without needing to register plugin operators in ONNXRuntime.
[0054] Optionally, the directed graph can be divided according to the marked nodes in the directed graph to obtain multiple connected subgraphs, including: deleting all marked nodes in the directed graph; and searching for all connected components in the remaining directed graph to obtain multiple connected subgraphs.
[0055] In this embodiment, such as Figure 8 As shown, for a labeled node M in a directed graph, removing M from the directed graph can form four connected subgraphs: {op0}, {op1, op2, op3, op4}, {op5}, and {op6, op7}.
[0056] Optionally, labeled nodes are determined as follows: when deriving the open neural network exchange model, the edges that need to be labeled in the implicit graph are labeled using the forward propagation function; when converting the implicit graph into a directed graph, the labeled edges are converted into labeled nodes.
[0057] In this embodiment, a class named `mark` is defined in PyTorch, inheriting from `torch.autograd.Function`. Its `forward` function takes a tensor as input and returns it as is. However, it is defined as a symbolic function used to customize the names of operators exported in ONNX. Each tensor in the `forward` function of the ONNX model submodule that needs to be sliced (i.e., the edges in the implicit graph) passes through the `mark` module, calling each tensor and returning it as is. This allows PyTorch to track the marked custom operators when exporting ONNX. This ensures that the PyTorch model functional code corresponds to a subset of ONNX operators, which is beneficial for subsequent work such as submodule latency analysis, TensorRT error localization, and model quantization.
[0058] Optionally, the connected subgraph is converted into a target open neural network exchange model file in the following manner: after removing pseudo-nodes from the connected subgraph, the nodes and edges of the target open neural network exchange model are determined based on the nodes and edges in the connected subgraph; the graph structure and model structure of the target open neural network exchange model are determined based on the metadata information of the implicit graph; and the nodes, edges, graph structure, and model structure of the target open neural network exchange model are combined and saved as a target open neural network exchange model file.
[0059] In this embodiment, pseudo-nodes in the directed graph represent the graph's input and output. These pseudo-nodes need to be removed when converting the connected subgraph to an ONNX model. Extracting the nodes and edges of the target ONNX model from the nodes and edges in the connected subgraph clarifies the model's computational structure, ensuring its integrity and accuracy. Metadata information from the implicit graph includes model version, training parameters, etc. Combining this metadata ensures that the graph structure and model structure of the target ONNX model are consistent with the original model, while preserving important model information. Finally, the extracted nodes, edges, graph structure, and model structure are combined to construct a complete ONNX model file, thus converting the connected subgraph into an independent, portable model file for easy deployment and use.
[0060] Optionally, determining the nodes and edges of the target open neural network exchange model based on the nodes and edges in the connected subgraph includes: determining the input and output edges of the connected subgraph based on the nodes and edges in the connected subgraph; determining the input and output in the target open neural network exchange model based on the input and output edges of the connected subgraph; determining the nodes in the target open neural network exchange model based on the node names in the connected subgraph; and determining the names of the edges in the target open neural network exchange model based on the attributes of the edges between nodes in the connected subgraph.
[0061] In this embodiment, after obtaining the connected subgraph, the nodes in ONNX are first determined based on the set of node names in the connected subgraph. Then, in the directed graph, based on the set of node names in the connected subgraph, all edges in the directed graph are traversed, and edges whose starting node name is not in the set of node names but whose ending node name is in the set of node names are used as inputs to ONNX. Similarly, edges whose starting node name is in the set of node names but whose ending node name is not in the set of node names are used as outputs to ONNX. Based on the input and output edges of the connected subgraph, the corresponding edge metadata information is found in ONNX and used as the input and output of the subgraph ONNX. Next, based on the edges within the set of node names, the corresponding edge information (valueinfo and initializer) is found in ONNX. Then, the pseudo-nodes of the original ONNX in the set of node names are removed. Finally, based on the input and output valueinfo, the valueinfo and initializer within the set of node names, the implicit graph of the subgraph ONNX is constructed, and the graph structure information of the original ONNX is added to construct the model structure, which is then saved as an ONNX file. This means that the ONNX file converts an explicit directed graph into an implicit graph representation.
[0062] The graph slicing method for open neural network exchange models provided in this disclosure not only meets diverse user needs for graph slicing but also provides convenient troubleshooting tools for subsequent deployment processes. Finally, it opens up specific operational space for ONNX processing, possessing significant potential for further development. By converting the implicit graph of ONNX into a directed graph of NetworkX, the advantages of NetworkX or graph theory algorithms can be leveraged to improve the ability to process ONNX implicit graphs, particularly the ability to slice ONNX graphs, introducing various convenient slicing methods. Furthermore, the embodiments of this disclosure also possess strong versatility and practicality, providing a wealth of fundamental means to solve various problems in vehicle-side model deployment.
[0063] Combination Figure 9 As shown, this disclosure provides a slicing device 900 for an open neural network exchange model, including a processor 901 and a memory 902. Optionally, the device may further include a communication interface 903 and a bus 904. The processor 901, communication interface 903, and memory 902 can communicate with each other via the bus 904. The communication interface 903 can be used for information transmission. The processor 901 can call logical instructions in the memory 902 to execute the slicing method for an open neural network exchange model described in the above embodiment.
[0064] Furthermore, the logic instructions in the aforementioned memory 902 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.
[0065] The memory 902, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 901 executes functional applications and data processing by running the program instructions / modules stored in the memory 902, that is, it implements the slicing method for the open neural network exchange model in the above embodiments.
[0066] The memory 902 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 902 may include high-speed random access memory and may also include non-volatile memory.
[0067] This disclosure provides an electronic device, including: an electronic device body, and the aforementioned slicing device for an open neural network exchange model. The slicing device for the open neural network exchange model is mounted on the electronic device body. The mounting relationship described herein is not limited to placement within the electronic device, but also includes mounting connections with other components of the electronic device, including but not limited to physical connections, electrical connections, or signal transmission connections. Those skilled in the art will understand that the slicing device for the open neural network exchange model can be adapted to feasible electronic device bodies to achieve other feasible embodiments.
[0068] This disclosure provides a computer-readable storage medium storing computer-executable instructions configured to execute the above-described slicing method for an open neural network exchange model.
[0069] This disclosure provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions that, when executed by a computer, cause the computer to perform the above-described slicing method for an open neural network exchange model.
[0070] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.
[0071] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media capable of storing program code; it can also be a transient storage medium.
[0072] The foregoing description and accompanying drawings fully illustrate embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included or substituted for parts and features of other embodiments. The scope of the embodiments of this disclosure includes the entire scope of the claims and all available equivalents of the claims. While the terms “first,” “second,” etc., may be used in this application to describe elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element may be called a second element without changing the meaning of the description, and similarly, a second element may be called a first element, provided that all occurrences of “first element” are consistently renamed and all occurrences of “second element” are consistently renamed. First and second elements are both elements, but may not be the same element. Moreover, the terminology used in this application is only for describing embodiments and is not intended to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to also include the plural forms. Similarly, the term “and / or” as used herein means including one or more of the associated listed any and all possible combinations. Additionally, when used herein, the terms “comprise” and its variations “comprises” and / or “comprising” refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase “comprising an…” does not exclude the presence of additional identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.
[0073] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0074] The methods and products disclosed in the embodiments herein (including but not limited to devices and equipment) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0075] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
Claims
1. A cutout method for an open neural network exchange model, characterized in that, The method comprises the following steps: Converting an implicit graph in an open neural network exchange model into an explicit directed graph; Splitting the directed graph into a plurality of connected subgraphs; Converting the plurality of connected subgraphs into a plurality of target open neural network exchange model files respectively.
2. The cutout method according to claim 1, wherein The method for converting the implicit graph in the open neural network exchange model into the explicit directed graph comprises the following steps: Constructing a first inverted dictionary and a second inverted dictionary according to nodes in the implicit graph; the first inverted dictionary is a dictionary of node input names to node names, and the second inverted dictionary is a dictionary of node output names to node names; Converting inputs and outputs in the implicit graph into pseudo nodes of the directed graph; Converting nodes and edges in the implicit graph into nodes and edges of the directed graph according to the first inverted dictionary and the second inverted dictionary.
3. The cutout method according to claim 2, wherein The method for converting the nodes and edges in the implicit graph into the nodes and edges of the directed graph according to the first inverted dictionary and the second inverted dictionary comprises the following steps: Traversing each node in the implicit graph, and traversing input edges and output edges of each node under each node; Finding relevant nodes of each input edge and output edge according to the first inverted dictionary and the second inverted dictionary, to determine relevant node names of each input edge and output edge; Constructing nodes and edges of the directed graph according to the relevant node names of each input edge and output edge; Defining attributes of edges in the directed graph as names of corresponding edges in the implicit graph according to a corresponding relationship between the edges in the implicit graph and the edges in the directed graph.
4. The method of claim 1, wherein, The method for splitting the directed graph into a plurality of connected subgraphs comprises the following steps: Splitting the directed graph into a preset number of node sets based on a community detection algorithm, and obtaining a preset number of connected subgraphs according to the node sets; or Splitting the directed graph according to cut points and biconnected components of the directed graph to obtain a plurality of connected subgraphs; or Forming a plurality of connected subgraphs according to all ancestor nodes of a target node; or Splitting the directed graph according to plug-in type nodes in the directed graph to obtain a plurality of connected subgraphs; or Splitting the directed graph according to marked nodes in the directed graph to obtain a plurality of connected subgraphs.
5. The method of claim 4, wherein, The method for splitting the directed graph according to the cut points and the biconnected components of the directed graph to obtain a plurality of connected subgraphs comprises the following steps: Determining the cut points and the biconnected components in the directed graph based on a preset function; Splitting the directed graph into connected subgraphs corresponding to the biconnected components after removing the cut points from the biconnected components, and the cut points; or splitting the directed graph into connected subgraphs corresponding to the biconnected components, and adjacent biconnected components having the same cut points.
6. The method of claim 4, wherein, The method for splitting the directed graph according to the plug-in type nodes in the directed graph to obtain a plurality of connected subgraphs comprises the following steps: Deleting all plug-in type nodes in the directed graph; Finding all connected components in the remaining directed graph to obtain a plurality of connected subgraphs.
7. The method of cutting a map according to any one of claims 1 to 6, wherein, The method for converting the connected subgraphs into the target open neural network exchange model files comprises the following steps: After removing the pseudo nodes in the connected subgraphs, determining nodes and edges of the target open neural network exchange model according to the nodes and edges in the connected subgraphs; Determining a graph structure and a model structure of the target open neural network exchange model according to metadata information of the implicit graph; Combining the nodes, the edges, the graph structure and the model structure of the target open neural network exchange model, and saving as a target open neural network exchange model file.
8. The cutout method according to claim 7, wherein According to the nodes and edges in the connected subgraph, nodes and edges of the target open neural network exchange model are determined, including: According to the nodes and edges in the connected subgraph, input edges and output edges of the connected subgraph are determined; According to the input edges and output edges of the connected subgraph, input and output in the target open neural network exchange model are determined; According to the node names of the connected subgraph, nodes in the target open neural network exchange model are determined; According to the attributes of the edges between the nodes in the connected subgraph, the names of the edges in the target open neural network exchange model are determined.
9. A cutout device for an open neural network exchange model, comprising a processor and a memory having stored program instructions, characterized in that, The processor is configured to execute the graph cutting method for the open neural network exchange model as claimed in any one of claims 1 to 8 when running the program instructions.
10. An electronic device, comprising: Comprising: An electronic device body; The graph cutting device for the open neural network exchange model as claimed in claim 9 is installed in the electronic device body.