Segmentation method and device of neural network model

Abstract

The invention discloses a segmentation method and device of a neural network model, and belongs to the technical field of neural network models. The method comprises the steps that a to-be-segmented neural network model is matched with a plurality of structure templates in a knowledge base, a target structure template matched with a target model structure in the neural network model is obtained, the knowledge base further records segmentation strategies of all the structure templates, the segmentation strategies indicate the mode of segmenting the structure templates, and the target structure template is obtained; the segmentation strategies recorded by the knowledge base can obtain optimization benefits, and the model structure and the structure template both comprise a plurality of computing nodes with specified connection relationships; obtaining a segmentation strategy of the target structure template; and based on the segmentation strategy of the target structure template, segmenting the target model structure to obtain a segmented neural network model. According to the method, after the neural network model is segmented, the pressure of caching the data of the computational nodes to the cache can be reduced on the premise of not influencing the performance of the computational nodes in the neural network model.

Description

Technical Field

[0001] The present application relates to the technical field of neural network models, and in particular to a method and device for segmenting a neural network model. Background Art

[0002] Generally, by performing data segmentation on the computing nodes of the neural network model, the amount of data of each data of the segmented computing nodes can be reduced, so that the data of the segmented computing nodes can be cached in the cache, thereby improving the efficiency of data movement. Data segmentation on the computing nodes is to segment the data of the computing nodes in a mathematically equivalent manner, so that the combination of multiple data after segmentation is mathematically equivalent to the data before segmentation.

[0003] Currently, computing devices usually automatically generate partitioning strategies by means of theoretical evaluation, etc. In this method, the computing device needs to evaluate the partitioning cost, determine the partitioning strategy based on the partitioning cost, and automatically generate the number of partitions for each computing node.

[0004] However, since it is very difficult to simulate the performance of computing nodes, there may be a gap between the results of theoretical evaluation and the performance of actual computing nodes, resulting in the problem of low accuracy in this method of determining the splitting strategy. Summary of the invention

[0005] The present application provides a method and device for segmenting a neural network model. The present application can improve the performance of the neural network model. For example, after segmenting the neural network model, the present application can reduce the pressure of caching the data of the computing node to the cache without affecting the performance of the computing node in the neural network model. The technical solution provided by the present application is as follows:

[0006] In the first aspect, the present application provides a method for segmenting a neural network model. The method includes: matching the neural network model to be segmented with multiple structural templates in a knowledge base to obtain a target structural template that matches the target model structure in the neural network model, the knowledge base also records the segmentation strategy of each structural template, the segmentation strategy indicates the way to segment the structural template, the segmentation strategy recorded in the knowledge base can all achieve optimization benefits, the model structure and the structural template both include multiple computing nodes with specified connection relationships; obtaining the segmentation strategy of the target structural template; based on the segmentation strategy of the target structural template, segmenting the target model structure to obtain a segmented neural network model.

[0007] In this segmentation method, since the segmentation strategies recorded in the knowledge base can all obtain optimization benefits, when the target model structure in the neural network model matches the target structure template, the target model structure is segmented based on the segmentation strategy of the target structure template, which ensures that the segmented target model structure can obtain optimization benefits compared to before segmentation, so that segmentation of the neural network model can obtain positive benefits, thereby improving the performance of the neural network model. For example, after the neural network model is segmented, the pressure of caching the data of the computing node to the cache can be reduced without affecting the performance of the computing nodes in the neural network model. In addition, the segmentation process can be realized automatically, which can reduce the workload and debugging difficulty of segmenting the neural network model.

[0008] In one implementation, matching the target model structure with the target structure template includes: a structural feature of the target model structure is the same as a structural feature of the target structure template.

[0009] Optionally, the structural feature includes one or more of the following: a topological feature and a connection feature, wherein the topological feature indicates an execution order between computing nodes, and the connection feature indicates a connection relationship between computing nodes.

[0010] When multiple target model structures in the neural network model are matched with multiple target structure templates in the knowledge base respectively, there may be model structures including the same computing nodes in the multiple target model structures. If there is a model structure including the same computing node in the multiple target model structures, since a computing node can only be segmented according to one segmentation strategy, further processing is required to ensure that there is no model structure including the same computing node in the multiple segmented target model structures. Then, based on the segmentation strategy of the target structure template, the target model structure is segmented, including: when there is no model structure including the same computing node in the multiple target model structures, the target model structure matching the target structure template is segmented based on the segmentation strategy of the multiple target structure templates; when there is a model structure including the same computing node in the multiple target model structures, the multiple target model structures are screened so that there is no model structure including the same computing node in the multiple screened target model structures, and the screened target model structures are segmented based on the segmentation strategy of the target structure template matching the screened target model structure.

[0011] In one implementation, multiple target model structures are screened so that no model structure including the same computing node exists among the screened multiple target model structures, including: combining multiple target model structures to obtain multiple model structure combinations, the model structure combination including some target model structures among the multiple target model structures, and the model structure set does not contain repeated computing nodes; based on the optimization benefits of the segmentation strategies of the multiple target structure templates, respectively obtaining benefits of the multiple model structure combinations; based on the benefits of the multiple model structure combinations, determining a target model structure combination among the multiple model structure combinations, the target model structure in the target model structure combination being the screened target model structure.

[0012] Optionally, before matching the neural network model to be segmented with multiple structural templates, the computing device may optionally split the neural network model into multiple model structures in advance, and then match the multiple model structures with multiple structural templates in the knowledge base. Since in the segmentation method of the neural network model provided in the present application, the segmentation strategy in the knowledge base is mainly used to segment the neural network model. Therefore, the computing device may optionally split the neural network model based on the structural templates in the knowledge base.

[0013] In one implementation, the neural network model is split based on the computing nodes included in the structural template in the knowledge base to obtain multiple model structures of the neural network model, including: among all the computing nodes in the neural network model, the computing nodes that do not conform to the structural features of all the structural templates in the knowledge base are deleted to obtain multiple model structures. When the structural features of any computing node in the neural network model are different from the structural features of all computing nodes in the knowledge base, it means that there is no computing node in the knowledge base that may match the computing node, and there is no need to match the computing node with the computing nodes in the knowledge base, so the computing node can be deleted from the neural network model.

[0014] In another implementation, the computing device obtains the computing nodes of all structural templates in the knowledge base, and compares the computing nodes in the neural network model with the computing nodes of all structural templates, and then deletes the computing nodes that are different from all computing nodes in the knowledge base among all computing nodes in the neural network model to obtain multiple model structures. Since the neural network model is segmented by segmenting the data of the computing nodes in a mathematically equivalent manner, and the computing nodes are used to represent a part of the calculation implemented by the neural network model, the neural network model is actually segmented based on the logic of the calculation implemented by the computing nodes. Then, when the computing logic of any computing node in the neural network model is different from the computing logic of all computing nodes in the knowledge base, it means that there is no computing node in the knowledge base that may match the computing node, and there is no need to match the computing node with the computing nodes in the knowledge base, and the computing node can be deleted from the neural network model.

[0015] Optionally, before matching multiple model structures with the structural templates in the knowledge base, the computing device may also screen the multiple model structures, and then match the screened model structures with the multiple structural templates in the knowledge base. By screening the model structures, the total number of model features that need to be matched with the structural templates can be reduced, thereby reducing the total time spent matching the neural network model to be segmented with the structural templates in the knowledge base, and speeding up the segmentation of the neural network model. In one implementation, the method further includes: screening multiple model structures based on the number of computing nodes included in the structural templates in the knowledge base. Then matching multiple model structures with the structural templates in the knowledge base includes: matching the screened model structures with the structural templates in the knowledge base.

[0016] In the present application, before matching the neural network model to be segmented with multiple structural templates in the knowledge base, the method also includes: obtaining the structural template based on the segmentation result of the template neural network model; and creating a knowledge base based on the structural template and its segmentation strategy.

[0017] According to the operation logic of the neural network model, data will be transferred between the computationally related computing nodes. When the computing node is a split computing node, the data originally provided to the computing node will still provide data to the multiple computing nodes obtained by splitting the computing node, and the computing node that originally received the data of the computing node will still receive the data provided by the multiple computing nodes obtained by splitting the computing node. The computing nodes in the same model structure are all related computing nodes, so the number of splits of different computing nodes in the same model structure will be an integer multiple.

[0018] In one implementation, after obtaining multiple model structures, the multiple model structures can be optimized based on the principle of segmentation convergence points and / or segmentation consistent regions, and the optimized model structure is the structural template. The principle based on segmentation convergence points means that the structural template needs to satisfy: all computing nodes except the first computing node (also called the first computing node) and the last computing node in the optimized model structure are segmented. In this way, for multiple computing nodes in any optimized model structure, except the first computing node and the last computing node, the remaining computing nodes are all computing nodes with reference significance for segmentation, so the model structure can be determined as a structural template for reference when segmenting other model structures. Even the first computing node and the last computing node are also segmented.

[0019] The principle based on the consistent region of segmentation means that the optimized model structure needs to meet the following requirements: the number of segments of different computing nodes in the optimized model structure is in an integer multiple relationship. According to the operation logic of the neural network model, there will be data transmission between computing nodes that are related in operation. When a computing node is a segmented computing node, the data originally provided to the computing node will still provide data to multiple computing nodes obtained by segmenting the computing node, and the computing node that originally received the data of the computing node will still receive the data provided by multiple computing nodes obtained by segmenting the computing node. The computing nodes in the same model structure are all related computing nodes, so the number of segments of different computing nodes in the same model structure will be in an integer multiple relationship.

[0020] In the second aspect, the present application provides a neural network model segmentation device. The device includes: a matching module, which is used to match the neural network model to be segmented with multiple structural templates in the knowledge base, and obtain a target structural template that matches the target model structure in the neural network model. The knowledge base also records the segmentation strategy of each structural template. The segmentation strategy indicates the way to segment the structural template. The segmentation strategies recorded in the knowledge base can all achieve optimization benefits. The model structure and the structural template both include multiple computing nodes with specified connection relationships; a first acquisition module, which is used to obtain the segmentation strategy of the target structural template; a segmentation module, which is used to segment the target model structure based on the segmentation strategy of the target structural template to obtain a segmented neural network model.

[0021] Optionally, matching the target model structure with the target structure template includes: a structural feature of the target model structure is the same as a structural feature of the target structure template.

[0022] Optionally, the structural feature includes one or more of the following: a topological feature and a connection feature, wherein the topological feature indicates an execution order between computing nodes, and the connection feature indicates a connection relationship between computing nodes.

[0023] Optionally, when multiple target model structures in the neural network model are respectively matched with multiple target structure templates in the knowledge base, the segmentation module is specifically used to: when there is no model structure including the same computing node among the multiple target model structures, segment the target model structures matching the target structure template based on the segmentation strategies of the multiple target structure templates; when there is a model structure including the same computing node among the multiple target model structures, screen the multiple target model structures so that there is no model structure including the same computing node among the screened multiple target model structures, and segment the screened target model structures based on the segmentation strategies of the target structure templates matching the screened target model structures.

[0024] Optionally, the segmentation module is specifically used to: combine multiple target model structures to obtain multiple model structure combinations, the model structure combination includes some target model structures in the multiple target model structures, and the model structure set does not contain repeated computing nodes; based on the optimization benefits of the segmentation strategy of multiple target structure templates, respectively obtain the benefits of the multiple model structure combinations; based on the benefits of the multiple model structure combinations, determine the target model structure combination in the multiple model structure combinations, the target model structure in the target model structure combination is a screened target model structure.

[0025] Optionally, the matching module is specifically used to: split the neural network model based on the computing nodes included in the structural template in the knowledge base to obtain multiple model structures of the neural network model; and match the multiple model structures with the structural template in the knowledge base.

[0026] Optionally, the matching module is specifically used to: delete the computing nodes that do not conform to the structural features of all the structural templates in the knowledge base among all the computing nodes of the neural network model, so as to obtain multiple model structures.

[0027] Optionally, the matching module is specifically used to: screen multiple model structures based on the number of computing nodes included in the structural template in the knowledge base; and match the screened model structure with the structural template in the knowledge base.

[0028] Optionally, the device further comprises: a second acquisition module and a creation module. The second acquisition module is used to acquire the structure template based on the segmentation result of the template neural network model. The creation module is used to create a knowledge base based on the structure template and its segmentation strategy.

[0029] Optionally, the structural template satisfies one or more of the following: the number of shares of different computing nodes in the structural template is an integer multiple; and all computing nodes except the first computing node and the last computing node in the structural template are divided.

[0030] Optionally, both the first compute node and the last compute node are split.

[0031] In a third aspect, the present application provides a computing device including a memory and a processor, wherein the memory stores program instructions, and the processor runs the program instructions to execute the method provided in the first aspect of the present application and any possible implementation thereof.

[0032] In a fourth aspect, the present application provides a computing device cluster, including multiple computing devices, the multiple computing devices including multiple processors and multiple memories, the multiple memories storing program instructions, and the multiple processors running the program instructions, so that the computing device cluster executes the method provided in the first aspect of the present application and any possible implementation thereof.

[0033] In a fifth aspect, the present application provides a computer-readable storage medium, which is a non-volatile computer-readable storage medium, and the computer-readable storage medium includes program instructions. When the program instructions are executed on a computing device, the computing device executes the method provided in the first aspect of the present application and any possible implementation thereof.

[0034] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method provided in the first aspect of the present application and any possible implementation thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a schematic diagram of a computational graph of a neural network model provided in an embodiment of the present application;

[0036] Figure 2 This is a method provided by the embodiment of the present application. Figure 1 Schematic diagram of the computational graph after the exponential operation node is split;

[0037] Figure 3 It is a structural schematic diagram of an implementation scenario involved in a neural network model segmentation method provided in an embodiment of the present application;

[0038] Figure 4 It is a structural schematic diagram of an implementation scenario involved in another neural network model segmentation method provided in an embodiment of the present application;

[0039] Figure 5 It is a flow chart of a neural network model segmentation method provided in an embodiment of the present application;

[0040] Figure 6 It is a schematic diagram of a process of matching a model structure with a structural template provided in an embodiment of the present application;

[0041] Figure 7 This is a flow chart of segmenting a target model structure based on a segmentation strategy of a target structure template provided in an embodiment of the present application;

[0042] Figure 8 It is a schematic diagram of a computational graph of a neural network model to be segmented provided in an embodiment of the present application;

[0043] Fig. 9 It is a schematic diagram of a structural template provided in an embodiment of the present application;

[0044] Fig.10 A flowchart of an embodiment of the present application for screening multiple target model structures so that no model structure including the same computing node exists in the screened multiple target model structures;

[0045] Fig.11 It is a schematic diagram of a plurality of target model structures including computing nodes and benefits provided in an embodiment of the present application;

[0046] Fig.12 It is a schematic diagram of a deployment method of a neural network model provided in an embodiment of the present application;

[0047] Fig.13 It is a schematic diagram of another deployment method of a neural network model provided in an embodiment of the present application;

[0048] Fig.14 This is a flowchart of creating a knowledge base provided by an embodiment of the present application;

[0049] Fig.15 It is a schematic diagram of a process of obtaining a structural template provided in an embodiment of the present application;

[0050] Fig.16 It is a process schematic diagram of a neural network model segmentation method provided in an embodiment of the present application;

[0051] Fig.17 It is a schematic diagram of a functional module of a segmentation method for implementing a neural network model provided in an embodiment of the present application;

[0052] Fig.18 is a schematic diagram of a neural network model segmentation device provided in an embodiment of the present application;

[0053] Fig.19 is a schematic diagram of another neural network model segmentation device provided in an embodiment of the present application;

[0054] Fig. 20 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0055] Fig.21It is a structural diagram of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0057] To facilitate understanding, the technology and background involved in the embodiments of the present application are first explained below.

[0058] A neural network model usually includes multiple neuron nodes with a connection relationship. Each neuron node is responsible for one or more computing operations. Such as matrix multiplication, addition and other computing operations. A neural network model can be represented by multiple computing nodes. The function of each computing node can be realized by one or more neuron nodes. A computing node can be regarded as a collection of one or more operations performed on the operation object. A computing node is also called an operator. When a neural network model is run on a computing device, the data required for the computing node operation needs to be cached in the memory. For example, when a neural network model is run on a chip, the data required for the operation of each computing node in the neural network model needs to be cached in the memory. However, when the amount of input data of the computing node is large, the input data cannot be cached in the memory at high speed, resulting in the process of moving the data to the cache becoming a performance bottleneck for the operation of the neural network model, affecting the computing efficiency of the neural network model.

[0059] Generally, by performing data segmentation on the computing nodes of the neural network model, the amount of data of each data of the segmented computing nodes can be reduced, so that the data of the segmented computing nodes can be cached in the cache, thereby improving the efficiency of data movement. Data segmentation on the computing nodes is to segment the data of the computing nodes in a mathematically equivalent manner, so that the combination of multiple data after segmentation is mathematically equivalent to the data before segmentation.

[0060] For example, Figure 1 is an example of a computation graph of a neural network model provided in an embodiment of the present application. Figure 1 As shown, the neural network model includes sequentially connected data nodes (such as Figure 1 Data), exponential operation nodes (such as Figure 1 Exp) and output nodes (such as Figure 1 Output in the table). Data nodes, exponential operation nodes, and output nodes are all computing nodes. Data nodes are used to input data to exponential operation nodes. Exponential operation nodes are used to obtain the exponential of input data with the natural constant e as the base. Output nodes are used to output the calculation results of exponential operation nodes. Figure 2 This is a method provided by the embodiment of the present application. Figure 1The computational graph after the exponential operation nodes are split. Figure 2 As shown, the exponential operation node is split into two, and the input end of each exponential operation node after the split is connected to the output end of the data node, and the output end of the exponential operation node after the split is connected to the input end of the output node. It can be seen that the combination of the input data of the two exponential operation nodes after the split is mathematically equivalent to the input data of the exponential operation node before the split.

[0061] Among them, the computational graph is a representation of the neural network model. The computational graph usually includes multiple graph nodes with a connection relationship. The graph nodes are used to represent the computational nodes in the neural network model. The edges between the graph nodes are used to represent the connection relationship between the computational nodes in the neural network model. When there is a connection relationship between the computational nodes, it means that there is data transfer between the two computational nodes. It should be noted that the neural network model here is represented by a computational graph for only an example. The neural network model can also be represented by other types of graphs, and the embodiments of the present application do not specifically limit it.

[0062] At present, there are two main ways to determine the splitting strategy of computing nodes in neural network models. One is to manually determine the splitting strategy based on expert strategies. The other is that the computing device automatically generates the splitting strategy with the help of theoretical evaluation and other means. In the first way to determine the splitting strategy, humans need to perform customized analysis based on each neural network model, and need to manually modify the script file of the neural network model to split and merge the nodes in the neural network model, so as to ensure that the computing nodes can be mathematically equivalent before and after the split. For example, for a calculation graph, humans can manually specify the number of splits of each graph node and manually control the order of each graph node in the calculation graph, so as to reduce the amount of data and cache the data in the cache. However, the workload of manually specifying each graph node is huge, the operation is cumbersome, and it is easy to make mistakes. In addition, the structure of the network model needs to be frequently modified, and debugging is difficult.

[0063] In the second method of determining the splitting strategy, the computing device needs to evaluate the cost of splitting, determine the splitting strategy based on the cost of splitting, and automatically generate the number of splits for each computing node. However, due to the great difficulty in simulating the performance of computing nodes, there may be a gap between the results of theoretical evaluation and the performance of actual computing nodes. There is no way to guarantee that all neural network models can achieve positive returns compared to before splitting. Some neural network models may even have negative returns due to inaccurate evaluation. At the same time, it is also difficult to simulate the behavior of cache, resulting in the problem of low precision in this method of determining the splitting strategy. Moreover, this impact is often not a linear change in performance for computing nodes of various sizes.

[0064] As can be seen from the above, data segmentation of the neural network model may cause a decrease in the performance of the computing nodes. Therefore, how to reasonably segment the neural network model is a relatively difficult problem to solve.

[0065] To this end, an embodiment of the present application provides a method for segmenting a neural network model. In this method, a computing device first matches the neural network model to be segmented with multiple structural templates in a knowledge base to obtain a target structural template that matches the target model structure in the neural network model; then, a segmentation strategy for the target structural template is obtained; and then, based on the segmentation strategy of the target structural template, the target model structure is segmented to obtain a segmented neural network model. Among them, both the model structure and the structural template include multiple computing nodes with specified connection relationships. The knowledge base records multiple structural templates and the segmentation strategy for each structural template. The segmentation strategy indicates the way to segment the structural template, and the segmentation strategies recorded in the knowledge base can all achieve optimization benefits. Obtaining optimization benefits here refers to obtaining positive benefits.

[0066] Since the segmentation strategies recorded in the knowledge base can all achieve optimization benefits, when the target model structure in the neural network model matches the target structure template, the target model structure is segmented based on the segmentation strategy of the target structure template, ensuring that the segmented target model structure can achieve optimization benefits compared to before segmentation, so that segmentation of the neural network model can achieve positive benefits, thereby improving the performance of the neural network model. For example, after the neural network model is segmented, the pressure of caching the data of the computing nodes to the cache can be reduced without affecting the performance of the computing nodes in the neural network model. In addition, the segmentation process can be automated, which can reduce the workload and debugging difficulty of segmenting the neural network model.

[0067] This article provides a detailed introduction to the technical solution of the present application from multiple perspectives, including implementation scenarios, method flow, hardware devices, and software devices.

[0068] The following first describes the application scenarios of the embodiments of the present application with examples.

[0069] Figure 3 Schematic diagram of a neural network model segmentation method provided in an embodiment of the present application. Figure 3 As shown, the implementation scenario includes: a computing device 10. The computing device 10 can segment the neural network model by executing the segmentation method of the neural network model provided in the embodiment of the present application.

[0070] In one implementation, the segmentation method of the neural network model provided in the embodiment of the present application can be implemented by running an executable program on the computing device 10. For example, the computing device 10 is configured with a processor and a memory, and an executable program for implementing the segmentation method of the neural network model is stored in the memory. The processor executes the executable program, and the computing device 10 can execute the segmentation method of the neural network model. Among them, the processor can be selected as a neural network model processor (neural network processing unit, NPU) or a graphics processing unit (graphics processing unit, GPU). In addition, the executable program of the segmentation method of the neural network model can also be presented in the form of an application installation package. After the application installation package is installed in the computing device 10, the segmentation method of the neural network model can be implemented by running the executable program.

[0071] Optionally, the computing device 10 may be a terminal. The terminal may be a computer, a personal computer, a laptop computer, a mobile phone, a smart phone, a tablet computer, a cloud host, a portable mobile terminal, a multimedia player, an e-book reader, a wearable device, a smart home appliance, an artificial intelligence device, a smart wearable device, a smart vehicle-mounted device, or an Internet of Things device, etc.

[0072] Figure 4 1 is a schematic diagram of another implementation scenario of a segmentation method of a neural network model provided in an embodiment of the present application. Figure 4 As shown, the implementation scenario may include a computing device 10 and a client 20. The client 20 can establish a communication connection with the computing device 10. For example, a communication connection can be established between the client 20 and the computing device 10 via a network. Optionally, the network may be a local area network, the Internet, or other networks, which are not limited in the embodiments of the present application.

[0073] In this Figure 4 In the implementation scenario, the user can interact with the computing device 10 through the client 20. For example, the user can provide a segmentation instruction and a model file of the neural network model to be segmented to the computing device 10 through the client 20. In one implementation, the segmentation instruction can optionally carry the model file of the neural network model to be segmented. Accordingly, the computing device 10 can execute the segmentation method of the neural network model provided in the embodiment of the present application based on the segmentation instruction of the client 20 to segment the neural network model.

[0074] In one implementation, the client 20 can be a computer, a personal computer, a laptop computer, a mobile phone, a smart phone, a tablet computer, a cloud host, a portable mobile terminal, a multimedia player, an e-book reader, a wearable device, a smart home appliance, an artificial intelligence device, a smart wearable device, a smart vehicle-mounted device or an Internet of Things device. The computing device 10 can be a server, or a server cluster consisting of several servers, or a cloud computing service center. Among them, a large number of basic resources owned by a cloud service provider are deployed in the cloud computing service center. For example, computing resources, storage resources, and network resources are deployed in the cloud computing service center. The cloud computing service center can use this large amount of basic resources to implement the segmentation method of the neural network model provided in the embodiment of the present application.

[0075] When the computing device 10 is implemented through a cloud computing service center, the user can access the cloud platform through the client 20, and provide segmentation instructions and the neural network model to be segmented to the computing device 10 through the cloud platform. At this time, the computing device 10 in the cloud platform can provide the user with the function of automatically segmenting the neural network model. Optionally, this function can be abstracted into a segmentation cloud service by the cloud service provider on the cloud platform. After the user purchases the segmentation cloud service on the cloud platform, the cloud platform can use the resources of the cloud computing service center and the neural network model segmentation method provided in the embodiment of the present application to segment the neural network model owned by the user. In addition, the segmentation cloud service can be provided as an independent cloud service or as an additional service to other cloud services. In addition, the cloud platform can be a cloud platform of a central cloud, a cloud platform of an edge cloud, or a cloud platform including a central cloud and an edge cloud, and the embodiment of the present application does not specifically limit it. It should be noted that in Figure 4 In the implementation scenario shown, the computing device 10 can also be implemented through other resource platforms besides the cloud platform, and the embodiments of the present application do not specifically limit it.

[0076] It should be understood that the above content is an exemplary description of the implementation scenario of the segmentation method of the neural network model provided in the embodiment of the present application, and does not constitute a limitation on the implementation scenario of the segmentation method of the neural network model. For example, the idea of ​​solving technical problems by the segmentation method of the neural network model provided in the embodiment of the present application can also be applied to other fields related to the graph. For example, it is applied to the field of chemistry, and the knowledge base of the chemical field records the chemical molecular structure. Based on the idea of ​​the present application, the chemical structure to be identified can be matched with the chemical molecular structure recorded in the knowledge base, and then the identification of the chemical structure can be realized based on the matched chemical molecular structure, thereby accelerating the identification of chemical molecules, etc. It is known to those of ordinary skill in the art that as business needs change, its implementation scenario can be adjusted according to application requirements, and the embodiments of the present application do not list them one by one.

[0077] The following is an explanation of the segmentation method of the neural network model provided in the embodiment of the present application. Figure 3 and Figure 4 The computing device in the Figure 5 As shown, the segmentation method of the neural network model includes the following steps:

[0078] Step 501: split the neural network model based on the computing nodes included in the structural template in the knowledge base to obtain multiple model structures of the neural network model. The knowledge base records multiple structural templates and the segmentation strategy of each structural template. The segmentation strategy indicates the way to segment the structural template. The segmentation strategies recorded in the knowledge base can all achieve optimization benefits. The model structure and the structural template both include multiple computing nodes with specified connection relationships.

[0079] Before matching the neural network model to be segmented with multiple structural templates, the computing device may optionally pre-split the neural network model into multiple model structures, and then match the multiple model structures with multiple structural templates. In the segmentation method of the neural network model provided in the present application, the neural network model is segmented mainly using the segmentation strategy in the knowledge base. Therefore, the computing device may optionally split the neural network model based on the structural template in the knowledge base. The following two splitting methods are used as examples to illustrate the implementation method of the computing device splitting the neural network model based on the structural template:

[0080] In the first implementation, the computing device obtains the computing nodes of all structural templates in the knowledge base, and compares the computing nodes in the neural network model with the computing nodes of all structural templates, and then deletes the computing nodes that do not conform to the structural characteristics of all structural templates in the knowledge base from all computing nodes in the neural network model to obtain multiple model structures. Since all computing nodes in the neural network model are organized in a network, when one or more computing nodes are deleted, the neural network model may be divided into at least two subnets, and such subnets are the model structures obtained by splitting.

[0081] Structural features include one or more of the following: topological features (also called topological order features) and connection features. Topological features indicate the execution order between computing nodes. For example, Figure 1 The topological characteristics of the model structure shown indicate that the data nodes, exponential operation nodes, and output nodes are executed in sequence. The connection characteristics indicate the connection relationship between the calculation nodes. For example, Figure 1The connection characteristics of the model structure shown indicate that: the data node inputs data to the exponential operation node, the exponential operation node inputs data to the output node, and the output node outputs the output result of the entire model structure. When the structural characteristics include connection characteristics, the first implementation method is equivalent to comparing the connection methods of all computing nodes in the neural network model with the connection methods of all computing nodes in the knowledge base, and then deleting the computing nodes whose connection methods in the neural network model are different from the connection methods of all computing nodes in the knowledge base, so as to split the neural network model into multiple model structures. When the structural characteristics of any computing node in the neural network model are different from the structural characteristics of all computing nodes in the knowledge base, it means that there is no computing node in the knowledge base that may match the computing node, and there is no need to match the computing node with the computing nodes in the knowledge base, so the computing node can be deleted from the neural network model.

[0082] In the second implementation, the computing device obtains the computing nodes of all structural templates in the knowledge base, and compares the computing nodes in the neural network model with the computing nodes of all structural templates, and then deletes the computing nodes that are different from all the computing nodes in the knowledge base among all the computing nodes in the neural network model to obtain multiple model structures. The computing nodes in the neural network model are different from the computing nodes in the knowledge base, and optionally the computing nodes in the neural network model and the computing nodes in the knowledge base implement different computing logics. Since the neural network model is segmented by dividing the data of the computing nodes in a mathematically equivalent manner, and the computing nodes are used to represent a part of the calculation implemented by the neural network model, the neural network model is actually segmented based on the logic of the calculation implemented by the computing nodes. Then, when the computing logic of any computing node in the neural network model is different from the computing logic of all computing nodes in the knowledge base, it means that there is no computing node in the knowledge base that may match the computing node, and there is no need to match the computing node with the computing nodes in the knowledge base, and the computing node can be deleted from the neural network model. For example, if Figure 6As shown, the computing device first obtains the computing nodes of all structural templates in the knowledge base to obtain a computing node set, and then the computing device compares the computing nodes in the neural network model to be split with all the computing nodes in the computing node set to determine whether there are computing nodes in the computing node set that are identical to the computing nodes in the neural network model to be split, and then deletes the computing nodes that are different from all the computing nodes in the knowledge base among all the computing nodes in the neural network model to be split, thereby obtaining multiple model structures.

[0083] It should be noted that the above implementation method of measuring whether the computing nodes in the neural network model are the same as the computing nodes in the knowledge base is only an example. Whether the computing nodes in the neural network model are the same as the computing nodes in the knowledge base can also be measured by other methods. The other methods can be optionally decided based on application requirements, and the embodiments of the present application do not list them one by one.

[0084] Optionally, after splitting the neural network model into multiple model structures, the computing device may also filter the multiple model structures, and then match the filtered model structures with multiple structural templates in the knowledge base. By filtering the model structures, the total number of model features that need to be matched with the structural templates can be reduced, thereby reducing the total time spent on matching the neural network model to be split with the structural templates in the knowledge base, and speeding up the segmentation of the neural network model. In one implementation, the computing device may optionally filter multiple model structures based on the number of computing nodes included in the structural templates in the knowledge base. For example, Figure 6 As shown, the computing device obtains the minimum value of the total number of computing nodes included in the structural template in the knowledge base, and obtains the total number of computing nodes included in each model structure. Among multiple model structures, the model structures including the total number of computing nodes less than the minimum value are deleted, and the remaining model structures are the screened model structures. If the total number of computing nodes included in the model structure is less than the minimum value, it means that there is no structural template in the knowledge base that may match the model structure, and there is no need to match the model structure with the structural template in the knowledge base, and the model structure can be deleted.

[0085] For another example, the computing device obtains the total number of computing nodes included in all structural templates in the knowledge base, and obtains the total number of computing nodes included in each model structure. Among multiple model structures, the model structures whose total number of computing nodes included is different from the total number of computing nodes included in any structural template in the knowledge base are deleted, and the remaining model structures are the screened model structures. If the total number of computing nodes included in the model structure is different from the total number of computing nodes included in any structural template in the knowledge base, it means that there is no structural template in the knowledge base that may match the model structure, and there is no need to match the model structure with the structural template in the knowledge base, and the model structure can be deleted.

[0086] Furthermore, before operating the neural network model, the neural network model can be optionally converted into a computational graph, and the structural template in the knowledge base can also be optionally represented by a computational graph. In this case, the processing of the neural network model based on the structural template is based on the computational graph. For example, Figure 6 As shown, the computing device first obtains the computation graph of each structural template in all structural templates in the knowledge base (hereinafter referred to as the template computation graph), obtains the computation nodes of the structural template according to the template computation graph, thereby obtaining a computation node set containing the computation nodes in all structural templates, and obtains the computation graph of the neural network model to be segmented, obtains all the computation nodes of the neural network model to be segmented, and then the computing device compares the computation nodes in the neural network model to be segmented with all the computation nodes in the computation node set. Among them, the computation graph can be optionally a directed acyclic graph (DAG). In one implementation, the computing device can optionally convert the neural network model into a computation graph according to the in-degree of the computation node. For example, the computing device obtains the in-degree of each computation node, and then divides the computation nodes with the same in-degree into the same layer, obtaining multiple layers, the computation nodes contained in the multiple layers have different in-degrees, and the multiple computation nodes contained in the same layer have the same in-degree. Then, according to the size of the in-degree of the computation nodes contained in the layer, the layer containing the computation nodes with smaller in-degree is sorted before the layer containing the computation nodes with larger in-degree, and the obtained set including multiple layers and having a sequence between the layers is the computation graph. Among them, the in-degree refers to the sum of the number of times a point in a directed graph is the end point of an edge in the graph. It should be understood that the neural network model here is represented by a computational graph, and the implementation method of obtaining the computational graph is an example. The neural network model can also be represented by other types of graphs, and the computational graph can also be obtained by other methods. The embodiments of this application do not specifically limit them.

[0087] It should be noted that step 501 is an optional step. That is, before matching the neural network model with multiple structural templates in the knowledge base, the computing device can optionally perform step 501 to split the neural network model into multiple model structures, and then match the multiple model structures with multiple structural templates in the knowledge base. Alternatively, the computing device may not perform step 501, that is, not split the neural network model into multiple model structures, and then the entire neural network model can be matched with the structural templates in the knowledge base.

[0088] Step 502: Match the neural network model to be segmented with multiple structural templates in the knowledge base to obtain a target structural template that matches the target model structure in the neural network model.

[0089] After the computing device obtains the neural network model, it can match the neural network model with multiple structural templates in the knowledge base to obtain a target structural template that matches the target model structure in the neural network model. The target model structure is any model structure in the neural network model that matches the structural template in the knowledge base. When step 501 is executed, step 502 is to match the multiple model structures of the neural network model to be split with the multiple structural templates in the knowledge base. When step 501 is not executed, step 502 is to match the entire neural network model to be split with the multiple structural templates in the knowledge base. For example, when step 501 splits the neural network model into multiple model structures and screens the multiple model structures, the schematic diagram of the overall process of step 501 and step 502 can be optionally as follows: Figure 6 . After the computing device obtains multiple model structures according to the steps shown in 6, it continues to match the multiple model structures with the multiple structural templates in the knowledge base. If step 501 is not performed, the same computing node may be divided into multiple model structures according to different division methods. When matching the neural network model with the structural template in the knowledge base, the multiple model structures need to be matched with all the structural templates in the knowledge base, and the matching task volume is large. And when the scale of the neural network model increases, the time complexity of matching will increase exponentially. When the aforementioned step 501 is performed, since the objects to be matched in the neural network model are pre-screened before matching the neural network model with the structural template, when matching the neural network model with the structural template in the knowledge base, only the screened model structure needs to be compared with the structural template, which can effectively reduce the time complexity of matching, thereby speeding up the matching speed.

[0090] In one embodiment, the target model structure matches the target structure template, including: the structural features of the target model structure are the same as the structural features of the target structure template. When the structural features include topological features, step 502 is to compare the execution order between the computing nodes included in the model structure with the execution order between the computing nodes included in the structure template. When the execution order between the computing nodes included in the model structure is the same as the execution order between the computing nodes included in the structure template, it is determined that the model structure matches the structure template. When the structural features include connection features, step 502 is to compare the connection relationship between the computing nodes included in the model structure with the connection relationship between the computing nodes included in the structure template, and when the connection relationship between the computing nodes included in the model structure is the same as the connection relationship between the computing nodes included in the structure template, it is determined that the model structure matches the structure template.

[0091] In one implementation, before the computing device matches the model structure with the structure template, it may first use a byte sequence to represent the model structure and the structure template, and then use a byte sequence matching method, such as the KMP algorithm, to match the byte sequence of the model structure with the byte sequence of the structure template.

[0092] In addition, when the neural network model and the structural template in the knowledge base are both represented by a calculation graph including multiple layers, when matching the neural network model to be segmented with any structural template, the hierarchical structural features of the calculation graph of the neural network model can be optionally compared with the hierarchical structural features of the calculation graph of the structural template. When the calculation graph of the neural network model has the hierarchical structural features of the calculation graph of the structural template, the neural network model and the structural template are matched in the previous manner, otherwise there is no need to continue matching. Among them, the hierarchical structural features include: the total number of multiple layers in the calculation graph, the order of each layer in the calculation graph, the number of computing nodes included in each layer, and the in-degree of the computing nodes in the layer. Optionally, the hierarchical structural features of the calculation graph of the neural network model are compared with the hierarchical structural features of the calculation graph of the structural template, including: performing a matching process on each layer of the calculation graph of the neural network model in order from small to large layers. The matching process includes: firstly comparing the computing nodes in the i-th layer (for example, i=1) of the computing graph of the neural network model with all the layers of the computing graph of the structural template. When the computing nodes in the i-th layer match the computing nodes in the j-th layer of the computing graph of the structural template, continue to compare the computing nodes in the i+1-th layer of the computing graph of the neural network model with the computing nodes in the j-th layer to the last layer of the computing graph of the structural template, and so on, until the comparison of each layer of the computing graph of the neural network model is completed. If the computing nodes in each layer of the computing graph of the neural network model can find matching points in the computing graph of the structural template, then the structural template is said to match the neural network model, that is, the computing graph of the neural network model to be processed has the hierarchical structural characteristics of the computing graph of the structural template, otherwise the computing graph of the neural network model to be processed does not have the hierarchical structural characteristics of the computing graph of the structural template. Among them, the layer where the computing node with the smallest in-degree is located has the smallest hierarchical order.

[0093] By performing a preliminary match on the neural network model according to the hierarchical structure features, the matching speed of the neural network model can be accelerated. In addition, since the layers in the calculation graph have an order, the preliminary match can ensure that the execution order of the calculation nodes does not violate the execution order of the calculation nodes in the structure template, which can further ensure the accuracy of the match.

[0094] Step 503: Obtain a segmentation strategy for the target structure template.

[0095] Since the knowledge base records multiple structure templates and the segmentation strategies of each structure template, after obtaining the target structure template that matches the target model structure, the computing device can obtain the segmentation strategy of the target structure template from the knowledge base, so as to segment the target model structure based on the segmentation strategy. The segmentation strategy indicates the way to segment the structure template. The segmentation strategies recorded in the knowledge base can all achieve optimization benefits. For example, suppose Figure 1 is a schematic diagram of the structural template. Figure 1 As shown, the target structure template includes three computing nodes, and its segmentation strategy is: do not segment the first computing node and the third computing node of the target structure template, and divide the second computing node of the target structure template into two computing nodes. In one implementation, the knowledge base records multiple structure templates, multiple segmentation strategies, and the corresponding relationship between multiple structure templates and segmentation strategies, then the computing device can optionally query the corresponding relationship based on the target structure template to obtain the segmentation strategy corresponding to the target structure template.

[0096] Step 504: based on the segmentation strategy of the target structure template, segment the target model structure to obtain a segmented neural network model.

[0097] After the computing device obtains the segmentation strategy of the target structure template, it can segment the target model structure that matches the target structure template based on the segmentation strategy to obtain a segmented neural network model. However, when multiple target model structures in the neural network model are respectively matched with multiple target structure templates in the knowledge base, if there are model structures including the same computing nodes in the multiple target model structures, since a computing node can only be segmented according to one segmentation strategy, further processing is required to ensure that there are no model structures including the same computing nodes in the multiple segmented target model structures. For example, if Figure 7 As shown, the target model structure is segmented based on the segmentation strategy of the target structure template, including:

[0098] Step 5041: When there is no model structure including the same computing node among the multiple target model structures, the target model structures matching the target structure template are segmented based on the segmentation strategies of the multiple target structure templates.

[0099] When there is no model structure including the same computing node among multiple target model structures, the computing device can directly segment the target model structure matching the target structure template according to the segmentation strategy of multiple target structure templates, and then splice the segmented target model structure with the model structure that does not need to be segmented according to the order of each model structure in the neural network model to obtain a segmented neural network model.

[0100] For example, suppose Figure 8 is the computational graph of the neural network model to be segmented, Figure 8 The numbers in the circles are used to identify different computing nodes. Figure 8 As shown, the neural network model includes two computing nodes as inputs, the outputs of the two computing nodes are connected to the same computing node, and the computing node is sequentially connected to four computing nodes. After step 502, it can be obtained that the neural network model includes model structure 1, model structure 2 and a computing node, and model structure 1 and Fig. 9 The structure template shown matches the model structure 2 with Figure 1 The structure template shown is matched. After step 503, Figure 1 The splitting strategy of the structure template shown is: do not split the first computing node and the third computing node of the target structure template, and split the second computing node of the target structure template into two computing nodes. Fig. 9 The splitting strategy of the structure template shown is: split an input node in the structure template into two computing nodes, and the remaining nodes are not split. Since the computing nodes included in the model structure and the model structure 2 are different, the computing device can directly follow Figure 1 and Fig. 9 The segmentation strategy of the structural template shown segments model structure 1 and model structure 2, and then splices the last computing node of model structure 1 with the first computing node of model structure 2, and splices the third computing node of model structure 2 with the last computing node of the neural network model to obtain a segmented neural network model.

[0101] Step 5042: When there are model structures including the same computing nodes among multiple target model structures, the multiple target model structures are screened so that there is no model structure including the same computing nodes among the screened multiple target model structures, and then the screened target model structures are segmented based on the segmentation strategies of the target structure templates that match the screened target model structures.

[0102] When there are model structures including the same computing nodes among multiple target model structures, the computing device may optionally first screen the multiple target model structures so that there are no model structures including the same computing nodes among the screened target model structures, and then segment the screened target model structures according to the segmentation strategy that matches the screened target model structures, and then splice the segmented target model structures with the model structures that do not need to be segmented according to the order of the model structures in the neural network model to obtain a segmented neural network model.

[0103] In one implementation, Fig.10As shown, the implementation process of screening multiple target model structures so that there is no model structure including the same computing node among the screened multiple target model structures includes:

[0104] Step 5042a: Combine multiple target model structures to obtain multiple model structure combinations, any one of which includes some of the target model structures in the multiple target model structures, and the model structure set does not contain duplicate computing nodes.

[0105] The process of combining multiple target model structures by the computing device is actually to screen the target model structure from all model structures matching the structure template, and ensure that the computing nodes included in every two model structures screened into a model structure combination are different. Fig.11 A schematic diagram of multiple target model structures including computational nodes and benefits that match the structure templates in the knowledge base. Fig.11 Each horizontal line in represents a target model structure, and the horizontal axis covered by the horizontal line represents the serial number of the computing nodes included in the target model structure. Fig.11 As shown, there are 5 target model structures Q1 to Q5 in the neural network model that match the structural template in the knowledge base. The target model structure Q1 includes computing nodes 1 to 6. The target model structure Q2 includes computing nodes 3 to 9. The target model structure Q3 includes computing nodes 7 to 11. The target model structure Q4 includes computing nodes 5 to 14. The target model structure Q5 includes computing nodes 11 to 16. According to the principle that any model structure set does not contain repeated computing nodes, the five target model structures are combined to obtain the following model structure combinations: model structure combination 1 includes target model structure Q1 and target model structure Q3, model structure combination 2 includes target model structure Q2 and target model structure Q5, and model structure combination 3 includes target model structure Q4. It can be seen that each model structure combination includes some target model structures in multiple target model structures, and each model structure set does not contain repeated computing nodes.

[0106] Step 5042b: Based on the optimization benefits of the segmentation strategies of the multiple target structure templates, the benefits of the multiple model structure combinations are obtained respectively.

[0107] After obtaining any model structure combination, the computing device obtains the optimization benefits of the target model structures included in the model structure combination, and obtains the benefit of the model structure combination based on the optimization benefits of multiple target model structures included in the model structure combination. For example, the benefit of the model structure combination can be optionally equal to the sum of the optimization benefits of all target model structures in the model structure combination. The benefit of the target model structure refers to the optimization benefit of the segmentation strategy of the target structure template matched by the target model structure. For example, continue with Fig.11 Take this as an example to illustrate. Fig.11 The vertical axis in the middle represents the optimization benefit that can be obtained by segmenting the target model structure according to the segmentation strategy of the target structure template that matches the target model structure. Fig.11 As shown, the optimization benefits of Q1 to Q5 are 6, 8, 5, 7 and 4 respectively. According to the model structure combination 1 including the target model structure Q1 and the target model structure Q3, the model structure combination 2 including the target model structure Q2 and the target model structure Q5, and the model structure combination 3 including the target model structure Q4, the benefit of the model structure combination 1 is 11, the benefit of the model structure combination 2 is 12, and the benefit of the model structure combination 3 is 7.

[0108] Step 5042c: based on the benefits of the multiple model structure combinations, determine a target model structure combination from the multiple model structure combinations, wherein the target model structure in the target model structure combination is a screened target model structure.

[0109] After the computing device obtains the benefits of multiple model structure combinations, it can select a target model structure combination from the multiple model structure combinations. Since the model structure set does not contain repeated computing nodes, the target model structure in the target model structure combination is a screened target model structure. In one implementation, the computing device can optionally adopt the principle of maximizing total benefits to select a target model structure combination from multiple model structure combinations. That is, the computing device can determine the model structure combination with the largest benefit from the multiple model structure combinations as the target model structure combination. For example, continue with Fig.11 Taking an example to illustrate, since the benefit of model structure combination 1 is 11, the benefit of model structure combination 2 is 12, and the benefit of model structure combination 3 is 7, it can be obtained that model structure combination 2 is the target model structure combination, and the target model structure Q2 and target model structure Q5 are the screened target model structures.

[0110] The process of determining the target model structure combination is essentially to solve the problem of the model structure combination with the highest return matching by using dynamic programming, which can be represented by the following dynamic programming equation.

[0111]

[0112] Where i is the serial number of the model structure, and q[i] is the benefit value of model structure i. dp[i-1] is the benefit of the model structure combination of model structure i-1 that was traversed before the current model structure i when traversing multiple model structures. frt[i] represents the number of the model structure combination with the maximum benefit among the model structure combinations with numbers less than i that do not contain repeated calculation nodes with model structure i. In this way, dp[frt[i]+q[i]] is the benefit of the model structure combination with the maximum benefit among the model structure combinations including model structure i. The dynamic programming equation is actually to compare dp[i-1] and q[i]+dp[frt[i]] for each model structure i under the consideration of the state transition equation, and take the larger one as dp[i], so as to find the structure combination with the maximum benefit through the state equation. It can be seen that determining the target model structure combination through the dynamic programming equation can ensure the maximization of the knowledge base benefit.

[0113] After the neural network model is segmented, the segmented neural network model can be deployed on a computing device for implementation. For example, the segmented neural network model is deployed on a chip for implementation. The deployment methods include at least the following two options: Fig.12 As shown in , the segmented neural network model can be deployed on a chip with a cache. In this case, the cache is used to cache data in the chip during the calculation process. Fig.13 As shown, the segmented neural network model is deployed in multiple chips, and the computing nodes that are segmented into multiple parts are deployed in multiple chips respectively.

[0114] The above-mentioned process of segmenting the neural network model based on the knowledge base is the application process of the knowledge base. In contrast to the application process of the knowledge base, the embodiment of the present application also provides a process of creating a knowledge base. Fig.14 As shown in the figure, the process of creating a knowledge base includes the following steps:

[0115] Step 1401: Obtain a structural template based on the segmentation result of the template neural network model.

[0116] The knowledge base can be created based on the segmentation results of the template neural network model. The template neural network model is obtained by segmentation based on a specified segmentation strategy. For example, the specified segmentation strategy can be obtained based on a search and tuning algorithm. Alternatively, the specified segmentation strategy is obtained based on an artificial expert strategy. Alternatively, the specified segmentation strategy can also be obtained based on other methods, which are not specifically limited in the embodiments of the present application. The computing device can obtain a structural template by extracting the structural features or node features of the template neural network model. Compared with the structural features, the node features also include information reflecting the properties of the computing node itself, such as the computing logic of the computing node. The structural template can be selected as all model structures in the template neural network model. Alternatively, the structural template is a model structure after screening the model structure in the template neural network model. The structural template can be obtained by splitting the template neural network model based on structural features. In addition, in order to ensure the optimization benefits of the segmentation strategy, the segmentation strategies recorded in the knowledge base are all verified strategies that can ensure the optimization benefits. Here, the template neural network model refers to a neural network model that has been segmented and at least part of the model structure in the neural network model is recorded in the knowledge base. The model structure recorded in the knowledge base in the template neural network model is a structural template. The following is an exemplary description of the process of obtaining the structure template:

[0117] For example, after the computing device obtains the segmented neural network model, it can obtain the segmentation results of each computing node in the neural network model, and group the computing nodes in the neural network model according to the segmentation results to obtain multiple model structures, and then optimize the multiple model structures to obtain one or more structural templates. Optionally, the computing device can group the computing nodes in the neural network model based on a disjoint-set data structure. A disjoint-set is a tree-like data structure used to handle the merging and querying of some disjoint sets. When grouping based on a disjoint-set, the tree-like data structure can be used to represent the entire neural network model, and then the computing nodes in the same tree can be divided into the same group, and the computing nodes that are no longer in the same tree can be divided into different groups. By filtering with a disjoint-set, the scale of the computational graph can be reduced.

[0118] In one implementation, after obtaining multiple model structures, the multiple model structures can be optimized based on the principle of segmentation convergence points and / or segmentation consistent regions, and the optimized model structure is the structural template. The principle based on segmentation convergence points means that the structural template needs to satisfy: all computing nodes except the first computing node (also called the first computing node) and the last computing node in the optimized model structure are segmented. In this way, for multiple computing nodes in any optimized model structure, except the first computing node and the last computing node, the remaining computing nodes are all computing nodes with reference significance for segmentation, so the model structure can be determined as a structural template for reference when segmenting other model structures. Even the first computing node and the last computing node are also segmented. Accordingly, the optimization process can be regarded as a process of discarding invalid computing nodes that have not been segmented in the model structure. For example, Fig.15 It is a schematic diagram of a process for obtaining a structural template provided in an embodiment of the present application. Fig.15 The left side of the figure shows the model structure obtained by splitting. In this model structure, the first computing node (i.e., the computing node for executing op type 1), the second computing node (i.e., the computing node for executing op type 2), the last computing node (i.e., the computing node for executing op type 6), and the second-to-last computing node (i.e., the computing node for executing op type 5) are not split, and the third computing node in this model structure is split into two computing nodes. Fig.15 The right side of the figure is a schematic diagram of the optimized model structure, that is, Fig.15 The right side is a schematic diagram of the structural template obtained after optimizing the model structure. Fig.15 It can be obtained that the structural template is compared with Fig.15 The model structure on the left reduces the first computing node and the last computing node that are not split in the model structure.

[0119] The principle based on the consistent region of segmentation means that the optimized model structure needs to meet the following requirements: the number of segments of different computing nodes in the optimized model structure is in an integer multiple relationship. According to the operation logic of the neural network model, there will be data transmission between computing nodes that are related in operation. When the computing node is a segmented computing node, the data originally provided to the computing node will still provide data to multiple computing nodes obtained by segmenting the computing node, and the computing node that originally received the data of the computing node will still receive the data provided by multiple computing nodes obtained by segmenting the computing node. The computing nodes in the same model structure are all related computing nodes. Therefore, the number of segments of different computing nodes in the same model structure will be in an integer multiple relationship. Among them, when optimizing the model structure, it is necessary to consider computing nodes that are not continuously segmented into 1 part. For any computing node that is not segmented into 1 part, check whether the input node of the computing node is segmented into 1 part. If the input node is not segmented into 1 part, divide the input node and the computing node into the same model structure. Similarly, check whether the output node of the computing node is split into 1 part. If the output node is not split into 1 part, divide the output node and the computing node into the same model structure. By analogy, the structural template in the template neural network model can be obtained. For example, in a neural network model, the number of parts of the computing nodes are: [1,1,1,1,1,1,2,2,8,8,8,2,1,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8]. According to the principle of dividing the consistent area, model structure 1 composed of computing nodes with split numbers of 2, 2, 8, 8, 8, 2 respectively, and model structure 2 composed of computing nodes with split numbers of 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8 respectively can be extracted, and model structure 1 and model structure 2 can be determined as structural templates.

[0120] Step 1402: Create a knowledge base based on the structure template and its segmentation strategy.

[0121] After obtaining the structural template, you can continue to obtain the segmentation strategy of the calculation nodes in the structural template to obtain the segmentation strategy of the structural template. Then, based on the structural template and its segmentation strategy, create a knowledge base. Optionally, before recording the structural template in the knowledge base, you can optionally perform a serialization operation on the structural template to convert the structural template into a byte sequence. Correspondingly, before matching based on the structural template, it is also necessary to perform a deserialization operation on the structural template obtained from the knowledge base to restore the byte sequence to the original representation of the structural template. It should be noted that in the application process of the knowledge base, the segmentation results of the neural network model can also be used to enrich the knowledge base, and the process can refer to the process of creating the knowledge base accordingly.

[0122] Fig.16 Schematic diagram of a neural network model segmentation method provided in an embodiment of the present application. Fig.16 As shown, the method includes two stages.

[0123] Phase 1: The phase of establishing knowledge base.

[0124] After the computing device obtains the model file of the template neural network model, it first converts it into a computational graph. Then, the computational graph is split based on the topological features to obtain multiple model structures. Then, multiple model structures are optimized, such as deleting unsplit computing nodes in the model structure to obtain multiple structural templates of the neural network model. And the optimization benefits of the segmentation strategy of each structural template are verified by the verification system to ensure that each structure is a valid structure that is being optimized. For example, the structural templates before and after segmentation are deployed and implemented on the chip respectively, and their respective benefits are obtained. When the benefit of the structural template after segmentation is greater than the benefit before segmentation, it is determined that the structural template is a valid structure that is being optimized and can be added to the knowledge base, otherwise the structural template cannot be added to the knowledge base. Then, a knowledge base is created based on the structural template and its segmentation strategy.

[0125] The second stage: the application stage of the knowledge base.

[0126] In the process of knowledge base application, after obtaining the structure template from the knowledge base, the structure template is matched with the neural network model to be segmented (i.e., the computational graph). When the target model structure in the neural network model matches the target structure template, the segmentation strategy of the target structure template is obtained from the knowledge base, and then the target model structure is segmented based on the segmentation strategy of the target structure template, and other model structures in the neural network model are segmented according to the logic, and then the segmented target model structure and the model structure that does not need to be segmented are spliced ​​according to the order of each model structure in the neural network model to obtain the segmented neural network model. Then, an application that can be deployed on the chip is generated based on the segmented model, and then the application is deployed on the chip.

[0127] In one implementation, the segmentation method of the neural network model provided in the embodiment of the present application can be implemented by multiple functional modules. For example, Fig.17 As shown, the computing device includes a strategy generation module, a model feature extraction module, a knowledge base module, a query matching module and a model segmentation module. The segmentation method of the neural network model is implemented through the collaborative work of these modules.

[0128] The strategy generation module is used to obtain a structural template based on a template neural network model. For example, the strategy generation module determines multiple alternative segmentation strategies for the template neural network model based on a search and tuning algorithm, and then verifies the multiple alternative segmentation strategies, and obtains the optimal segmentation strategy from the multiple alternative segmentation strategies based on the verification results, and provides the optimal segmentation strategy to the model feature extraction module. For example, the strategy for segmenting the template neural network in the above step 1401 is provided by the strategy generation module. Optionally, the neural network model can be a model such as Tensorfiow, pytorch, or other neural network models, which is not specifically limited in the embodiments of the present application.

[0129] The model feature extraction module is used to segment the template neural network model based on the optimal segmentation strategy, and extract the structural template based on the template neural network model before and after segmentation. For example, the structural template is obtained by extracting the structural features or node features of the template neural network model. For example, the process of segmenting the template neural network in the above step 1401 and obtaining the structural module based on the segmentation result is performed by the model feature extraction module.

[0130] The knowledge base module is used to create a knowledge base based on the structure template and its segmentation strategy. For example, the above step 1402 can be performed by the knowledge base module.

[0131] The query matching module is used to obtain the optimal segmentation strategy of the neural network model to be segmented according to the neural network model to be segmented and the knowledge base, and provide the segmentation strategy to the model segmentation module. For example, the above steps 501 to 503 can be performed by the query matching module.

[0132] The model segmentation module is used to segment the neural network model to be segmented based on the segmentation strategy provided by the query matching module. When there are model structures including the same computing nodes in multiple target model structures, the model segmentation module is also used to determine the target model structure combination, and segment the target model structure combination based on the segmentation strategy matched by the target model structure combination. For example, the above step 504 can be performed by the model segmentation module.

[0133] In summary, in the segmentation method of the neural network model provided in the embodiment of the present application, since the segmentation strategies recorded in the knowledge base can all obtain optimization benefits, therefore, when the target model structure in the neural network model matches the target structure template, the target model structure is segmented based on the segmentation strategy of the target structure template, ensuring that the segmented target model structure can obtain optimization benefits compared to before segmentation, so that segmenting the neural network model can obtain positive benefits, thereby improving the performance of the neural network model. For example, after the present application segments the neural network model, it can reduce the pressure of caching the data of the computing node to the cache without affecting the performance of the computing nodes in the neural network model. In addition, the segmentation process can be realized automatically, which can reduce the workload and debugging difficulty of segmenting the neural network model.

[0134] It should be noted that the sequence of steps of the segmentation method of the neural network model provided in the embodiment of the present application can be appropriately adjusted, and the steps can also be increased or decreased accordingly according to the situation. Any technician familiar with the technical field can easily think of a change within the technical scope disclosed in this application, which should be included in the protection scope of this application, so it will not be repeated.

[0135] The following is an example of a virtual device in an embodiment of the present application.

[0136] The above introduces the method for segmenting the neural network model of the embodiment of the present application. Corresponding to the above method, the embodiment of the present application also provides a device for segmenting the neural network model. Fig.18 Schematic diagram of a neural network model segmentation device provided in an embodiment of the present application. Fig.18 The following multiple components are shown: Fig.18 The segmentation device of the neural network model shown can perform the above Figure 5 and Fig.14 It should be understood that the device may include more additional components than those shown or omit some of the components shown, and the embodiments of the present application are not limited to this. Fig.18 As shown, the segmentation device 180 of the neural network model may include:

[0137] Matching module 1801 is used to match the neural network model to be segmented with multiple structural templates in the knowledge base to obtain a target structural template that matches the target model structure in the neural network model. The knowledge base also records the segmentation strategy of each structural template. The segmentation strategy indicates the way to segment the structural template. The segmentation strategies recorded in the knowledge base can all achieve optimization benefits. The model structure and the structural template both include multiple computing nodes with specified connection relationships.

[0138] The first acquisition module 1802 is used to acquire the segmentation strategy of the target structure template.

[0139] The segmentation module 1803 is used to segment the target model structure based on the segmentation strategy of the target structure template to obtain a segmented neural network model.

[0140] Optionally, matching the target model structure with the target structure template includes: a structural feature of the target model structure is the same as a structural feature of the target structure template.

[0141] Optionally, the structural feature includes one or more of the following: a topological feature and a connection feature, wherein the topological feature indicates an execution order between computing nodes, and the connection feature indicates a connection relationship between computing nodes.

[0142] Optionally, when multiple target model structures in the neural network model are respectively matched with multiple target structure templates in the knowledge base, the segmentation module 1803 is specifically used to: when there is no model structure including the same computing node among the multiple target model structures, segment the target model structures matching the target structure template based on the segmentation strategies of the multiple target structure templates; when there is a model structure including the same computing node among the multiple target model structures, filter the multiple target model structures so that there is no model structure including the same computing node among the filtered multiple target model structures, and segment the filtered target model structures based on the segmentation strategies of the target structure templates matching the filtered target model structures.

[0143] Optionally, the segmentation module 1803 is specifically used to: combine multiple target model structures to obtain multiple model structure combinations, the model structure combination includes some target model structures in the multiple target model structures, and the model structure set does not contain repeated computing nodes; based on the optimization benefits of the segmentation strategy of multiple target structure templates, respectively obtain the benefits of the multiple model structure combinations; based on the benefits of the multiple model structure combinations, determine the target model structure combination in the multiple model structure combinations, the target model structure in the target model structure combination is a screened target model structure.

[0144] Optionally, the matching module 1801 is specifically used to: split the neural network model based on the computing nodes included in the structural template in the knowledge base to obtain multiple model structures of the neural network model; and match the multiple model structures with the structural template in the knowledge base.

[0145] Optionally, the matching module 1801 is specifically used to: delete the computing nodes that do not conform to the structural features of all the structural templates in the knowledge base among all the computing nodes of the neural network model, so as to obtain multiple model structures.

[0146] Optionally, the matching module 1801 is specifically used to: screen multiple model structures based on the number of computing nodes included in the structure template in the knowledge base; and match the screened model structure with the structure template in the knowledge base.

[0147] Alternatively, if Fig.19 As shown, the segmentation device 180 of the neural network model also includes:

[0148] The second acquisition module 1804 is used to acquire the structure template based on the segmentation result of the template neural network model.

[0149] The creation module 1805 is used to create a knowledge base based on the structure template and its segmentation strategy.

[0150] Optionally, the structural template satisfies one or more of the following: the number of shares of different computing nodes in the structural template is an integer multiple; and all computing nodes except the first computing node and the last computing node in the structural template are divided.

[0151] Optionally, both the first compute node and the last compute node are split.

[0152] Here, please refer to the description in the previous method embodiment for the detailed working process of the matching module 1801, the first acquisition module 1802, the segmentation module 1803, the second acquisition module 1804 and the creation module 1805. For example, the matching module 1801 matches the neural network model to be segmented with multiple structural templates in the knowledge base in the manner described in step 502 above; the first acquisition module 1802 obtains the segmentation strategy of the target structural template in the manner described in step 502 above; the segmentation module 1803 obtains the segmented neural network model in the manner described in step 504 above; the second acquisition module 1804 obtains the structural template in the manner described in step 1401 above; the creation module 1805 creates the knowledge base in the manner described in step 1402 above. The embodiments of this application will not be described again here.

[0153] In summary, in the segmentation device of the neural network model provided in the embodiment of the present application, the matching module first matches the neural network model to be segmented with multiple structural templates in the knowledge base to obtain a target structural template that matches the target model structure in the neural network model; then, the first acquisition module obtains the segmentation strategy of the target structural template; the segmentation module segments the target model structure based on the segmentation strategy of the target structural template to obtain a segmented neural network model. Among them, the model structure and the structural template both include multiple computing nodes with specified connection relationships. The knowledge base records multiple structural templates and the segmentation strategy of each structural template. The segmentation strategy indicates the way to segment the structural template, and the segmentation strategy recorded in the knowledge base can obtain optimization benefits. Obtaining optimization benefits here means obtaining positive benefits. In addition, the segmentation process can be realized automatically, which can reduce the workload and debugging difficulty of segmenting the neural network model.

[0154] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of each component described above can refer to the corresponding content in the aforementioned method embodiment, and will not be repeated here.

[0155] The following is an illustration of the basic hardware structure involved in the embodiments of the present application.

[0156] The embodiment of the present application provides a computing device. The computing device is used to implement part or all of the functions of the segmentation method of the neural network model provided in the embodiment of the present application. Fig. 20 Schematic diagram of a computing device provided in an embodiment of the present application. Fig. 20 As shown, the computing device 2000 includes a processor 2001, a memory 2002, a communication interface 2003 and a bus 2004. The processor 2001, the memory 2002 and the communication interface 2003 are connected to each other through the bus 2004.

[0157] Processor 2001 may include a general processor and / or a dedicated hardware chip. A general processor may include: a central processing unit (CPU), a microprocessor or a graphics processing unit (GPU). The CPU is, for example, a single-core processor (single-CPU), or a multi-core processor (multi-CPU). A dedicated hardware chip is a hardware module for high-performance processing. A dedicated hardware chip includes at least one of a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or a network processor (NP). Processor 2001 may also be an integrated circuit chip having signal processing capabilities. In the implementation process, some or all of the functions of the segmentation method of the neural network model of the present application may be completed by the hardware integrated logic circuit in processor 2001 or instructions in software form.

[0158] The memory 2002 is used to store computer programs, and the computer programs include an operating system 2002a and executable codes (i.e., program instructions) 2002b. The memory 2002 is, for example, a read-only memory or other types of static storage devices that can store static information and instructions, or a random access memory or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory, a read-only optical disc or other optical disc storage, an optical disc storage (including a compressed optical disc, a laser disc, an optical disc, a digital versatile disc, a blue-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired executable code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. For example, the memory 2002 is used to store an outbound port queue, etc. The memory 2002 is, for example, independent and connected to the processor 2001 via a bus 2004. Or the memory 2002 and the processor 2001 are integrated together. The memory 2002 can store executable code. When the executable code stored in the memory 2002 is executed by the processor 2001, the processor 2001 is used to perform part or all of the functions of the segmentation method of the neural network model provided in the embodiment of the present application. Please refer to the relevant description in the above embodiment for the implementation of the process executed by the processor 2001. The memory 2002 may also include software modules and data required for other running processes such as the operating system.

[0159] The communication interface 2003 uses a transceiver module such as, but not limited to, a transceiver to achieve communication with other devices or communication networks. For example, the communication interface 2003 can be any one or any combination of the following devices: a network interface (such as an Ethernet interface), a wireless network card, and other devices with network access functions.

[0160] The bus 2004 is any type of communication bus for interconnecting the internal devices (e.g., memory 2002, processor 2001, communication interface 2003) of the computing device. For example, a system bus. The embodiment of the present application takes the above-mentioned devices inside the computing device as an example of interconnecting through the bus 2004. Optionally, the above-mentioned devices inside the computing device 2000 can also be connected to each other in communication with each other using other connection methods other than the bus 2004. For example, the above-mentioned devices inside the computing device 2000 are interconnected through an internal logical interface.

[0161] It should be noted that the above-mentioned multiple devices can be respectively arranged on independent chips, or at least partially or completely arranged on the same chip. Whether to independently arrange each device on different chips or to integrate and arrange it on one or more chips often depends on the needs of product design. The embodiments of the present application do not limit the specific implementation form of the above-mentioned devices. The descriptions of the processes corresponding to the above-mentioned figures have different focuses. For the parts not described in detail in a certain process, please refer to the relevant descriptions of other processes.

[0162] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product providing a program development platform includes one or more computer instructions, and when these computer program instructions are loaded and executed on a computing device, the functions of the segmentation method of the neural network model provided in the embodiments of the present application are implemented in whole or in part.

[0163] Furthermore, computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium stores computer program instructions that provide a program development platform.

[0164] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0165] Optionally, the structure of at least one computing device included in the computing device cluster can be found in Fig. 20 The computing device 2000 shown. The memory 2002 in one or more computing devices 2000 in the computing device cluster may store the same instructions for executing the segmentation method of the neural network model.

[0166] In some possible implementations, the memory 2002 of one or more computing devices 2000 in the computing device cluster may also store partial instructions for executing the segmentation method of the neural network model. In other words, the combination of one or more computing devices 2000 can jointly execute the instructions for executing the segmentation method of the neural network model.

[0167] It should be noted that the memory 2002 in different computing devices 2000 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the segmentation device of the neural network model. That is, the instructions stored in the memory 2002 in different computing devices 2000 can realize the functions of one or more modules among the matching module 1801, the first acquisition module 1802, the segmentation module 1803, the second acquisition module 1804 and the creation module 1805.

[0168] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.21 A possible implementation is shown. Fig.21 As shown, two computing devices 2100A and 2100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, computing devices 2100A and 2100B include a bus 2102, a processor 2104, a memory 2106, and a communication interface 2108. Instructions for executing the functions of the matching module 1801, the first acquisition module 1802, and the segmentation module 1803 are stored in the memory 2106 in the computing device 2100A. At the same time, instructions for executing the functions of the second acquisition module 1804 and the creation module 1805 are stored in the memory 2106 in the computing device 2100B.

[0169] It should be understood that Fig.21The function of the computing device 2100A shown in the figure can also be completed by multiple computing devices 2100. Similarly, the function of the computing device 2100B can also be completed by multiple computing devices 2100. And the deployment method of the module for implementing the segmentation method of the neural network model in the computing device can also be adjusted according to the application requirements.

[0170] An embodiment of the present application also provides a computer-readable storage medium, which is a non-volatile computer-readable storage medium. The computer-readable storage medium includes program instructions. When the program instructions are executed on a computing device, the computing device implements a neural network model segmentation method as provided in an embodiment of the present application.

[0171] The embodiment of the present application also provides a computer program product comprising instructions. When the computer program product is executed on a computer, the computer implements the neural network model segmentation method provided in the embodiment of the present application.

[0172] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0173] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the original data and executable code involved in this application are all obtained with full authorization.

[0174] In the embodiments of the present application, the terms "first", "second" and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. The term "at least one" means one or more, and the term "plurality" means two or more, unless otherwise expressly defined.

[0175] The term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0176] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the concept and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A segmentation method for a neural network model, characterized in that: The method comprises: Matching the neural network model to be segmented with a plurality of structural templates in a knowledge base to obtain a target structural template that matches the target model structure in the neural network model, wherein the knowledge base also records segmentation strategies for each structural template, wherein the segmentation strategies indicate a method for segmenting the structural template, and the segmentation strategies recorded in the knowledge base can all achieve optimization benefits, and the model structure and the structural template both include a plurality of computing nodes with a specified connection relationship; Obtaining a segmentation strategy for the target structure template; Based on the segmentation strategy of the target structure template, the target model structure is segmented to obtain a segmented neural network model.

2. The method according to claim 1, characterized in that The matching of the target model structure with the target structure template includes: a structural feature of the target model structure is the same as a structural feature of the target structure template.

3. The method according to claim 2, characterized in that The structural features include one or more of the following: topological features and connection features, wherein the topological features indicate the execution order between computing nodes, and the connection features indicate the connection relationship between computing nodes.

4. The method according to any one of claims 1 to 3, characterized in that: When multiple target model structures in the neural network model are matched with multiple target structure templates in the knowledge base respectively, the segmentation strategy based on the target structure template segments the target model structure, including: When there is no model structure including the same computing node among the multiple target model structures, segmenting the target model structures matching the target structure template based on the segmentation strategies of the multiple target structure templates respectively; When there are model structures including the same computing nodes among the multiple target model structures, the multiple target model structures are screened so that there are no model structures including the same computing nodes among the screened multiple target model structures, and the screened target model structures are segmented based on the segmentation strategies of the target structure templates that match the screened target model structures.

5. The method according to claim 4, characterized in that The screening of the multiple target model structures so that no model structure including the same computing node exists among the screened multiple target model structures comprises: Combining the multiple target model structures to obtain multiple model structure combinations, wherein the model structure combinations include some of the target model structures in the multiple target model structures, and the model structure set does not contain repeated computing nodes; Based on the optimization benefits of the segmentation strategies of the multiple target structure templates, respectively obtaining benefits of the multiple model structure combinations; Based on the benefits of the multiple model structure combinations, a target model structure combination is determined from the multiple model structure combinations, wherein the target model structure in the target model structure combination is a screened target model structure.

6. The method according to any one of claims 1 to 5, characterized in that: Before matching the neural network model to be segmented with a plurality of structural templates in the knowledge base, the method further includes: Splitting the neural network model based on the computing nodes included in the structural template in the knowledge base to obtain multiple model structures of the neural network model; The matching of the neural network model to be segmented with multiple structural templates in the knowledge base includes: The multiple model structures are matched with structure templates in the knowledge base.

7. The method according to claim 6, characterized in that The neural network model is split based on the computing nodes included in the structure template in the knowledge base to obtain multiple model structures of the neural network model, including: Among all the computing nodes of the neural network model, the computing nodes that do not conform to the structural features of all the structural templates in the knowledge base are deleted to obtain the multiple model structures.

8. The method according to claim 6 or 7, characterized in that Before matching the multiple model structures with the structure templates in the knowledge base, the method further includes: Screening the multiple model structures based on the number of computing nodes included in the structure template in the knowledge base; The matching of the multiple model structures with the structure templates in the knowledge base includes: The screened model structures are matched with the structural templates in the knowledge base.

9. The method according to any one of claims 1 to 8, characterized in that: Before matching the neural network model to be segmented with a plurality of structural templates in the knowledge base, the method further includes: Based on the segmentation result of the template neural network model, obtaining the structure template; The knowledge base is created based on the structure template and its segmentation strategy.

10. The method according to claim 9, characterized in that The structural template satisfies one or more of the following: The number of divisions of different computing nodes in the structure template is in integer multiples; Furthermore, all computing nodes except the first computing node and the last computing node in the structure template are split.

11. The method according to claim 10, characterized in that The first computing node and the last computing node are both split.

12. A neural network model segmentation device, characterized in that: The device comprises: A matching module is used to match the neural network model to be segmented with a plurality of structural templates in a knowledge base, and obtain a target structural template that matches the target model structure in the neural network model, wherein the knowledge base also records segmentation strategies of each structural template, wherein the segmentation strategies indicate a method for segmenting the structural template, and the segmentation strategies recorded in the knowledge base can all achieve optimization benefits, and the model structure and the structural template both include a plurality of computing nodes with a specified connection relationship; A first acquisition module, used to acquire a segmentation strategy of the target structure template; A segmentation module is used to segment the target model structure based on the segmentation strategy of the target structure template to obtain a segmented neural network model.

13. The device according to claim 12, characterized in that The matching of the target model structure with the target structure template includes: a structural feature of the target model structure is the same as a structural feature of the target structure template.

14. The device according to claim 13, characterized in that The structural features include one or more of the following: topological features and connection features, wherein the topological features indicate the execution order between computing nodes, and the connection features indicate the connection relationship between computing nodes.

15. The device according to any one of claims 12 to 14, characterized in that: When the multiple target model structures in the neural network model are matched with the multiple target structure templates in the knowledge base respectively, the segmentation module is specifically used to: When there is no model structure including the same computing node among the multiple target model structures, segmenting the target model structures matching the target structure template based on the segmentation strategies of the multiple target structure templates respectively; When there are model structures including the same computing nodes among the multiple target model structures, the multiple target model structures are screened so that there are no model structures including the same computing nodes among the screened multiple target model structures, and the screened target model structures are segmented based on the segmentation strategies of the target structure templates that match the screened target model structures.

16. The device according to claim 15, characterized in that The segmentation module is specifically used for: Combining the multiple target model structures to obtain multiple model structure combinations, wherein the model structure combinations include some of the target model structures in the multiple target model structures, and the model structure set does not contain repeated computing nodes; Based on the optimization benefits of the segmentation strategies of the multiple target structure templates, respectively obtaining benefits of the multiple model structure combinations; Based on the benefits of the multiple model structure combinations, a target model structure combination is determined from the multiple model structure combinations, wherein the target model structure in the target model structure combination is a screened target model structure.

17. The device according to any one of claims 12 to 16, characterized in that: The matching module is specifically used for: Splitting the neural network model based on the computing nodes included in the structural template in the knowledge base to obtain multiple model structures of the neural network model; The multiple model structures are matched with structure templates in the knowledge base.

18. The device according to claim 17, characterized in that The matching module is specifically used for: Among all the computing nodes of the neural network model, the computing nodes that do not conform to the structural features of all the structural templates in the knowledge base are deleted to obtain the multiple model structures.

19. The device according to claim 17 or 18, characterized in that The matching module is specifically used for: Screening the multiple model structures based on the number of computing nodes included in the structure template in the knowledge base; The screened model structures are matched with the structural templates in the knowledge base.

20. The device according to any one of claims 12 to 19, characterized in that The device also includes: A second acquisition module is used to acquire the structure template based on the segmentation result of the template neural network model; A creation module is used to create the knowledge base based on the structure template and its segmentation strategy.

21. The device according to claim 20, characterized in that The structural template satisfies one or more of the following: The number of divisions of different computing nodes in the structure template is in integer multiples; Furthermore, all computing nodes except the first computing node and the last computing node in the structure template are split.

22. The device according to claim 21, characterized in that The first computing node and the last computing node are both split.

23. A computing device cluster, characterized in that: The method comprises at least one computing device, each computing device comprises a processor and a memory, and the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 11.

24. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 11.

25. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 11.

Images