Reasoning method, device, computer equipment and storage medium for deep learning model

Abstract

A method for reasoning a deep learning model includes: obtaining a model reasoning task, the model reasoning task carrying a model identifier; obtaining a corresponding deep learning model according to the model identifier, parsing the deep learning model to obtain a model network corresponding to the deep learning model, the model network including a plurality of nodes; identifying a target node and an end node corresponding to the target node in the model network; traversing the model network in reverse according to the end node, and determining a target node corresponding to the end node in the target node; merging the target nodes corresponding to the end nodes in the model network to obtain an optimized deep learning model; and reasoning according to the optimized deep learning model to obtain a model reasoning result.

Description

Technical Field

[0001] The present application relates to a method, apparatus, computer device and storage medium for reasoning a deep learning model. Background Art

[0002] A deep learning model refers to a neural network model based on deep learning. Deep learning models are widely used in speech recognition, image recognition, and natural language processing. With the development of computer technology, the network structure of deep learning models has become increasingly complex, with the number of layers increasing from dozens to thousands or even more. As the network structure of deep learning models becomes more numerous and complex, it takes more time to infer deep learning models, that is, to input data and run the entire deep learning model to obtain output data. The traditional method is to infer the operation layers in the network structure of the deep learning model one by one, which will take up more memory space and require a lot of memory reading and writing time, resulting in a slow model inference speed. Therefore, how to improve the model inference speed has become a technical problem that needs to be solved. Summary of the invention

[0003] According to various embodiments disclosed in the present application, a method, apparatus, computer device and storage medium for reasoning a deep learning model that can improve the model reasoning speed are provided.

[0004] A method for reasoning a deep learning model, comprising:

[0005] Obtain a model reasoning task, where the model reasoning task carries a model identifier;

[0006] Acquire a corresponding deep learning model according to the model identifier, parse the deep learning model, and obtain a model network corresponding to the deep learning model, wherein the model network includes a plurality of nodes;

[0007] Identifying a target node in the model network, and an end node corresponding to the target node;

[0008] Reversely traverse the model network according to the end node, and determine a target node corresponding to the end node among the target nodes;

[0009] Merging the target nodes corresponding to the end nodes in the model network to obtain an optimized deep learning model; and

[0010] Reasoning is performed based on the optimized deep learning model to obtain a model reasoning result.

[0011] A reasoning device for a deep learning model, comprising:

[0012] An acquisition module, used to acquire a model reasoning task, wherein the model reasoning task carries a model identifier;

[0013] A parsing module, used to obtain a corresponding deep learning model according to the model identifier, parse the deep learning model, and obtain a model network corresponding to the deep learning model, wherein the model network includes a plurality of nodes;

[0014] An identification module, used to identify a target node and an end node corresponding to the target node in the model network;

[0015] A determination module, configured to reversely traverse the model network according to the end node, and determine a target node corresponding to the end node among the target nodes;

[0016] A merging module, used to merge the target nodes corresponding to the end nodes in the model network to obtain an optimized deep learning model; and

[0017] The inference module is used to perform inference based on the optimized deep learning model to obtain a model inference result.

[0018] A computer device includes a memory and one or more processors, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processors, the one or more processors perform the following steps:

[0019] Obtain a model reasoning task, where the model reasoning task carries a model identifier;

[0020] Acquire a corresponding deep learning model according to the model identifier, parse the deep learning model, and obtain a model network corresponding to the deep learning model, wherein the model network includes a plurality of nodes;

[0021] Identifying a target node in the model network, and an end node corresponding to the target node;

[0022] Reversely traverse the model network according to the end node, and determine a target node corresponding to the end node among the target nodes;

[0023] Merging the target nodes corresponding to the end nodes in the model network to obtain an optimized deep learning model; and

[0024] Reasoning is performed based on the optimized deep learning model to obtain a model reasoning result.

[0025] One or more computer-readable storage media storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the following steps:

[0026] Obtain a model reasoning task, where the model reasoning task carries a model identifier;

[0027] Acquire a corresponding deep learning model according to the model identifier, parse the deep learning model, and obtain a model network corresponding to the deep learning model, wherein the model network includes a plurality of nodes;

[0028] Identifying a target node in the model network, and an end node corresponding to the target node;

[0029] Reversely traverse the model network according to the end node, and determine a target node corresponding to the end node among the target nodes;

[0030] Merging the target nodes corresponding to the end nodes in the model network to obtain an optimized deep learning model; and

[0031] Reasoning is performed based on the optimized deep learning model to obtain a model reasoning result.

[0032] The details of one or more embodiments of the present application are set forth in the following drawings and description. Other features and advantages of the present application will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 An application environment diagram of the reasoning method of the deep learning model in one or more embodiments.

[0035] Figure 2 A flowchart of a method for reasoning a deep learning model in one or more embodiments.

[0036] Figure 3 It is a local abstract graph in the abstract graph corresponding to the model network in one or more embodiments.

[0037] Figure 4 This is a schematic diagram of performing reverse traversal from the end node “OP3” in one or more embodiments.

[0038] Figure 5 This is a schematic diagram of performing reverse traversal from the end node “OP4” in one or more embodiments.

[0039] Figure 6 A schematic diagram of a model network after merging in one or more embodiments.

[0040] Figure 7 A flowchart of the step of identifying a target node in a model network according to a topological sequence in one or more embodiments.

[0041] Figure 8 A flowchart of a reasoning method for a deep learning model in another embodiment.

[0042] Fig. 9 A block diagram of an inference device for a deep learning model in another embodiment.

[0043] Fig.10 is a block diagram of a computer device in accordance with one or more embodiments. DETAILED DESCRIPTION

[0044] In order to make the technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] The reasoning method of the deep learning model provided in this application can be applied to a computer device, which can be a terminal or a server. It is understandable that the reasoning method of the deep learning model provided in this application can be applied to a terminal, a server, or a system including a terminal and a server, and is implemented through the interaction between the terminal and the server.

[0046] The inference method of the deep learning model provided in this application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The terminal 102 can obtain a model reasoning task, and the model reasoning task carries a model identifier. The terminal 102 obtains the corresponding deep learning model according to the model identifier, parses the deep learning model, and obtains a model network corresponding to the deep learning model. The model network includes multiple nodes, identifies the target node and the end node in the model network, and then reversely traverses the model network according to the end node, determines the target node corresponding to the end node in the target node, merges the target nodes corresponding to the end node in the model network, and obtains an optimized deep learning model, and then performs reasoning according to the optimized deep learning model to obtain a model reasoning result, and sends the model reasoning result to the server 104 for storage. The terminal 102 may specifically include but is not limited to various personal computers, laptops, smart phones, and tablet computers. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0047] It is understandable that the reasoning method of the deep learning model provided in the present application realizes reasoning of the deep learning model, which can be applied to a variety of application environments, and the deep learning model can include multiple types. For example, the deep learning model can include a convolutional neural network model, a recurrent neural network model, and a recursive neural network model. The deep learning model can be used to process a variety of different data. For example, the deep learning model can specifically include an image recognition model, a feature extraction model, a speech recognition model, a text recognition model, and a scene classification model.

[0048] In one of the embodiments, the reasoning method of the deep learning model provided in the present application can be specifically applied in the field of autonomous driving, and the deep learning model can specifically include at least one of an image recognition model or a trajectory prediction model.

[0049] In one embodiment, the reasoning method of the deep learning model provided in the present application can be specifically applied in the text field, and the deep learning model can specifically include at least one of an image recognition model, a behavior prediction model, or a risk assessment model.

[0050] In one embodiment, if Figure 2 As shown, a method for reasoning a deep learning model is provided, and the method is applied to a computer device as an example for explanation. The computer device may be Figure 1 The terminal or server in the embodiment includes the following steps:

[0051] Step 202: Obtain a model reasoning task, where the model reasoning task carries a model identifier.

[0052] Model reasoning refers to the operation of the data input into the deep learning model according to the model network structure sequence of the deep learning model and the arithmetic operations corresponding to each of the multiple computing layers included in the network structure, so as to obtain the reasoning result output by the deep learning model. The model reasoning task is used to instruct the computer device to reason about the corresponding deep learning model. The computer device can be a terminal or a server. The model identifier refers to the unique identifier for marking the deep learning model, which is used to distinguish the deep learning models.

[0053] When model reasoning is required, the computer device can obtain the model reasoning task, parse the model reasoning task, and obtain the model identifier carried in the model reasoning task. Specifically, when the user needs to perform model reasoning, the computer device can determine the deep learning model specified by the user according to the user's operation instructions received, and generate a model reasoning task carrying the model identifier. The computer device can also determine the deep learning model that needs to be called according to actual operating requirements and generate a model reasoning task. For example, in the process of image recognition, when it is necessary to call the image recognition model to perform recognition processing on the image, the computer device can generate a model reasoning task, so that according to the model reasoning task, the image recognition model is reasoned after the image is input, and the image recognition result output by the image recognition model is obtained.

[0054] In one embodiment, the computer device may pre-store an inference engine, and the computer device may perform a model inference task through the inference engine to infer the deep learning model corresponding to the model identifier. The inference engine refers to a functional module in the computer device for completing inference.

[0055] Step 204, obtain the corresponding deep learning model according to the model identifier, parse the deep learning model, and obtain a model network corresponding to the deep learning model, wherein the model network includes multiple nodes.

[0056] A deep learning model is pre-stored in the computer device. The deep learning model is obtained by training with a large amount of sample data, so that the corresponding deep learning model can be obtained according to the model identifier. The deep learning model corresponding to the model identifier may include at least one of multiple types of neural network models. For example, depending on the network structure of the deep learning model, it may specifically include at least one of a convolutional neural network model (CNN), a recurrent neural network model (RNN), and a recursive neural network model. Depending on the function of the deep learning model, the neural network model may specifically include at least one of an image recognition model, a feature extraction model, a speech recognition model, a text recognition model, and a scene classification model.

[0057] The computer device analyzes the acquired deep learning model to obtain a model network corresponding to the deep learning model. The model network refers to the network structure corresponding to the deep learning model. The model network can be expressed in the form of an abstract graph corresponding to the network structure of the model network, such as Figure 3 shown. Figure 3It is a local abstract graph in the abstract graph corresponding to the model network. The abstract graph corresponding to the model network is a directed graph, and the abstract graph includes multiple nodes and directed edges between nodes. The nodes in the abstract graph can represent the operation layer in the deep learning model. For example, the operation layer can include an input layer, a convolution layer, an addition layer, a multiplication layer, and an output layer. The node can be represented by the arithmetic operation corresponding to the operation layer, or by the operation layer identifier corresponding to the operation layer. The arithmetic operation corresponding to the operation layer can be called an operator, which refers to the arithmetic operation performed on the input information of the operation layer. For example, the node can be represented by an arithmetic operation operator, such as "+", "-", "Sqrt", "Floor", etc. Sqrt refers to square root, and Floor refers to rounding down to the nearest integer. The directed edges between nodes represent the input-output relationship between the operation layers of the deep learning model, and the output result of the node can be output to the node pointed to by the arrow according to the directed edge. Each node can include the operation layer information of the corresponding operation layer, and the operation layer information can specifically include the operation layer identifier corresponding to the operation layer, the arithmetic operation corresponding to the operation layer, and the arithmetic operation type. The operation layer information of the operation layer is the operation layer information of the corresponding node.

[0058] Step 206, identifying the target node and the end node in the model network.

[0059] The target node refers to the node that needs to be merged. The end node refers to the node pointed to by the directed edge arrow of the final output result of the target node.

[0060] In a deep learning model, there are a large number of operation layers, and there may be some operation layers that only perform basic arithmetic operations. These operation layers are inferred one by one, and each inference will temporarily store the inference results in the memory, which will take up more memory space and require a lot of memory reading and writing time, resulting in a slow model inference speed. It is not conducive to data processing processes with high real-time requirements. For example, in the field of autonomous driving, it is necessary to quickly infer data processing results based on deep learning models. Therefore, the computer device can identify the target nodes that need to be merged and the end nodes in the model network. By merging the nodes, the corresponding arithmetic operations represented by the nodes can be merged to reduce the memory space occupied by the reasoning process and reduce the memory reading and writing time.

[0061] The model network includes multiple nodes and directed edges between nodes. The computer device can determine the reasoning order between nodes based on the directed edges, so that each node can be searched from top to bottom in turn according to the reasoning order, and the target node and the end node in the model network can be identified. Specifically, the computer device obtains the corresponding arithmetic operation in the operation layer information corresponding to each node. The arithmetic operation is the operator used in the operation layer represented by the node. The computer device identifies whether the arithmetic operation corresponding to each node meets the preset conditions and obtains the recognition result. The preset condition can be that the type of arithmetic operation is a basic arithmetic operation. Basic arithmetic operations can include addition, subtraction, multiplication and division, basic functions, and arithmetic operations that can be expressed by expressions formed by simple combinations of the above arithmetic operations. When the arithmetic operations corresponding to multiple consecutive nodes in the recognition result all meet the preset conditions, the multiple consecutive nodes that meet the preset conditions can be determined as target nodes. Continuous nodes refer to multiple nodes connected in sequence by directed edges, and multiple refers to at least two. Therefore, the nodes in the target node include at least two consecutive nodes connected by directed edges, and the consecutive nodes can be merged. As Figure 3 As shown in FIG. 1 , the node “+”, the node “Sqrt”, and the node “Floor” are target nodes. After identifying the target node, the computer device may obtain the output result in the target node, determine the node pointed to by the arrow of the directed edge of the output result, and thereby determine the node pointed to by the arrow as the end node corresponding to the target node. The end node may be an output node of the target node, used to receive the output result of the target node. The end node may include at least one node. Figure 3 As shown, the node "OP3" and the node "OP4" are end nodes.

[0062] Step 208, reversely traverse the model network according to the end node, and determine the target node corresponding to the end node in the target nodes.

[0063] After the computer device identifies the end node in the model network, it can reversely traverse the nodes in the model network from the end node. Reverse traversal refers to reasoning in the opposite direction of the arrows of the directed edges between nodes. Through reverse traversal, the target node corresponding to each end node can be obtained from the identified target nodes. Figure 3 Take this as an example to illustrate: Figure 3 The end nodes in are "OP3" and "OP4". The computer device can perform reverse traversal on the end node "OP3" and the end node "OP4" respectively, so as to obtain the target node corresponding to the end node "OP3" and the target node corresponding to the end node "OP4". Figure 4 As shown in FIG. 1 , it is a schematic diagram of reverse traversal from the end node “OP3”, and the dotted box represents the end node “OP3” and the corresponding target node. Figure 5As shown, it is a schematic diagram of reverse traversal from the end node "OP4", and the dotted box represents the end node "OP4" and the corresponding target node. By determining the target node corresponding to each end node, the output result of the end node can be obtained according to the output result of the target node corresponding to each end node after the target nodes are merged.

[0064] In one embodiment, the reverse traversal of the model network may be performed by any of the existing reverse traversal methods, for example, a DFS (Depth First Search) method or a BFS (Breadth First Search) method.

[0065] Step 210, merge the target nodes corresponding to the end nodes in the model network to obtain an optimized deep learning model.

[0066] After obtaining the target node corresponding to each end node by reverse traversing the model network, the target nodes corresponding to all end nodes can be merged. The computer device counts the output information and output results of all the merged target nodes, and records the counted output information and output results. Thus, the computer device determines the input information of each end node based on the counted output information and output results, and then obtains the merged abstract graph, that is, the merged model network. The merged model network can be Figure 6 As shown in the figure, "Merged OP" means Figure 3 The merged node is obtained by merging the target nodes "+", "Sqrt", and "Floor" in the "Merged OP". The expression contained in the directed edge of "Merged OP" is the input information of the end node. The computer device can generate an optimized deep learning model based on the merged model network. The arithmetic operations used by the deep learning model are merged by merging the nodes in the model network to optimize the deep learning model.

[0067] Step 212, perform reasoning based on the optimized deep learning model to obtain a model reasoning result.

[0068] After the computer device obtains the optimized deep learning model, the optimized deep learning model reduces the reasoning steps in the reasoning process compared with the traditional deep learning model. By merging the arithmetic operations of multiple operation layers, the memory usage in the reasoning process is reduced, and the memory read and write time is reduced, thereby improving the model reasoning speed.

[0069] The computer device can perform reasoning according to the optimized deep learning model, and perform operations in sequence according to the arithmetic operations of the operation layer corresponding to the optimized deep learning model to obtain the inferred data results. For example, the computer device can perform operations on the input image in sequence according to the deep learning model and the arithmetic operation sequence corresponding to the optimized deep learning model to obtain the recognized image results.

[0070] In this embodiment, the computer device obtains the model reasoning task, parses the model reasoning task, obtains the model identifier, parses the deep learning model corresponding to the model identifier, obtains the model network corresponding to the deep learning model, identifies the target node in the model network, and the end node corresponding to the target node, thereby reversely traversing the model network according to the end node, determining the target node corresponding to the end node in the target node, merging the target nodes corresponding to the end node in the model network, obtaining the optimized deep learning model, and then reasoning according to the optimized deep learning model to obtain the model reasoning result. After obtaining the model reasoning task, the computer device can automatically obtain the deep learning model corresponding to the model reasoning task, analyze the model network corresponding to the deep learning model, and automatically merge the nodes in the model network, reducing the development workload of the technician. By merging nodes, the arithmetic operations corresponding to the nodes are merged, the memory usage of the reasoning process is reduced, and the memory reading time is reduced, which effectively improves the speed of model reasoning. In addition, compared with the way of model reasoning based on the graph-based deep model acceleration engine, the skill requirements for technicians are lower, avoiding additional learning costs, and there is no need for technicians to develop third-party plug-ins, avoiding additional development time, and reducing the development burden of technicians.

[0071] In one embodiment, identifying a target node in a model network and an end node corresponding to the target node includes: topologically sorting the nodes in the model network to obtain a topological sequence; and identifying the target node in the model network and an end node corresponding to the target node according to the topological sequence.

[0072] The computer device can topologically sort the nodes in the model network to obtain a topological sequence. Topological sorting refers to arranging the nodes in a directed network into a sequence that satisfies the topological order. The sequence obtained by topological sorting is a one-dimensional linear sequence. The order between nodes in the linear sequence obtained by topological sorting by the computer device is fixed. Since each node corresponds to an arithmetic operation, by topologically sorting the nodes in the model network, the arithmetic operations corresponding to the nodes are topologically sorted, and the order of obtaining the arithmetic operations is the same as the order of obtaining the nodes.

[0073] The computer device can sequentially obtain the nodes in the model network according to the order of the topological sequence, and obtain the arithmetic operation corresponding to the node, identify whether the arithmetic operation corresponding to each node meets the preset conditions, and obtain the recognition result. When the arithmetic operations corresponding to multiple continuous nodes in the recognition result all meet the preset conditions, the multiple continuous nodes that meet the preset conditions can be determined as the target node. Continuous nodes refer to multiple nodes connected in sequence by directed edges, and multiple refers to at least two. Therefore, the nodes in the target node include at least two continuous nodes connected by directed edges, and the continuous nodes can be merged. After identifying the target node, the computer device can obtain the output result in the target node, determine the node pointed to by the arrow of the directed edge of the output result, and thereby determine the node pointed to by the arrow as the end node corresponding to the target node.

[0074] In this embodiment, the computer device topologically sorts the nodes in the model network to ensure that when the arithmetic operation corresponding to a node is inferred, the input information of the node, that is, the arithmetic operation corresponding to the previous node has been inferred, thereby improving the recognition accuracy of the target node and the end node.

[0075] In one embodiment, identifying a target node in a model network according to a topological sequence and an end node corresponding to the target node includes: identifying a target node in a model network according to a topological sequence and generating an output result of the target node; and identifying an end node corresponding to the target node according to the target node and the output result of the target node.

[0076] The topological sequence includes the order of obtaining each node in the model network. The computer device obtains each node in the model network in turn according to the topological sequence. When there are multiple consecutive nodes in the model network and the corresponding arithmetic operations are all basic arithmetic operations, the consecutive nodes are used as target nodes to generate the output results of the target nodes. The basic arithmetic operations may include addition, subtraction, multiplication, division, basic functions, and arithmetic operations that are simply combined from the above arithmetic operations and can be expressed in expressions.

[0077] In one embodiment, identifying a target node in a topological network and generating an output result of the target node includes: traversing all nodes in the model network according to a topological sequence, and when the current node is identified as belonging to the target node, obtaining input information of the current node; generating an output result of the current node according to the input information, until the traversal is completed and the output result of the target node is obtained.

[0078] The computer device traverses the nodes in the model network according to the node order in the topological sequence, obtains the current node, and when the arithmetic operation corresponding to the current node and the arithmetic operation corresponding to the node connected to the current node are both basic arithmetic operations, the current node and the node connected to the current node can be determined as the target node, and the node connected to the current node can be the previous node or the next node of the current node. At this time, the computer device can obtain the input information of the current node, and the input information refers to the output result of the previous node of the current node, that is, the arithmetic expression corresponding to the previous node. Thus, the computer device outputs the output result of the current node according to the input information of the current node and the arithmetic operation corresponding to the current node, that is, generates the arithmetic expression of the current node. For example, the current node is "+", indicating that the arithmetic operation corresponding to the node is addition, and the input information of the current node is "OP1, OP2", then the output result of the current node is "OP1+OP2". The current node is "Sqrt", indicating that the arithmetic operation corresponding to the node is Sqrt, and the input information of the current node is "OP1+OP2", then the output result of the current node is "Sqrt(OP1+OP2)". The current node is "Floor", which means that the arithmetic operation corresponding to the node is addition. The input information of the current node is "Sqrt(OP1+OP2)", and the output result of the current node is "Floor(Sqrt(OP1+OP2))". After the traversal is completed, the computer device obtains the output results of all target nodes. By generating the output results of each target node, it is helpful to determine the input information of the end node when merging nodes later, thereby improving the accuracy of node merging.

[0079] After identifying the target node, the computer device can obtain the output result in the target node, determine the node pointed to by the arrow of the directed edge of the output result, and thus determine the node pointed to by the arrow as the end node corresponding to the target node.

[0080] In this embodiment, the computer device identifies the target node in the model network according to the topological sequence, generates the output result of the target node, and identifies the end node corresponding to the target node according to the target node and the output result of the target node. By generating the output result of the target node, the end node can be quickly identified, and after the node is merged, the input information of the end node is determined, thereby improving the accuracy of the node merging.

[0081] In one embodiment, if Figure 7 As shown, the nodes in the model network are connected by directed edges, and the steps of identifying the target node in the model network according to the topological sequence include:

[0082] Step 702: traverse all nodes in the model network according to the topological sequence to obtain the arithmetic operation corresponding to each node.

[0083] Step 704: When there are multiple consecutive nodes connected by directed edges and the arithmetic operations corresponding to the nodes are all basic arithmetic operations, the consecutive nodes are determined as target nodes.

[0084] The computer device traverses the model network according to the topological sequence. During the traversal process, the arithmetic operation corresponding to each node can be obtained from the operation layer information corresponding to each node, and whether the arithmetic operation corresponding to the node is a basic arithmetic operation is identified. If so, the node connected to the current node through a directed edge is determined. The node connected to the current node through a directed edge can be the previous node or the next node of the current node. The current node and the previous node or the current node and the next node can both be called continuous nodes. The continuous node may include at least two nodes. The computer device identifies whether the arithmetic operation corresponding to the node connected to the current node through a directed edge is a basic arithmetic operation. When the arithmetic operation corresponding to the previous node is a basic arithmetic operation or the arithmetic operation corresponding to the next node is a basic arithmetic operation, the current node and the node connected to the current node through the directed edge corresponding to the basic arithmetic operation are determined as the target node. Therefore, during the traversal process, when the computer device identifies that the arithmetic operations corresponding to multiple continuous nodes connected through directed edges are all basic arithmetic operations, the continuous node is determined as the target node.

[0085] In this embodiment, the computer device traverses all nodes in the model network according to the topological sequence, obtains the arithmetic operation corresponding to each node, and when there are multiple continuous nodes connected by directed edges whose corresponding arithmetic operations are all basic arithmetic operations, the continuous nodes are determined as target nodes. The target node can be determined by simply identifying whether the arithmetic operation corresponding to the continuous nodes is a basic arithmetic operation, which further improves the recognition efficiency of the target node.

[0086] In one embodiment, identifying the end node corresponding to the target node in the topological network based on the target node and the output result of the target node includes: obtaining input information of the target node and adding the input information to an input set; adding the output result of the target node to an output set; determining the target output result based on the input set and the output set, and determining the end node corresponding to the target node in the topological network based on the target output result.

[0087] The computer device traverses the nodes in the model network according to the node order in the topological sequence. When the node is identified as a target node, the input information of the node can be read in the memory and the input information can be added to the input set. When the computer device generates the output result of the node according to the input information and the arithmetic operation corresponding to the node, the output result is added to the output set. After the traversal is completed, the input set includes the output information of all target nodes, and the output set includes the output results of all target nodes. The time consumption of adding the input information to the input set can be determined according to the number of nodes, that is, the number of arithmetic operations corresponding to the nodes. Similarly, the time consumption of adding the output results to the output set can be determined according to the number of nodes, that is, the number of arithmetic operations corresponding to the nodes. There is overlapping information in the input set and the output set, and the computer device only needs to determine the target output result in the input set and the output set. The target output result refers to the output result of the node that needs to be input to the node other than the target node. The target output result can include at least one. The computer device can determine the node pointed to by the directed edge arrow corresponding to the target output result as the end node. One target output result corresponds to one end node.

[0088] In this example, the computer device obtains the input information of the target node, adds the input information to the input set, adds the output result of the target node to the output set, determines the target output result according to the input set and the output set, and determines the end node corresponding to the target node in the topological network according to the target output result. By adding the input information to the input set and the output result of the target node to the output set, the identified target node information can be obtained, which is conducive to determining the target node corresponding to the end node in the target node during the subsequent reverse traversal of the model network, and determining the end node through the target output result can quickly determine the end node.

[0089] In another embodiment, Figure 8 As shown, a reasoning method for a deep learning model is provided, comprising the following steps:

[0090] Step 802: Obtain a model reasoning task, where the model reasoning task carries a model identifier.

[0091] Step 804, obtaining a corresponding deep learning model according to the model identifier, parsing the deep learning model, and obtaining a model network corresponding to the deep learning model, wherein the model network includes multiple nodes.

[0092] Step 806, topologically sort the nodes in the model network to obtain a topological sequence.

[0093] Step 808, identifying the target node in the model network according to the topological sequence, and generating an output result of the target node.

[0094] Step 810: Identify the end node corresponding to the target node according to the target node and the output result of the target node.

[0095] Step 812, reversely traverse the model network according to the end node, and determine the target node corresponding to the end node in the target nodes.

[0096] Step 814, merging the target nodes corresponding to the end nodes in the model network to obtain a merged group.

[0097] Step 816: Determine the output result of each target node in the merge group in the generated output result.

[0098] Step 818, generating input information corresponding to the end node according to the output result of the target node corresponding to the end node, and obtaining an optimized deep learning model.

[0099] Step 820, perform reasoning based on the optimized deep learning model to obtain a model reasoning result.

[0100] The computer device reversely traverses the model network according to the end node, and after determining the target node corresponding to the end node in the target node, merges the target nodes corresponding to all the end nodes in the model network to obtain a merged group. Since the computer device generates the output result of the target node when identifying the target node in the model network according to the topological sequence. The output result refers to the arithmetic expression corresponding to the target node. The computer device can determine the output result of each target node in the merged group in the generated output result, thereby obtaining the output result of the target node corresponding to each end node. Then, according to the output result of the target node corresponding to each end node, the input information of the corresponding end node is generated, thereby obtaining the directed edge between the merged node corresponding to the merged group and the corresponding end node, and the merged model network is obtained, and then the optimized deep learning model is generated according to the merged model network. The computer device can perform reasoning according to the optimized deep learning model, and perform operations in sequence according to the arithmetic operations of the operation layer corresponding to the optimized deep learning model to obtain the inferred data results. For example, the computer device can perform operations on the input image in sequence according to the deep learning model and the arithmetic operation sequence corresponding to the optimized deep learning model to obtain the recognized image result.

[0101] In this embodiment, since the generated output result is generated by the computer device after identifying the target node in the model network according to the topological sequence, the accurate output result of each target node can be obtained, and the output result of the target node corresponding to each end node is determined in the generated output result, thereby generating the input information of the end node, thereby improving the accuracy of the input information of the end node. At the same time, the target nodes are merged to generate the input information of the end node, and the optimized deep learning model is obtained. Reasoning is performed through the optimized deep learning model, which reduces the memory usage of the reasoning process, and at the same time reduces the memory reading time, effectively improving the speed of model reasoning.

[0102] It should be understood that although Figure 2 , 7 The steps in the flowcharts of and 8 are shown in sequence as indicated by arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 , 7 At least part of the steps in 8 may include multiple sub-steps or multiple stages. These sub-steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages does not necessarily have to be sequentially, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0103] In one embodiment, if Fig. 9 As shown, a reasoning device for a deep learning model is provided, including: an acquisition module 902, a parsing module 904, an identification module 906, a determination module 908, a merging module 910 and a reasoning module 912, wherein:

[0104] The acquisition module 902 is used to acquire a model reasoning task, where the model reasoning task carries a model identifier.

[0105] The parsing module 904 is used to obtain the corresponding deep learning model according to the model identifier, parse the deep learning model, and obtain a model network corresponding to the deep learning model, wherein the model network includes multiple nodes.

[0106] The identification module 906 is used to identify the target node and the end node corresponding to the target node in the model network.

[0107] The determination module 908 is used to reversely traverse the model network according to the end node, and determine the target node corresponding to the end node in the target nodes.

[0108] The merging module 910 is used to merge the target nodes corresponding to the end nodes in the model network to obtain an optimized deep learning model.

[0109] The reasoning module 912 is used to perform reasoning based on the optimized deep learning model to obtain a model reasoning result.

[0110] In one embodiment, the identification module 906 is further used to topologically sort the nodes in the model network to obtain a topological sequence; and identify the target node in the model network and the end node corresponding to the target node according to the topological sequence.

[0111] In one embodiment, the identification module 906 is further used to identify the target node in the model network according to the topological sequence and generate an output result of the target node; and to identify the end node corresponding to the target node according to the target node and the output result of the target node.

[0112] In one of the embodiments, nodes in the model network are connected by directed edges, and the identification module 906 is also used to traverse all nodes in the model network according to the topological sequence to obtain the arithmetic operation corresponding to each node; when there are multiple consecutive nodes connected by directed edges and the arithmetic operations corresponding to them are all basic arithmetic operations, the consecutive nodes are determined as target nodes.

[0113] In one embodiment, the identification module 906 is also used to traverse all nodes in the model network according to the topological sequence, and when it is identified that the current node belongs to the target node, the input information of the current node is obtained; the output result of the current node is generated according to the input information until the traversal is completed and the output result of the target node is obtained.

[0114] In one embodiment, the identification module 906 is also used to obtain input information of the target node and add the input information to the input set; add the output result of the target node to the output set; determine the target output result based on the input set and the output set, and determine the end node corresponding to the target node based on the target output result.

[0115] In one embodiment, the merging module 910 is also used to merge the target nodes corresponding to the end nodes in the model network to obtain a merged group; determine the output results of each target node in the merged group in the generated output results; generate input information of the corresponding end node according to the output results of the target node corresponding to the end node, and obtain an optimized deep learning model.

[0116] For the specific definition of the inference device of the deep learning model, please refer to the definition of the inference method of the deep learning model above, which will not be repeated here. Each module in the above-mentioned inference device of the deep learning model can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0117] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer-readable instructions and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium. The database of the computer device is used to store data of an inference method of a deep learning model. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer-readable instructions are executed by the processor, an inference method of a deep learning model is implemented.

[0118] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0119] A computer device includes a memory and one or more processors. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the one or more processors, the one or more processors execute the steps in the above-mentioned method embodiments.

[0120] One or more computer-readable storage media storing computer-readable instructions, when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps in the above-mentioned various method embodiments. The computer storage medium is a readable storage medium, and the readable storage medium can be non-volatile or volatile.

[0121] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile computer-readable storage medium. When the computer-readable instructions are executed, they may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0122] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for reasoning a deep learning model, comprising: Obtain a model reasoning task, where the model reasoning task carries a model identifier; Acquire a corresponding deep learning model according to the model identifier, acquire an input image, parse the deep learning model through the input image, and obtain a model network corresponding to the deep learning model, wherein the model network includes a plurality of nodes; the deep learning model includes an image recognition model; Identifying a target node in the model network and an end node corresponding to the target node, including: topologically sorting the nodes in the model network to obtain a topological sequence, wherein the nodes in the model network are connected by directed edges; traversing all nodes in the model network according to the topological sequence to obtain an arithmetic operation corresponding to each node; when there are multiple continuous nodes connected by directed edges and the corresponding arithmetic operations are all basic arithmetic operations, determining the continuous nodes as target nodes and generating an output result of the target node; identifying an end node corresponding to the target node according to the target node and the output result of the target node; Reversely traverse the model network according to the end node, and determine a target node corresponding to the end node among the target nodes; Merging the target nodes corresponding to the end nodes in the model network to obtain an optimized deep learning model; and The image to be recognized is operated according to the optimized deep learning model to obtain a model inference result.

2. The method according to claim 1, characterized in that The output result of generating the target node includes: Traversing all nodes in the model network according to the topological sequence, and when identifying that the current node belongs to the target node, obtaining input information of the current node; and The output result of the current node is generated according to the input information until the traversal is completed and the output result of the target node is obtained.

3. The method according to claim 1, characterized in that The step of identifying the end node corresponding to the target node according to the target node and the output result of the target node comprises: Obtain input information of the target node, and add the input information to the input set; Add the output result of the target node to the output set; and A target output result is determined according to the input set and the output set, and an end node corresponding to the target node is determined according to the target output result.

4. The method according to claim 1, characterized in that The step of merging the target nodes corresponding to the end nodes in the model network to obtain an optimized deep learning model includes: Merging the target nodes corresponding to the end nodes in the model network to obtain a merged group; Determining the output result of each target node in the merged group in the generated output result; and The input information of the corresponding end node is generated according to the output result of the target node corresponding to the end node, so as to obtain an optimized deep learning model.

5. A reasoning device for a deep learning model, comprising: An acquisition module, used to acquire a model reasoning task, wherein the model reasoning task carries a model identifier; A parsing module, used to obtain a corresponding deep learning model according to the model identifier, obtain an input image, parse the deep learning model through the input image, and obtain a model network corresponding to the deep learning model, wherein the model network includes a plurality of nodes; the deep learning model includes an image recognition model; An identification module is used to identify a target node in the model network and an end node corresponding to the target node, including: topologically sorting the nodes in the model network to obtain a topological sequence, wherein the nodes in the model network are connected by directed edges; traversing all nodes in the model network according to the topological sequence to obtain an arithmetic operation corresponding to each node; when there are multiple continuous nodes connected by directed edges and the arithmetic operations corresponding to the continuous nodes are all basic arithmetic operations, the continuous nodes are determined as target nodes and an output result of the target node is generated; and an end node corresponding to the target node is identified according to the target node and the output result of the target node; A determination module, configured to reversely traverse the model network according to the end node, and determine a target node corresponding to the end node among the target nodes; A merging module, used to merge the target nodes corresponding to the end nodes in the model network to obtain an optimized deep learning model; and The inference module is used to perform operations on the image to be recognized according to the optimized deep learning model to obtain a model inference result.

6. A computer device comprising a memory and one or more processors, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the one or more processors perform the following steps: Obtain a model reasoning task, where the model reasoning task carries a model identifier; Acquire a corresponding deep learning model according to the model identifier, acquire an input image, parse the deep learning model through the input image, and obtain a model network corresponding to the deep learning model, wherein the model network includes a plurality of nodes; the deep learning model includes an image recognition model; Identifying a target node in the model network and an end node corresponding to the target node, including: topologically sorting the nodes in the model network to obtain a topological sequence, wherein the nodes in the model network are connected by directed edges; traversing all nodes in the model network according to the topological sequence to obtain an arithmetic operation corresponding to each node; when there are multiple continuous nodes connected by directed edges and the corresponding arithmetic operations are all basic arithmetic operations, determining the continuous nodes as target nodes and generating an output result of the target node; identifying an end node corresponding to the target node according to the target node and the output result of the target node; Reversely traverse the model network according to the end node, and determine a target node corresponding to the end node among the target nodes; Merging the target nodes corresponding to the end nodes in the model network to obtain an optimized deep learning model; and The image to be recognized is operated according to the optimized deep learning model to obtain a model inference result.

7. The computer device according to claim 6, characterized in that When the processor executes the computer-readable instructions, the processor further performs the following steps: traversing all nodes in the model network according to the topological sequence, and when recognizing that the current node belongs to the target node, obtaining input information of the current node; And generate the output result of the current node according to the input information, until the traversal is completed, and obtain the output result of the target node.

8. The computer device according to claim 6, characterized in that When the processor executes the computer-readable instructions, the processor further performs the following steps: merging the target nodes corresponding to the end nodes in the model network to obtain a merged group; determining the output result of each target node in the merged group in the generated output result; And generate input information of the corresponding end node according to the output result of the target node corresponding to the end node to obtain an optimized deep learning model.

9. One or more computer-readable storage media storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the following steps: Obtain a model reasoning task, where the model reasoning task carries a model identifier; Acquire a corresponding deep learning model according to the model identifier, acquire an input image, parse the deep learning model through the input image, and obtain a model network corresponding to the deep learning model, wherein the model network includes a plurality of nodes; the deep learning model includes an image recognition model; Identifying a target node in the model network and an end node corresponding to the target node, including: topologically sorting the nodes in the model network to obtain a topological sequence, wherein the nodes in the model network are connected by directed edges; traversing all nodes in the model network according to the topological sequence to obtain an arithmetic operation corresponding to each node; when there are multiple continuous nodes connected by directed edges and the corresponding arithmetic operations are all basic arithmetic operations, determining the continuous nodes as target nodes and generating an output result of the target node; identifying an end node corresponding to the target node according to the target node and the output result of the target node; Reversely traverse the model network according to the end node, and determine a target node corresponding to the end node among the target nodes; Merging the target nodes corresponding to the end nodes in the model network to obtain an optimized deep learning model; and The image to be recognized is operated according to the optimized deep learning model to obtain a model inference result.

10. The storage medium according to claim 9, characterized in that The computer-readable instructions, when executed by the processor, further perform the following steps: traverse all nodes in the model network according to the topological sequence, and when it is identified that the current node belongs to the target node, obtain input information of the current node; And generate the output result of the current node according to the input information, until the traversal is completed, and obtain the output result of the target node.

11. The storage medium according to claim 9, characterized in that When the computer-readable instructions are executed by the processor, the following steps are further performed: merging the target nodes corresponding to the end nodes in the model network to obtain a merged group; determining the output result of each target node in the merged group in the generated output result; And generate input information of the corresponding end node according to the output result of the target node corresponding to the end node to obtain an optimized deep learning model.

Images