Neural network calculation graph processing method and processing device

Abstract

The present disclosure provides a method for processing a neural network computation graph, a processing device, an electronic device, and a computer-readable medium. The neural network computation graph includes multiple operator nodes. The processing method includes: splitting the first operator node into multiple target operator nodes according to the dimension length of the batch dimension of the input tensor corresponding to the first operator node, the first operator node is an operator node whose dimension length of the batch dimension of the corresponding input tensor is greater than 1, and the dimension length of the batch dimension corresponding to each target operator node is less than the dimension length of the batch dimension corresponding to the first operator node; when there are any multiple target operator nodes with the same dimension length of the corresponding batch dimension and a first fusion condition is met, merging the multiple target operator nodes into one target operator node; and obtaining a target neural network computation graph based on the unsplit operator nodes and all target operator nodes.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for processing a neural network computation graph, an electronic device, and a computer-readable storage medium. Background Art

[0002] Many-core architecture chips based on integrated storage and computing are an important development direction for many-core chips because they place both computing and storage on-chip, reducing data transfer time and power consumption.

[0003] Deep learning frameworks (such as TensorFlow or ONNX) typically use computational graphs to express the computations of deep learning models (neural networks). Neural network computational graphs typically require compilation to generate instruction streams (executable files or code) that can run on hardware. This hardware can be based on a multi-core chip that integrates both storage and computing, typically consisting of multiple physical cores. Summary of the Invention

[0004] The present disclosure provides a method and device for processing a neural network computation graph, an electronic device, and a computer-readable storage medium.

[0005] In a first aspect, the present disclosure provides a method for processing a neural network computation graph, wherein the original neural network computation graph includes a plurality of operator nodes, and the processing method includes:

[0006] For each first operator node, split the first operator node into multiple target operator nodes according to the dimension length of the batch dimension of the input tensor corresponding to the first operator node, where the first operator node is an operator node whose dimension length of the batch dimension of the corresponding input tensor is greater than 1, and the dimension length of the batch dimension corresponding to each of the target operator nodes is less than the dimension length of the batch dimension corresponding to the first operator node;

[0007] When there are any multiple target operator nodes with the same dimension length of the corresponding batch processing dimension and the first fusion condition is met, the multiple target operator nodes are merged into one target operator node;

[0008] Obtain a target neural network computation graph based on the unsplit operator nodes and all the target operator nodes.

[0009] In a second aspect, the present disclosure provides a processing device for processing a neural network computation graph to be processed, wherein the neural network computation graph includes a plurality of operator nodes, and the processing device includes:

[0010] An operator splitting module is used to split each first operator node into multiple target operator nodes according to the dimension length of the batch processing dimension of the input tensor corresponding to the first operator node, where the first operator node is an operator node whose dimension length of the batch processing dimension of the corresponding input tensor is greater than 1, and the dimension length of the batch processing dimension corresponding to each of the target operator nodes is less than the dimension length of the batch processing dimension corresponding to the first operator node;

[0011] An operator fusion module, configured to merge any multiple target operator nodes with the same dimension length of the corresponding batch processing dimension into one target operator node when the first fusion condition is met;

[0012] A computational graph reconstruction module is used to obtain a target neural network computational graph based on the unsplit operator nodes and all the target operator nodes.

[0013] In a third aspect, the present disclosure provides an electronic device, the electronic device comprising:

[0014] at least one processor;

[0015] and a memory communicatively coupled to the at least one processor;

[0016] In which, the memory stores one or more computer programs that can be executed by the at least one processor, and one or more of the computer programs are executed by the at least one processor so that the at least one processor can execute the above-mentioned neural network computation graph processing method.

[0017] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned method for processing a neural network computation graph when executed by a processor.

[0018] According to the technical solution of the processing method of the neural network calculation graph provided by the embodiment of the present disclosure, the processing method is used to process any suitable neural network calculation graph, and can optimize the original neural network calculation graph into a target neural network calculation graph composed of operator nodes corresponding to a lower batch size, so as to reduce the configuration requirements of the target neural network calculation graph, thereby effectively reducing the occupation of chip resources by the neural network calculation graph during compilation, thereby effectively reducing the compilation difficulty of the neural network calculation graph, improving the compilation efficiency and effect of the neural network calculation graph, and effectively reducing the requirements of the compilation of the neural network calculation graph for chip hardware storage resources, which is conducive to solving the problem that the storage resources required for the compilation of the neural network calculation graph are large but the actual chip hardware storage resources cannot meet them, realizing the rational use of chip hardware storage resources and improving the utilization efficiency of chip hardware storage resources. Moreover, the optimized target neural network calculation graph can effectively obtain the support of the underlying compilation execution, which is conducive to improving the compilation efficiency and effect of the neural network calculation graph.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings. In the accompanying drawings:

[0021] Figure 1 A flowchart of a method for processing a neural network calculation graph provided by an embodiment of the present disclosure;

[0022] Figure 2 A flowchart of another method for processing a neural network computation graph provided by an embodiment of the present disclosure;

[0023] Figure 3 A block diagram of a processing device according to an embodiment of the present disclosure;

[0024] Figure 4 A block diagram of the composition of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] To enable those skilled in the art to better understand the technical solutions of the present disclosure, exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0026] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0027] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0028] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. Similar words such as "connected" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0029] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined as such herein.

[0030] In related technologies, the computing, storage and other resources of the chip are limited. The computational amount and data volume required for the computational graph of a large neural network are usually large. The computing and storage resources of the chip are usually unable to meet the resource requirements of the entire neural network computational graph. In addition, there are usually some restrictions in the underlying compilation and execution, which cannot support the compilation and execution of operator nodes of the computational graph with a batch size (Batch Size) greater than 1, resulting in the compilation of the neural network computational graph being difficult and inefficient.

[0031] To this end, the embodiments of the present disclosure provide a method and processing device for processing a neural network computation graph, an electronic device, and a computer-readable storage medium, aiming to effectively solve at least one of the technical problems existing in the above-mentioned related technologies.

[0032] The processing method of the embodiments of the present disclosure may be executed by a processing device as an execution subject. The processing device may be integrated into an electronic device such as a terminal device or a server through software and / or hardware. For example, the terminal device may be an in-vehicle device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, etc. In some embodiments, the processing method of the embodiments of the present disclosure may be implemented by a processor calling computer-readable program instructions stored in a memory, or the processing method of the embodiments of the present disclosure may be executed by a server.

[0033] Figure 1 A flowchart of a method for processing a neural network computation graph provided in an embodiment of the present disclosure.

[0034] An embodiment of the present disclosure provides a method for processing a neural network computation graph, which is used to process an original neural network computation graph to be processed so as to optimize the graph structure of the neural network computation graph and obtain a target neural network computation graph, wherein the original neural network computation graph may include multiple operator nodes, and the operator nodes are the basic computing units that constitute the neural network. The operator nodes may be, for example, convolution, pooling and other computing operations in the neural network. The neural network may be any type of deep learning network. The neural network may be used to perform any one of image processing tasks, speech processing tasks, text processing tasks, and video processing tasks. The input data of the neural network may be any one of image data, speech data, text data, and video data.

[0035] Reference Figure 1 , the processing method may include: steps S11 to S13.

[0036] Step S11: For each first operator node, split the first operator node into multiple target operator nodes according to the dimension length of the batch processing dimension of the input tensor corresponding to the first operator node.

[0037] Among them, the first operator node is an operator node whose dimension length of the batch processing dimension of the corresponding input tensor is greater than 1, and the dimension length of the batch processing dimension corresponding to each target operator node is smaller than the dimension length of the batch processing dimension corresponding to the first operator node.

[0038] Step S12: When there are any multiple target operator nodes with the same dimension length of the corresponding batch processing dimension and the first fusion condition is met, the multiple target operator nodes are merged into one target operator node.

[0039] Step S13: Obtain the target neural network calculation graph based on the unsplit operator nodes and all target operator nodes.

[0040] In an embodiment of the present disclosure, for an original neural network calculation graph to be processed, before splitting the first operator node into multiple target operator nodes according to the dimension length of the batch dimension of the input tensor corresponding to the first operator node for each first operator node, that is, before step S11, the processing method also includes: traversing the entire original neural network calculation graph to obtain node information of each operator node in the original neural network calculation graph.

[0041] Among them, the node information of the operator node may include but is not limited to: the input connection relationship of the operator node, the output connection relationship, the required parameter information, the attribute information of the operator node, the execution order of the operator node, the tensor attribute information of the input tensor corresponding to the operator node, etc.

[0042] Among them, the input connection relationship of the operator node describes the connection relationship between the input of the operator node and the output of other operator nodes in the neural network calculation graph; the output connection relationship of the operator node describes the connection relationship between the output of the operator node and the input of other operator nodes in the neural network calculation graph; the parameter information required by the operator node includes but is not limited to the pre-configured weight parameters required to implement the operation of the operator node; the attribute information of the operator node is information that characterizes the characteristic attributes of the operator node, and the attribute information of the operator node may include but is not limited to the type of operator node (such as convolution, addition, multiplication, etc.), the computational amount and storage amount required by the operator node, etc.; the execution order of the operator node is the time sequence of executing the operation of the operator node; the tensor attribute information of the input tensor corresponding to the operator node is information that characterizes the characteristic attributes of the input tensor, and the tensor attribute information may include but is not limited to: the rank, axis and shape of the input tensor. The rank of the tensor refers to the dimension in the tensor, that is, the number of axes (dimensions) of the tensor. The axis of the tensor is the specific dimension in the tensor. The shape of the tensor characterizes the length of each axis of the tensor, that is, the length of each dimension. For example, assuming that an input tensor is a 4th-order tensor, the input tensor has 4 axes (dimensions). Assuming that the lengths of the 4 dimensions are N, H, W, and C respectively, the shape of the input tensor is [N, H, W, C].

[0043] In the disclosed embodiment, the input tensor corresponding to the operator node is a multi-order tensor with multiple dimensions (axes), one of which is the batch dimension. The dimension length of the batch dimension is represented as the batch size or batch size (BatchSize). In neural networks, batch processing of samples (such as images) is usually required in some operations (operator nodes). The dimension length (batch size) of the batch dimension of the input tensor corresponding to the operator node represents the number of samples that the operator node needs to process in batches.

[0044] Exemplarily, the neural network is a convolutional neural network, and the input tensor of the operator node in the convolutional neural network is usually a 4th-order tensor with 4 dimensions, namely, the batch dimension, the image pixel height dimension, the image pixel width dimension, and the image color channel dimension. The tensor shape is usually expressed as [N, H, W, C]. Each indicator in the tensor shape represents a dimension, and the value of the indicator represents the dimension length of the dimension, where N represents the dimension length of the batch dimension (i.e., the batch size), that is, the number of image samples to be batched, H represents the image pixel height, that is, the number of pixels of the image in the vertical direction, W represents the image pixel width, that is, the number of pixels of the image in the horizontal direction, and C represents the number of color channels of the image (such as the number of color channels of an RGB color image C = 3).

[0045] Before step S11, the node information of each operator node in the original neural network calculation graph can be pre-configured, and the dimension length (i.e., batch size) of the input tensor corresponding to each operator node in the original neural network calculation graph in the batch processing dimension can be obtained in sequence along the execution order direction of multiple operator nodes in the original neural network calculation graph, so as to determine the operator node whose corresponding input tensor dimension length in the batch processing dimension is greater than 1 as the first operator node.

[0046] After determining all first operator nodes in the original neural network calculation graph, in step S11, for each first operator node, the first operator node is split into multiple target operator nodes according to the dimension length of the batch dimension of the input tensor corresponding to the first operator node. For example, if the dimension length of the batch dimension corresponding to the first operator node is 8, the first operator node can be split into 8 target operator nodes, each of which has a corresponding batch dimension length of 1. The operation type of each target operator node remains the same as the operation type of the corresponding first operator node, and the dimension length (batch size) of the batch dimension corresponding to each target operator node is smaller than the dimension length (batch size) of the batch dimension corresponding to the corresponding first operator node.

[0047] In step S12, when there are any multiple target operator nodes with the same dimension length of the corresponding batch processing dimension, and the first fusion condition is met, the multiple target operator nodes are merged into one target operator node. The merging of the multiple target operator nodes into one target operator node can be understood as retaining only any one target operator node among the multiple target operator nodes, while removing the other target operator nodes. Exemplarily, there are three target operator nodes with the dimension length of the corresponding batch processing dimension being 1, and the three target operator nodes meet the first fusion condition, then the three target operator nodes are merged into one target operator node with the dimension length of the corresponding batch processing dimension being 1.

[0048] In step S13, a target neural network calculation graph can be obtained based on all unsplit operator nodes and all remaining target operator nodes, as well as input connection relationships and output connection relationships. In the target neural network calculation graph, the input and output connection relationships between the unsplit operator nodes and the target operator nodes are the same as the input and output connection relationships between the corresponding operator nodes in the original neural network calculation graph, and the weight parameters required by the target operator nodes are the same as the weight parameters required by the corresponding first operator nodes, and the execution order of the target operator nodes is the same as the execution order of the corresponding first operator nodes, the order and axis of the input tensor corresponding to the target operator node remain unchanged, the dimension length of the batch dimension in the shape of the input tensor corresponding to the target operator node is different from the dimension length of the batch dimension in the shape of the input tensor corresponding to the corresponding first operator node, and the dimension lengths of other dimensions remain the same.

[0049] According to the technical solution of the processing method of the neural network calculation graph provided by the embodiment of the present disclosure, the processing method is used to process any suitable neural network calculation graph, and can optimize the original neural network calculation graph into a target neural network calculation graph composed of operator nodes corresponding to a lower batch size, so as to reduce the configuration requirements of the target neural network calculation graph, thereby effectively reducing the occupation of chip resources by the neural network calculation graph during compilation, thereby effectively reducing the compilation difficulty of the neural network calculation graph, improving the compilation efficiency and effect of the neural network calculation graph, and effectively reducing the requirements of the compilation of the neural network calculation graph for chip hardware storage resources, which is conducive to solving the problem that the storage resources required for the compilation of the neural network calculation graph are large but the actual chip hardware storage resources cannot meet them, realizing the rational use of chip hardware storage resources and improving the utilization efficiency of chip hardware storage resources. Moreover, the optimized target neural network calculation graph can effectively obtain the support of the underlying compilation execution, which is conducive to improving the compilation efficiency and effect of the neural network calculation graph. In practical applications, the neural network calculation graph is processed according to the method of the embodiment of the present disclosure. During actual compilation, the operator nodes in the original neural network calculation graph whose dimension length is n (n is greater than 1) corresponding to the batch dimension only need to correspond to one piece of configuration information, while in related technologies, the operator nodes in the unprocessed neural network calculation graph whose dimension length is n (n is greater than 1) corresponding to the batch dimension usually need to correspond to n pieces of configuration information. Therefore, compared with related technologies, the target neural network calculation graph is obtained by processing using the processing method of the embodiment of the present disclosure, which can effectively save the on-chip storage resources of the compiled multi-core chip during actual compilation.

[0050] In some embodiments, the processing method is used to process a neural network computation graph that meets processing conditions, where the processing conditions may include: multiple operator nodes in the neural network computation graph include at least one first operator node and at least one second operator node, the second operator node is an operator node whose dimension length of the batch processing dimension of the corresponding input tensor is equal to 1, and the first operator node is an operator node whose dimension length of the batch processing dimension of the corresponding input tensor is greater than 1. For neural network computation graphs that meet the above processing conditions, processing using the processing method of the embodiment of the present disclosure can effectively improve processing efficiency and is conducive to improving the compilation and operation effects of neural network computation graphs that meet the above processing conditions.

[0051] In some embodiments, before step S11, the processing method further includes: determining all first operator nodes and all second operator nodes in the neural network calculation graph based on the node information of each operator node in the neural network calculation graph; determining at least one target calculation subgraph in the neural network calculation graph based on the first operator node and the second operator node in the neural network calculation graph, the first operator node and the last operator node in the execution order of the target calculation subgraph are both second operator nodes, and there is at least one first operator node between the first operator node and the last operator node; and splitting the target calculation subgraph from the neural network calculation graph.

[0052] In some embodiments, each target computation subgraph is processed according to the processing manner of step S11 and step S12.

[0053] In some embodiments, in step S11, the dimension lengths of the batch dimensions corresponding to each target operator node are the same. In some embodiments, in step S11, the ratio of the dimension length of the batch dimension corresponding to the first operator node to the dimension length of the batch dimension corresponding to any target operator node is a positive integer. For example, if the dimension length of the batch dimension corresponding to the first operator node is 8, the first operator node can be split into 8 target operator nodes with a corresponding batch dimension length of 1, or split into 4 target operator nodes with a corresponding batch dimension length of 2, or split into 2 target operator nodes with a corresponding batch dimension length of 4.

[0054] In some embodiments, the dimension length of the batch processing dimension corresponding to each of the target operator nodes obtained by splitting is 1, which is conducive to enabling the optimized target neural network calculation graph to effectively obtain support from the underlying compilation execution.

[0055] In some embodiments, in step S12, the first fusion condition may include: the weight information corresponding to the multiple target operator nodes is the same, wherein the weight information corresponding to the target operator node includes pre-configured weight parameters required to implement the operation of the target operator node.

[0056] In some embodiments, when weight information exists for the multiple target operator nodes, it is determined whether the multiple target operator nodes meet the first fusion condition. When the multiple target operator nodes meet the first fusion condition, it indicates that the weight information corresponding to the multiple target operator nodes is the same. Therefore, the multiple target operator nodes can be merged into one target operator node, thereby reducing the configuration requirements of the target neural network calculation graph and reducing the chip resources occupied by the neural network calculation graph during compilation.

[0057] When the multiple target operator nodes do not meet the first fusion condition, it means that the weight information corresponding to the multiple target operator nodes is different and is not suitable for fusion. Therefore, the multiple target operator nodes may not be further processed.

[0058] In some embodiments, for a target operator node obtained by merging multiple target operator nodes that meet the first fusion condition, the dimension length of the batch dimension corresponding to the target operator node during compilation and runtime is specified through additional flag information.

[0059] By fusing multiple target operator nodes that meet the first fusion condition into one target operator node, in practical applications, the configuration of the weight information of the operator nodes can be effectively reduced, and only one copy of the same weight information needs to be stored on the compiled chip, thereby effectively saving the on-chip storage resources of the chip.

[0060] In some embodiments, for multiple target operator nodes with the same dimension length of corresponding batch dimensions, when there is no weight information on the multiple target operator nodes, the processing method may further include: determining whether the multiple target operator nodes meet the second fusion condition.

[0061] The second fusion condition includes: the computational operations corresponding to the multiple target operator nodes are the same, and no input-output dependency exists between any two of the multiple target operator nodes. It should be noted that input-output dependency describes the connection relationship between the inputs and outputs of different target operator nodes. When the input of one target operator node is connected to the output of another target operator node and depends on the output of the other target operator node, it indicates that an input-output dependency exists between the target operator node and the other target operator node.

[0062] When the multiple target operator nodes meet the second fusion condition, it means that the calculation operations corresponding to the multiple target operator nodes are the same, and there is no input-output dependency between any two target operator nodes among the multiple target operator nodes. Therefore, the multiple target operator nodes can be merged into one target operator node, thereby reducing the configuration requirements of the target neural network calculation graph and reducing the chip resources occupied by the neural network calculation graph during compilation.

[0063] When the multiple target operator nodes do not meet the above-mentioned second fusion condition, it means that the multiple target operator nodes correspond to different calculation operations or there are target operator nodes with input-output dependencies among the multiple target operator nodes, which are not suitable for fusion. Therefore, the multiple target operator nodes do not need to be further processed.

[0064] Since the dimension length of the batch dimension corresponding to the processed target operator node is shorter than the dimension length of the batch dimension corresponding to the first operator node, in order to be able to use the target operator node to achieve the same operation effect as the corresponding operator node, obtain the same calculation result as the corresponding operator node, and improve the operating efficiency of the neural network at the same time, in practical applications, the target neural network calculation graph can be compiled to the many-core system for execution, and the pipelined task processing mechanism of the many-core system can be used to pipeline the calculation tasks of each target operator node, thereby achieving the same operation effect as the corresponding operator node, obtaining the same calculation result as the corresponding operator node, and ensuring the operating efficiency of the neural network at the same time.

[0065] Figure 2 A flowchart of another method for processing a neural network calculation graph provided by an embodiment of the present disclosure is provided. Figure 2 In some embodiments, after obtaining the target neural network calculation graph, that is, after step S13, the processing method may further include: step S14.

[0066] Step S14: compile the target neural network computation graph into the many-core system.

[0067] In some embodiments, a many-core system may include one or more many-core chips. A many-core chip is a chip based on a many-core architecture that integrates storage and computing. Each many-core chip may include multiple physical cores (also called compute cores), each with independent memory. In some embodiments, a target neural network computation graph may be compiled and executed on one or more of the many-core chips in the many-core system.

[0068] In step S14, for the target neural network calculation graph obtained in step S13, an executable file corresponding to the target neural network calculation graph is generated based on the target neural network calculation graph. The executable file is an execution code that can be executed on the many-core chip of the many-core system, thereby compiling the target neural network calculation graph to run on the many-core system, so that the many-core system executes the corresponding computing tasks of the target neural network calculation graph.

[0069] It is understood that the above-mentioned various method embodiments mentioned in this disclosure can be combined with each other to form combined embodiments without violating the principle logic. Due to space limitations, this disclosure will not go into details. It is understood by those skilled in the art that in the above-mentioned methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0070] In addition, the present disclosure also provides a processing device, an electronic device, and a computer-readable storage medium. The above-mentioned processing device is used to implement the processing method of the neural network calculation graph provided by the present disclosure. The above-mentioned electronic device and computer-readable storage medium can both be used to implement the processing method of the neural network calculation graph provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part, which will not be repeated here.

[0071] Figure 3 A block diagram of a processing device according to an embodiment of the present disclosure is provided. Figure 3 The embodiment of the present disclosure provides a processing device 30, which is used to process the original neural network calculation graph to be processed. The original neural network calculation graph to be processed includes multiple operator nodes. The processing device 30 includes: an operator splitting module 31, an operator fusion module 32 and a calculation graph reconstruction module 33.

[0072] Among them, the operator splitting module 31 is used to split the first operator node into multiple target operator nodes according to the dimension length of the batch processing dimension of the input tensor corresponding to the first operator node for each first operator node. The first operator node is an operator node whose dimension length of the batch processing dimension of the corresponding input tensor is greater than 1, and the dimension length of the batch processing dimension corresponding to each target operator node is smaller than the dimension length of the batch processing dimension corresponding to the first operator node.

[0073] The operator fusion module 32 is configured to merge any multiple target operator nodes with the same dimension length of the corresponding batch processing dimension into one target operator node when the first fusion condition is met.

[0074] The computation graph reconstruction module 33 is used to obtain the target neural network computation graph based on the unsplit operator nodes and all target operator nodes.

[0075] In some embodiments, in the original neural network computation graph, the plurality of operator nodes include at least one first operator node and at least one second operator node, the second operator node being an operator node whose dimension length of the batch dimension of the corresponding input tensor is equal to 1.

[0076] In some embodiments, the first fusion condition includes: the weight information corresponding to the multiple target operator nodes is the same.

[0077] In some embodiments, the operator fusion module 32 is further configured to: if none of the multiple target operator nodes have weight information, determine whether the multiple target operator nodes meet a second fusion condition; and if the multiple target operator nodes meet the second fusion condition, merge the multiple target operator nodes into one target operator node. The second fusion condition includes: the computing operations corresponding to the multiple target operator nodes are the same, and there is no input-output dependency between any two target operator nodes in the multiple target operator nodes.

[0078] In some embodiments, the batch processing dimensions corresponding to each target operator node obtained by splitting have the same dimension length.

[0079] In some embodiments, the dimension length of the batch processing dimension corresponding to each target operator node obtained by splitting is 1.

[0080] In some embodiments, the processing device 30 further includes a compilation module, which is used to compile the target neural network computation graph into a many-core system, where the many-core system includes at least one many-core chip, and the many-core chip includes multiple physical cores.

[0081] The processing device provided in the embodiments of the present disclosure is used to implement the processing method provided in the above embodiments. For detailed description, please refer to the relevant description of the processing method in the above embodiments, which will not be repeated here.

[0082] Figure 4 This is a block diagram of an electronic device provided in an embodiment of the present disclosure, with reference to Figure 4 An embodiment of the present disclosure provides an electronic device, which includes: at least one processor 41; at least one memory 42, and one or more I / O interfaces 43, connected between the processor 41 and the memory 42; wherein the memory 42 stores one or more computer programs that can be executed by the at least one processor 41, and the one or more computer programs are executed by the at least one processor 41 to enable the at least one processor 41 to execute the above-mentioned neural network computation graph processing method.

[0083] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned method for processing a neural network computation graph. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0084] An embodiment of the present disclosure also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned method for processing a neural network computation graph.

[0085] It will be understood by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium).

[0086] As is well known to those skilled in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those skilled in the art, communication media typically contains computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0087] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0088] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0089] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0090] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0091] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0092] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0093] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0094] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly stated, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A method for processing a neural network computation graph, characterized in that: The original neural network calculation graph includes multiple operator nodes, and the processing method includes: For each first operator node, split the first operator node into multiple target operator nodes according to the dimension length of the batch dimension of the input tensor corresponding to the first operator node, where the first operator node is an operator node whose dimension length of the batch dimension of the corresponding input tensor is greater than 1, and the dimension length of the batch dimension corresponding to each of the target operator nodes is less than the dimension length of the batch dimension corresponding to the first operator node; When there are any multiple target operator nodes with the same dimension length of the corresponding batch processing dimension and the first fusion condition is met, the multiple target operator nodes are merged into one target operator node; Obtain a target neural network computation graph according to the unsplit operator nodes and all the target operator nodes; Compiling the target neural network computation graph into a many-core system, wherein the many-core system includes at least one many-core chip, and the many-core chip includes multiple physical cores; The first fusion condition includes: the weight information corresponding to the multiple target operator nodes is the same; the many-core chip stores the weight information corresponding to the operator nodes in the target neural network calculation graph, and the weight information corresponding to the merged target operator nodes in the target neural network calculation graph is the same as the weight information corresponding to the target operator nodes involved in the merger; The compiling of the target neural network computation graph into the many-core system includes: according to the multiple physical cores of the many-core chip, pipeline processing of the computation task of each target operator node in the target neural network computation graph is performed to obtain the same computation result as the first operator node corresponding to the target operator node.

2. The processing method according to claim 1, characterized in that The multiple operator nodes include at least one first operator node and at least one second operator node, where the second operator node is an operator node whose dimension length of the batch processing dimension of the corresponding input tensor is equal to 1.

3. The processing method according to claim 1, characterized in that The processing method further comprises: If none of the target operator nodes has weight information, determining whether the target operator nodes meet a second fusion condition; the second fusion condition includes: the computing operations corresponding to the target operator nodes are the same, and there is no input-output dependency relationship between any two target operator nodes in the target operator nodes; When the multiple target operator nodes meet the second fusion condition, the multiple target operator nodes are merged into one target operator node.

4. The processing method according to claim 1, characterized in that The batch processing dimensions corresponding to each target operator node obtained by splitting have the same dimension length.

5. The processing method according to claim 1, characterized in that The dimension length of the batch processing dimension corresponding to each target operator node obtained by splitting is 1.

6. A processing device, characterized in that The processing device is used to process a neural network calculation graph to be processed, wherein the neural network calculation graph includes a plurality of operator nodes, and the processing device includes: An operator splitting module is used to split each first operator node into multiple target operator nodes according to the dimension length of the batch processing dimension of the input tensor corresponding to the first operator node, where the first operator node is an operator node whose dimension length of the batch processing dimension of the corresponding input tensor is greater than 1, and the dimension length of the batch processing dimension corresponding to each of the target operator nodes is less than the dimension length of the batch processing dimension corresponding to the first operator node; An operator fusion module, configured to merge any multiple target operator nodes with the same dimension length of the corresponding batch processing dimension into one target operator node when the first fusion condition is met; A computational graph reconstruction module, configured to obtain a target neural network computational graph based on the unsplit operator nodes and all the target operator nodes; The processing device is further configured to: compile the target neural network computation graph into a many-core system, wherein the many-core system includes at least one many-core chip, and the many-core chip includes a plurality of physical cores; The first fusion condition includes: the weight information corresponding to the multiple target operator nodes is the same; the many-core chip stores the weight information corresponding to the operator nodes in the target neural network calculation graph, and the weight information corresponding to the merged target operator nodes in the target neural network calculation graph is the same as the weight information corresponding to the target operator nodes involved in the merger; The compiling of the target neural network computation graph into the many-core system includes: according to the multiple physical cores of the many-core chip, pipeline processing of the computation task of each target operator node in the target neural network computation graph is performed to obtain the same computation result as the first operator node corresponding to the target operator node.

7. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor. The one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the processing method according to any one of claims 1 to 5.

Images