Methods and apparatus for processing neural network computation graphs

Abstract

This disclosure provides a method, apparatus, electronic device, and computer-readable medium for processing neural network computation graphs. The processing method includes: obtaining the batch size of the input tensor corresponding to each original operator node in the neural network computation graph; performing a dimension length transformation on the batch processing dimension corresponding to each original operator node based on the batch size to obtain a target operator node corresponding to each original operator node; the dimension length of the batch processing dimension corresponding to each target operator node is less than the dimension length of the batch processing dimension corresponding to the corresponding original operator node; and obtaining a target neural network computation graph based on multiple target operator nodes. According to the technical solution of this disclosure, the neural network computation graph can be optimized, reducing the chip resource consumption of the neural network computation graph during compilation, improving the compilation efficiency and effect of the neural network computation graph, and reducing the compilation difficulty.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for processing neural network computation graphs, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Many-core architecture chips based on in-memory computing reduce data transfer time and power consumption by placing both computing and storage on-chip, making it an important development direction for many-core chips.

[0003] Deep learning frameworks (such as TensorFlow or ONNX) typically use computation graphs to represent the computations of deep learning models (neural networks). Neural network computation graphs usually need to be compiled by a compiler to generate a stream of instructions (executable files or code) that can run on hardware. This hardware can be based on in-memory computing many-core chips, which typically consist of multiple physical cores. Summary of the Invention

[0004] This disclosure provides a method and apparatus for processing neural network computation graphs, an electronic device, and a computer-readable storage medium.

[0005] In a first aspect, this disclosure provides a method for processing a neural network computation graph, wherein the original neural network computation graph includes multiple original operator nodes, and the method includes:

[0006] Obtain the batch size of the input tensor corresponding to each original operator node in the original neural network computation graph, wherein the batch size of the input tensor is the length of the batch processing dimension of the input tensor;

[0007] The batch processing dimension corresponding to each original operator node is transformed according to the batch size corresponding to each original operator node to obtain the target operator node corresponding to each original operator node; the dimension length of the batch processing dimension corresponding to each target operator node is less than the dimension length of the batch processing dimension corresponding to the corresponding original operator node.

[0008] Based on the multiple target operator nodes, obtain the target neural network computation graph.

[0009] Secondly, this disclosure provides a processing apparatus for processing a neural network computation graph to be processed, the neural network computation graph including a plurality of primitive operator nodes, the processing apparatus comprising:

[0010] The dimension acquisition module is used to acquire the batch size of the input tensor corresponding to each original operator node in the original neural network computation graph, wherein the batch size of the input tensor is the dimension length of the batch processing dimension of the input tensor;

[0011] The dimension transformation module is used to transform the batch processing dimension corresponding to each original operator node according to the batch size corresponding to each original operator node, so as to obtain the target operator node corresponding to each original operator node; the dimension length of the batch processing dimension corresponding to each target operator node is less than the dimension length of the batch processing dimension corresponding to the corresponding original operator node;

[0012] The computation graph reconstruction module is used to obtain the computation graph of the target neural network based on multiple target operator nodes.

[0013] Thirdly, this disclosure provides an electronic device comprising:

[0014] At least one processor;

[0015] and a memory communicatively connected to the at least one processor;

[0016] The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the above-described neural network computation graph processing method.

[0017] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described method for processing neural network computation graphs.

[0018] According to the technical solution of the neural network computation graph processing method provided in the embodiments of this disclosure, the processing method is used to process any suitable neural network computation graph. It can optimize the original neural network computation graph into a target neural network computation graph composed of target operator nodes with corresponding lower batch sizes, thereby reducing the configuration requirements of the target neural network computation graph. This effectively reduces the chip resource occupation of the neural network computation graph during compilation, thereby effectively reducing the compilation difficulty of the neural network computation graph, improving the compilation efficiency and effect of the neural network computation graph, and effectively reducing the requirements of the compilation of the neural network computation graph on the chip hardware storage resources. This helps to solve the problem that the storage resources required for the compilation of the neural network computation graph are large but the actual chip hardware storage resources cannot meet them, realizing the rational utilization of chip hardware storage resources and improving the utilization efficiency of chip hardware storage resources.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0021] Figure 1 A flowchart illustrating a method for processing a neural network computation graph provided in an embodiment of this disclosure;

[0022] Figure 2 for Figure 1 A flowchart illustrating a specific implementation of step S12;

[0023] Figure 3 This is a schematic diagram of the structure of a primitive neural network computation graph;

[0024] Figure 4 To Figure 3 A schematic diagram of the structure of the target neural network computation graph obtained by processing the neural network computation graph shown;

[0025] Figure 5 A flowchart illustrating another method for processing a neural network computation graph provided in this embodiment of the present disclosure;

[0026] Figure 6 This is a block diagram of a processing apparatus provided in an embodiment of the present disclosure;

[0027] Figure 7 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0029] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0030] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0031] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0032] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0033] In related technologies, chips have limited computing and storage resources. The computational and data volume required for large neural network computation graphs is usually large. The computing and storage resources of chips are usually unable to meet the resource requirements of the entire neural network computation graph, resulting in high compilation difficulty and low efficiency of neural network computation graphs.

[0034] Therefore, this disclosure provides a method and apparatus for processing neural network computation graphs, an electronic device, and a computer-readable storage medium, which are intended to effectively solve at least one of the technical problems existing in the above-mentioned related technologies.

[0035] The processing method of this disclosure can be executed by a processing device, which can be integrated into an electronic device such as a terminal device or a server through software and / or hardware. For example, the terminal device can be an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, wearable device, etc. In some embodiments, the processing method of this disclosure can be implemented by a processor calling computer-readable program instructions stored in memory, or the processing method of this disclosure can be executed by a server.

[0036] Figure 1 This is a flowchart illustrating a method for processing a neural network computation graph, as provided in an embodiment of this disclosure.

[0037] This disclosure provides a method for processing a neural network computation graph. This method processes the original neural network computation graph to optimize its graph structure and obtain a target neural network computation graph. The original neural network computation graph may include multiple original operator nodes, which are the basic computational units constituting the neural network. These original operator nodes can be, for example, operations such as convolution and pooling in the neural network. The neural network can be any type of deep learning network and can be used to perform any of the following tasks: image processing, speech processing, text processing, and video processing. The input data of the neural network can be any of the following: image data, speech data, text data, and video data.

[0038] Reference Figure 1 The processing method may include steps S11 to S13.

[0039] Step S11: Obtain the batch size of the input tensor corresponding to each original operator node in the original neural network computation graph. The batch size of the input tensor is the length of the batch processing dimension of the input tensor.

[0040] Step S12: Based on the batch size corresponding to each original operator node, perform dimension length transformation on the batch processing dimension corresponding to each original operator node to obtain the target operator node corresponding to the original operator node. The dimension length of the batch processing dimension corresponding to each target operator node is less than the dimension length of the batch processing dimension corresponding to the original operator node.

[0041] Step S13: Obtain the target neural network computation graph based on multiple target operator nodes.

[0042] In this embodiment of the disclosure, for a raw neural network computation graph to be processed, before obtaining the batch size of the input tensor corresponding to each raw operator node in the raw neural network computation graph, i.e. before step S11, the processing method further includes: traversing the entire raw neural network computation graph to obtain the node information of each raw operator node in the raw neural network computation graph.

[0043] The node information of the original operator node may include, but is not limited to: the input connection relationship, output connection relationship, required parameter information, attribute information of the original operator node, execution order of the original operator node, and tensor attribute information of the input tensor corresponding to the original operator node.

[0044] The input connection relationship of the original operator node describes the connection relationship between the input of the original operator node and the output of other original operator nodes in the neural network computation graph; the output connection relationship of the original operator node describes the connection relationship between the output of the original operator node and the input of other original operator nodes in the neural network computation graph; the parameter information required by the original operator node includes, but is not limited to, the pre-configured weight parameters required to implement the operation of the original operator node; the attribute information of the original operator node is the information representing the feature attributes of the original operator node, which may include, but is not limited to, the type of the original operator node (such as convolution, addition, multiplication, etc.), the amount of computation and storage required by the original operator node, etc.; the execution order of the original operator node is the temporal order of the operation of the original operator node; the tensor attribute information of the input tensor corresponding to the original operator node is the information representing the feature attributes of the input tensor, which may include, but is not limited to: the order, axes, and shape of the input tensor. The order of the tensor refers to the dimension in the tensor, that is, the number of axes (dimensions) of the tensor. The axes of the tensor are the specific dimensions in the tensor. The shape of the tensor represents the length of each axis of the tensor, that is, the length of each dimension. For example, suppose an input tensor is a 4th order tensor. This input tensor has 4 axes (dimensions). If the lengths of these 4 dimensions are N, H, W, and C respectively, then the shape of the input tensor is [N, H, W, C].

[0045] In this embodiment, the input tensor corresponding to the original operator node is a multi-order tensor with multiple dimensions (axes), one of which is the batch dimension. The length of the batch dimension is represented by the batch size. In neural networks, some operations (original operator nodes) often require batch processing of samples (such as images). The length of the batch dimension (batch size) of the input tensor corresponding to the original operator node represents the number of samples that the original operator node needs to process in batches.

[0046] For example, the neural network is a convolutional neural network. The input tensor of the operator node in a convolutional neural network is usually a 4th-order tensor with four dimensions: batch size, image pixel height, image pixel width, and image color channels. The tensor shape is usually represented as [N, H, W, C]. Each index in the tensor shape represents a dimension, and the value of the index represents the length of that dimension. Here, N represents the length of the batch size (i.e., the batch size), which is the number of image samples to be processed in batches; H represents the image pixel height, which is the number of pixels in the vertical direction of the image; W represents the image pixel width, which is the number of pixels in the horizontal direction of the image; and C represents the number of color channels of the image (e.g., the number of color channels of an RGB color image is C = 3).

[0047] In step S11, the node information of each original operator node in the original neural network computation graph can be pre-configured, and the dimension length (i.e., batch size) of the input tensor corresponding to each original operator node in the original neural network computation graph can be obtained sequentially along the execution order direction of multiple original operator nodes in the original neural network computation graph.

[0048] In step S12, each original operator node is processed according to its batch size. Specifically, the batch processing dimension of the input tensor of each original operator node is transformed while keeping the lengths of other dimensions unchanged, to obtain the target operator node corresponding to each original operator node, i.e., the original operator node is converted into a target operator node. The operation type of each target operator node is the same as the operation type of the corresponding original operator node, and the length (batch size) of the batch processing dimension corresponding to each target operator node is less than the length (batch size) of the batch processing dimension corresponding to the corresponding original operator node.

[0049] In step S13, the target neural network computation graph can be obtained based on the target operator node corresponding to each original operator node, as well as the input and output connection relationships. Specifically, in the target neural network computation graph, the input and output connection relationships between the target operator nodes are identical to those between the corresponding original operator nodes in the original neural network computation graph. Furthermore, the weight parameters required by the target operator nodes are the same as those required by the corresponding original operator nodes, and the attribute information and execution order of the target operator nodes are identical to those of the corresponding original operator nodes. The order and axes of the input tensor corresponding to the target operator node remain unchanged. The length of the batch processing dimension in the shape of the input tensor corresponding to the target operator node differs from the length of the batch processing dimension in the shape of the input tensor corresponding to the corresponding original operator node, while the lengths of other dimensions remain the same.

[0050] According to the technical solution of the neural network computation graph processing method provided in the embodiments of this disclosure, the processing method is used to process any suitable neural network computation graph. It can optimize the original neural network computation graph into a target neural network computation graph composed of target operator nodes with corresponding lower batch sizes, thereby reducing the configuration requirements of the target neural network computation graph. This effectively reduces the chip resource occupation of the neural network computation graph during compilation, thereby effectively reducing the compilation difficulty of the neural network computation graph, improving the compilation efficiency and effect of the neural network computation graph, and effectively reducing the requirements of the compilation of the neural network computation graph on the chip hardware storage resources. This helps to solve the problem that the storage resources required for the compilation of the neural network computation graph are large but the actual chip hardware storage resources cannot meet them, realizing the rational utilization of chip hardware storage resources and improving the utilization efficiency of chip hardware storage resources. In practical applications, the method of this disclosure is used to process the original neural network computation graph. For operator nodes in the original neural network computation graph with a batch processing dimension length of n (n is greater than 1), since they have been processed and optimized to target operator nodes corresponding to lower batch sizes, it is not necessary to configure n sets of configuration information during actual compilation, but only a number of configuration information less than n can be configured. In contrast, in related technologies, operator nodes with a batch processing dimension length of n (n is greater than 1) in the unprocessed neural network computation graph usually require n sets of configuration information. Therefore, compared with related technologies, the target neural network computation graph obtained by the processing method of this disclosure can effectively save on-chip storage resources of the compiled many-core chip during actual compilation.

[0051] In some embodiments, the original operator nodes in the neural network computation graph are connected in a serial manner, that is, any two original operator nodes are connected in a serial manner. The above processing method is applicable to processing neural network computation graphs in which the original operator nodes are connected in a serial manner.

[0052] In some embodiments, the processing method is used to process a neural network computation graph that meets certain processing conditions. These conditions may include: the batch size of the input tensor corresponding to each original operator node in the neural network computation graph is greater than 1, and the ratio of the first batch size to the second batch size of any two original operator nodes is a positive integer; wherein the first batch size is the batch size corresponding to one of the two original operator nodes, the second batch size is the batch size corresponding to the other original operator node, and the first batch size is greater than or equal to the second batch size. For neural network computation graphs that meet the above processing conditions, processing using the method of this embodiment can effectively improve processing efficiency and enhance the compilation and execution performance of neural network computation graphs that meet the above processing conditions.

[0053] In some embodiments, obtaining the batch size of the input tensor corresponding to each original operator node in the original neural network computation graph in step S11 may further include: determining the length of the batch processing dimension of the input tensor corresponding to the original operator node based on the tensor shape of the input tensor corresponding to the original operator node, so as to obtain the batch size of the input tensor. For example, if the tensor shape of the input tensor corresponding to the original operator node is [N,H,W,C], it can be known that the first dimension in the tensor shape is the batch processing dimension, and the dimension length is N.

[0054] Figure 2 for Figure 1 A flowchart illustrating a specific implementation of step S12 is shown in some embodiments. Figure 2 In step S12, the batch processing dimension corresponding to each original operator node is transformed according to the batch size corresponding to each original operator node to obtain the target operator node corresponding to each original operator node. This may further include steps S21 to S22.

[0055] Step S21: Determine the smallest batch size among all the batch sizes corresponding to the original operator nodes.

[0056] Step S22: For each original operator node, based on the ratio of the batch size to the minimum batch size corresponding to the original operator node, perform a dimension length transformation on the batch processing dimension corresponding to the original operator node to obtain the target operator node corresponding to the original operator node.

[0057] The length of the batch dimension corresponding to the target operator node is the ratio of the batch size corresponding to the original operator node to the minimum batch size.

[0058] Figure 3 This is a schematic diagram of the structure of a primitive neural network computation graph, referencing... Figure 3 For example, the original neural network computation graph includes original operator nodes 1 to 9, a total of 9 original operator nodes. The batch size of the input tensor corresponding to original operator nodes 1, 2, 8, and 9 is 3, and the batch size of the input tensor corresponding to original operator nodes 3 to 7 is 24. Therefore, the minimum batch size is 3.

[0059] Figure 4 To Figure 3 A schematic diagram of the structure of the target neural network computation graph obtained by processing the neural network computation graph shown, referencing... Figure 4 For example, the above processing method is used to... Figure 3 The neural network computation graph shown is processed to obtain... Figure 4The diagram shows the computational graph of the target neural network consisting of target operator node a to target operator node i.

[0060] Reference Figure 3 and Figure 4 Using step S22 above, the original operator node 1 is processed, and the dimension length (batch size) of the batch processing dimension corresponding to the original operator node 1 is modified to the ratio of the batch size corresponding to the original operator node 1 to the minimum batch size, to obtain the target operator node a. The dimension length (batch size) of the batch processing dimension corresponding to the target operator node a is the ratio of the batch size corresponding to the original operator node 1 to the minimum batch size, and this ratio is 3 / 3 = 1. Similarly, using step S22 above, the original operator node 2 is processed to obtain the target operator node b. The dimension length (batch size) of the batch processing dimension corresponding to the target operator node b is 3 / 3 = 1. Similarly, using step S22 above, the original operator node 3 is processed to obtain the target operator node c. The batch processing dimension of the target operator node c is... The batch size (length) of the processing dimension is 24 / 3 = 8. Similarly, the original operator nodes 4 to 9 are processed to obtain the target operator nodes d to i. The batch size (length) of the processing dimension corresponding to the target operator node d is 24 / 3 = 8, the batch size (length) of the processing dimension corresponding to the target operator node e is 24 / 3 = 8, the batch size (length) of the processing dimension corresponding to the target operator node f is 24 / 3 = 8, the batch size (length) of the processing dimension corresponding to the target operator node g is 24 / 3 = 8, the batch size (length) of the processing dimension corresponding to the target operator node h is 3 / 3 = 1, and the batch size (length) of the processing dimension corresponding to the target operator node i is 3 / 3 = 1.

[0061] Because the batch dimension length corresponding to the target operator node is shorter than that corresponding to the original operator node, in order to achieve the same computational effect and obtain the same computational result as the original operator node while improving the running efficiency of the neural network, the target neural network computation graph can be compiled into a many-core system in practical applications. The pipelined task processing mechanism of the many-core system can then be used to process the computational tasks of each target operator node in a pipelined manner, thereby achieving the same computational effect and obtaining the same computational result as the original operator node, while ensuring the running efficiency of the neural network. Based on the above... Figure 3 The original operator node 1 and Figure 4Taking the target operator node a as an example, the batch processing dimension length corresponding to the original operator node 1 is 3, while the batch processing dimension length corresponding to the optimized target operator node a is 1. After compiling the target operator node a into the many-core chip, during runtime, three input data (such as an image) can be continuously input into the target operator node a, one input data at a time, and the process can be repeated three times to perform pipelined processing. This achieves the same computational operation effect as the corresponding original operator node 1 and obtains the same calculation result as the corresponding original operator node 1, while improving the running efficiency of the neural network.

[0062] Figure 5 A flowchart illustrating another method for processing neural network computation graphs provided in this disclosure is shown below. Figure 5 In some embodiments, after obtaining the target neural network computation graph, i.e. after step S13, the processing method may further include step S14.

[0063] Step S14: Compile the target neural network computation graph into the many-core system.

[0064] In some embodiments, a many-core system may include one or more many-core chips, which are chips based on a memory-based many-core architecture. Each many-core chip may include multiple physical cores (also called computing cores), and each physical core has independent memory. In some embodiments, the target neural network computation graph can be compiled to run on one or more many-core chips in the many-core system.

[0065] In step S14, for the target neural network computation graph obtained in step S13, an executable file corresponding to the target neural network computation graph is generated. The executable file is executable code that can be executed on the many-core chip of the many-core system, thereby enabling the target neural network computation graph to be compiled and run on the many-core system so that the many-core system can execute the corresponding computation tasks of the target neural network computation graph.

[0066] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0067] In addition, this disclosure also provides a processing apparatus, an electronic device, and a computer-readable storage medium. The processing apparatus is used to implement the neural network computation graph processing method provided by this disclosure. The electronic device and the computer-readable storage medium can both be used to implement the neural network computation graph processing method provided by this disclosure. The corresponding technical solutions and descriptions are described in the corresponding descriptions in the method section, and will not be repeated here.

[0068] Figure 6 This is a block diagram of a processing apparatus provided in an embodiment of the present disclosure, with reference to... Figure 6 This disclosure provides a processing device 60 for processing an original neural network computation graph to be processed. The original neural network computation graph to be processed includes multiple original operator nodes. The processing device 60 includes: a dimension acquisition module 61, a dimension transformation module 62, and a computation graph reconstruction module 63.

[0069] The dimension acquisition module 61 is used to acquire the batch size of the input tensor corresponding to each original operator node in the original neural network computation graph. The batch size of the input tensor is the dimension length of the batch processing dimension of the input tensor.

[0070] The dimension transformation module 62 is used to transform the batch processing dimension corresponding to each original operator node according to the batch size corresponding to each original operator node, so as to obtain the target operator node corresponding to each original operator node; the dimension length of the batch processing dimension corresponding to each target operator node is less than the dimension length of the batch processing dimension corresponding to the corresponding original operator node.

[0071] The computation graph reconstruction module 63 is used to obtain the computation graph of the target neural network based on multiple target operator nodes.

[0072] In some embodiments, the batch size of the input tensor corresponding to each original operator node is greater than 1, and the ratio of the first batch size to the second batch size of any two original operator nodes is a positive integer; wherein, the first batch size is the batch size corresponding to one of the two original operator nodes, the second batch size is the batch size corresponding to the other original operator node, and the first batch size is greater than or equal to the second batch size.

[0073] In some embodiments, the dimension transformation module 62 includes: a determination submodule, configured to determine the minimum batch size among all batch sizes corresponding to the original operator nodes; and a transformation submodule, configured to, for each original operator node, perform a dimension length transformation on the batch processing dimension corresponding to the original operator node based on the ratio of the batch size corresponding to the original operator node to the minimum batch size, to obtain the target operator node corresponding to the original operator node. Wherein, the dimension length of the batch processing dimension corresponding to the target operator node is the ratio of the batch size corresponding to the original operator node to the minimum batch size.

[0074] In some embodiments, the dimension acquisition module 61 is used to determine the dimension length of the batch processing dimension of the input tensor corresponding to the original operator node based on the tensor shape of the input tensor corresponding to the original operator node, so as to obtain the batch size of the input tensor.

[0075] In some embodiments, the processing device 60 further includes a compilation module for compiling a target neural network computation graph into a many-core system, the many-core system including at least one many-core chip, and the many-core chip including multiple physical cores.

[0076] The processing apparatus provided in this disclosure is used to implement the processing method provided in the above embodiments. For a detailed description, please refer to the relevant description in the processing method of the above embodiments, which will not be repeated here.

[0077] Figure 7 This is a block diagram of an electronic device provided in an embodiment of the present disclosure, with reference to... Figure 7 This disclosure provides an electronic device, which includes: at least one processor 71; at least one memory 72; and one or more I / O interfaces 73 connected between the processor 71 and the memory 72; wherein the memory 72 stores one or more computer programs that can be executed by at least one processor 71, and the one or more computer programs are executed by at least one processor 71 to enable at least one processor 71 to perform the above-described neural network computation graph processing method.

[0078] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the aforementioned neural network computation graph processing method. The computer-readable storage medium may be volatile or non-volatile.

[0079] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the above-described neural network computation graph processing method.

[0080] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0081] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0082] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0083] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0084] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0085] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0086] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0087] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0088] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0089] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A method for processing neural network computation graphs, characterized in that, The original neural network computation graph includes multiple original operator nodes, and the method includes: Obtain the batch size of the input tensor corresponding to each original operator node in the original neural network computation graph, wherein the batch size of the input tensor is the length of the batch processing dimension of the input tensor; The batch processing dimension corresponding to each original operator node is transformed according to the batch size corresponding to each original operator node to obtain the target operator node corresponding to each original operator node; the dimension length of the batch processing dimension corresponding to each target operator node is less than the dimension length of the batch processing dimension corresponding to the corresponding original operator node. Based on the multiple target operator nodes, obtain the target neural network computation graph; After obtaining the target neural network computation graph, the processing method further includes: The target neural network computation graph is compiled into a many-core system, the many-core system including at least one many-core chip, and the many-core chip including multiple physical cores; The step of compiling the target neural network computation graph into the many-core system includes: for each target operator node in the target neural network computation graph, performing pipelined processing on the computation task of the target operator node according to the physical core of the many-core chip to obtain the same computation result as the original operator node corresponding to the target operator node.

2. The processing method according to claim 1, characterized in that, The batch size of the input tensor corresponding to each of the original operator nodes is greater than 1, and the ratio of the first batch size to the second batch size in the batch sizes corresponding to any two original operator nodes is a positive integer. Wherein, the first batch size is the batch size corresponding to one of the two original operator nodes, the second batch size is the batch size corresponding to the other original operator node, and the first batch size is greater than or equal to the second batch size.

3. The processing method according to claim 2, characterized in that, The step of performing a dimension length transformation on the batch processing dimension corresponding to each original operator node based on the batch size corresponding to each original operator node to obtain the target operator node corresponding to each original operator node includes: Determine the smallest batch size among all the batch sizes corresponding to the original operator nodes; For each of the original operator nodes, the batch processing dimension corresponding to the original operator node is transformed according to the ratio of the batch size corresponding to the original operator node to the minimum batch size to obtain the target operator node corresponding to the original operator node. Wherein, the length of the batch processing dimension corresponding to the target operator node is the ratio of the batch size corresponding to the original operator node to the minimum batch size.

4. The processing method according to claim 1, characterized in that, The step of obtaining the batch size of the input tensor corresponding to each original operator node in the original neural network computation graph includes: Based on the tensor shape of the input tensor corresponding to the original operator node, the length of the batch processing dimension of the input tensor corresponding to the original operator node is determined to obtain the batch size of the input tensor.

5. The processing method according to any one of claims 1-4, characterized in that, The original operator nodes in the neural network computation graph are connected in a serial manner.

6. A processing apparatus, characterized in that, The processing device is used to process the neural network computation graph to be processed, the neural network computation graph including multiple primitive operator nodes, and the processing device includes: The dimension acquisition module is used to acquire the batch size of the input tensor corresponding to each original operator node in the original neural network computation graph, wherein the batch size of the input tensor is the dimension length of the batch processing dimension of the input tensor; The dimension transformation module is used to transform the batch processing dimension corresponding to each original operator node according to the batch size corresponding to each original operator node, so as to obtain the target operator node corresponding to each original operator node; the dimension length of the batch processing dimension corresponding to each target operator node is less than the dimension length of the batch processing dimension corresponding to the corresponding original operator node; The computation graph reconstruction module is used to obtain the computation graph of the target neural network based on multiple target operator nodes; After acquiring the target neural network computation graph, the processing device is further configured to: The target neural network computation graph is compiled into a many-core system, the many-core system including at least one many-core chip, and the many-core chip including multiple physical cores; The step of compiling the target neural network computation graph into the many-core system includes: for each target operator node in the target neural network computation graph, performing pipelined processing on the computation task of the target operator node according to the physical core of the many-core chip to obtain the same computation result as the original operator node corresponding to the target operator node.

7. The processing apparatus according to claim 6, characterized in that, The batch size of the input tensor corresponding to each of the original operator nodes is greater than 1, and the ratio of the first batch size to the second batch size in the batch sizes corresponding to any two original operator nodes is a positive integer. Wherein, the first batch size is the batch size corresponding to one of the two original operator nodes, the second batch size is the batch size corresponding to the other original operator node, and the first batch size is greater than or equal to the second batch size.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the processing method as described in any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the processing method as described in any one of claims 1-5.

Images