Data processing method, device, computer equipment and storage medium

Abstract

The present disclosure relates to a data processing method, apparatus, computer equipment and storage medium. The product includes a processor, and the processor includes multiple processing units and a storage unit. The multiple processing units are used to execute instruction sequences such as updating calculation graphs and processing data. The storage unit is used to store data and may include a random access memory (RAM) and a register file. The multiple processing units in the processor can share part of the storage space, such as sharing part of the RAM storage space and the register file, and can also have their own storage space at the same time. By using the above processor, the present disclosure can improve the computing efficiency of related products when performing operations on neural network models.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the advancement of computer technology, dataflow programming has emerged. For ease of programming and flexibility, fine-grained operators are used to simulate computations and stitch them together into deep neural networks. This means that inference within a deep neural network requires executing hundreds or even thousands of nodes sequentially. Each operator execution requires the processor to call its kernel function and copy data from global memory to the chip. This incurs performance overhead, in addition to computational overhead, including data copying between nodes and kernel launches. To reduce this overhead, these fine-grained nodes, all executable on the same device, are often fused into a single, larger fused node before actual computation. Only this fused node needs to be executed during execution, reducing the number of kernel function calls and the data copy overhead within the fused node. However, some nodes in the computation graph may not support fusion, resulting in fragmented fusion and reduced optimization effectiveness. Summary of the Invention

[0003] Based on this, it is necessary to provide a data processing method, device, computer equipment and storage medium that can improve computing efficiency in response to the above technical problems.

[0004] According to one aspect of the present disclosure, a data processing method is provided, including: determining, based on the types of multiple nodes in a computation graph, a first constant node that does not support fusion in the computation graph; performing constant elimination processing on the first constant node to obtain an elimination result; and updating the computation graph based on the elimination result to obtain an updated computation graph.

[0005] In a possible implementation, performing constant elimination processing on the first constant node to obtain an elimination result includes: removing the first constant node, an input edge, and an output edge of the first constant node to obtain the elimination result.

[0006] In one possible implementation, the first constant node includes a third constant node, wherein the node pointed to by the output edge of the third constant node includes a non-constant node, and removing the first constant node and the input edge and output edge of the first constant node includes: obtaining output information of the third constant node based on the data to be processed input into the calculation graph; saving the output information of the third constant node; and removing the third constant node and the input edge and output edge of the third constant node.

[0007] In one possible implementation, the first constant node includes a second constant node, wherein the nodes pointed to by the output edges of the second constant node are all the first constant nodes, and removing the first constant node and the input edges and output edges of the first constant node includes: directly removing the second constant node and the input edges and output edges of the second constant node.

[0008] In one possible implementation, the first constant node includes a second constant node and a third constant node, and removing the first constant node and the input edge and the output edge of the first constant node includes: simultaneously removing the second constant node and the third constant node, as well as the input edge and the output edge of the second constant node and the third constant node.

[0009] In one possible implementation, determining the first constant node that does not support fusion in the computation graph based on the types of multiple nodes in the computation graph includes: determining a first target node of a preset type among the multiple nodes in the computation graph based on the types of multiple nodes in the computation graph; performing constant discrimination processing on the output node of the first target node to obtain a second target node whose input information is a constant, the output node of the first target node including a direct output node and an indirect output node, the direct output node including a node directly pointed to by the output edge of the first target node, and the indirect output node including a node indirectly pointed to by the output edge of the first target node via other nodes; obtaining the first constant node based on the first target node and the second target node.

[0010] In one possible implementation, constant discrimination processing is performed on the output node of the first target node to obtain a second target node whose input information is a constant, including: determining whether the input information of the i-th direct output node of the first target node includes only constant information, where i is a positive integer; if the input information of the i-th direct output node of the first target node includes only constant information, determining the i-th direct output node of the first target node as the second target node, and determining whether the input information of the j-th indirect output node of the first target node corresponding to the i-th direct output node of the first target node includes only constant information, where j is a positive integer; if the input information of the j-th indirect output node of the first target node includes only constant information, determining the j-th indirect output node of the first target node as the second target node.

[0011] In one possible implementation, constant discrimination processing is performed on the output node of the first target node to obtain a second target node whose input information is a constant, and the method further includes: when the i-th direct output node of the first target node or the j-th indirect output node of the first target node includes non-constant information, stopping constant discrimination processing on the remaining indirect output nodes corresponding to the i-th direct output node of the first target node or the j-th indirect output node of the first target node, wherein the remaining indirect output nodes include indirect output nodes that have not been subjected to constant discrimination processing among the indirect output nodes corresponding to the i-th direct output node of the first target node or the j-th indirect output node of the first target node.

[0012] In a possible implementation, the data to be processed includes at least one of images, videos, voices, and texts.

[0013] According to one aspect of the present disclosure, a data processing device is provided, including: a determination module for determining, based on the types of multiple nodes in a computation graph, a first constant node that does not support fusion in the computation graph; an elimination module for performing constant elimination processing on the first constant node to obtain an elimination result; and an update module for updating the computation graph according to the elimination result to obtain an updated computation graph.

[0014] In a possible implementation, the elimination module is further configured to remove the first constant node, the input edge, and the output edge of the first constant node to obtain the elimination result.

[0015] In one possible implementation, the first constant node includes a third constant node, wherein the nodes pointed to by the output edge of the third constant node include non-constant nodes, and the elimination module is further used to: obtain output information of the third constant node based on the data to be processed input into the calculation graph; save the output information of the third constant node; and remove the third constant node, the input edge, and the output edge of the third constant node.

[0016] In one possible implementation, the first constant node includes a second constant node, wherein the nodes pointed to by the output edges of the second constant node are all the first constant nodes, and the elimination module is further used to: directly remove the second constant node, the input edges and the output edges of the second constant node.

[0017] In a possible implementation, the first constant node includes a second constant node and a third constant node, and the elimination module is further used to: simultaneously remove the second constant node and the third constant node, as well as the input edges and output edges of the second constant node and the third constant node.

[0018] In one possible implementation, the determination module is further used to: determine a first target node of a preset type among multiple nodes in the computational graph based on the types of multiple nodes in the computational graph; perform constant discrimination processing on the output node of the first target node to obtain a second target node whose input information is a constant, the output node of the first target node including a direct output node and an indirect output node, the direct output node including the node directly pointed to by the output edge of the first target node, and the indirect output node including the node indirectly pointed to by the output edge of the first target node via other nodes; obtain the first constant node based on the first target node and the second target node.

[0019] In one possible implementation, the determination module is further used to: determine whether the input information of the i-th direct output node of the first target node includes only constant information, where i is a positive integer; if the input information of the i-th direct output node of the first target node includes only constant information, determine the i-th direct output node of the first target node as the second target node, and determine whether the input information of the j-th indirect output node of the first target node corresponding to the i-th direct output node of the first target node includes only constant information, where j is a positive integer; if the input information of the j-th indirect output node of the first target node includes only constant information, determine the j-th indirect output node of the first target node as the second target node.

[0020] In one possible implementation, the determination module is further used to: when the i-th direct output node of the first target node or the j-th indirect output node of the first target node includes non-constant information, stop performing constant discrimination processing on the remaining indirect output nodes corresponding to the i-th direct output node of the first target node or the j-th indirect output node of the first target node, and the remaining indirect output nodes include indirect output nodes that have not been subjected to constant discrimination processing among the indirect output nodes corresponding to the i-th direct output node of the first target node or the j-th indirect output node of the first target node.

[0021] In a possible implementation, the data to be processed includes at least one of images, videos, voices, and texts.

[0022] According to one aspect of the present disclosure, an artificial intelligence chip is provided, comprising the data processing device.

[0023] According to one aspect of the present disclosure, an electronic device is provided, comprising the artificial intelligence chip.

[0024] According to one aspect of the present disclosure, a board is provided, comprising: a storage device, an interface device, a control device, and the artificial intelligence chip; wherein the artificial intelligence chip is connected to the storage device, the control device, and the interface device, respectively; the storage device is used to store data; the interface device is used to realize data transmission between the artificial intelligence chip and an external device; and the control device is used to monitor the status of the artificial intelligence chip.

[0025] According to one aspect of the present disclosure, a computer device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the data processing method.

[0026] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the data processing method when executed by a processor.

[0027] According to the embodiments of the present disclosure, it is possible to solve the problem that certain types of nodes do not support fusion, which leads to fusion segmentation and poor fusion effect. By eliminating the first constant node that does not support fusion to optimize the calculation graph, the calculation graph segmentation will be reduced, the fusion effect will be improved, and the processing overhead of data call and transmission will be reduced, the computational efficiency of the calculation graph will be improved, and the computing performance of the processor will be brought into play.

[0028] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0030] Figure 1 A schematic diagram illustrating a processor of a data processing method according to an embodiment of the present disclosure;

[0031] Figure 2 A flowchart showing a data processing method according to an embodiment of the present disclosure is shown;

[0032] Figure 3 A schematic diagram illustrating a first constant node according to an embodiment of the present disclosure is shown;

[0033] Figure 4A and Figure 4B A schematic diagram illustrating an application example of the data processing method according to an embodiment of the present disclosure;

[0034] Figure 5A block diagram showing a data processing device according to an embodiment of the present disclosure;

[0035] Figure 6 is a structural diagram illustrating a combined processing device according to an embodiment of the present disclosure;

[0036] Figure 7 is a schematic structural diagram showing a board according to an embodiment of the present disclosure;

[0037] Figure 8 A block diagram of a computer device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0038] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the embodiments described are part of the embodiments of the present disclosure, not all of them. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0039] It should be understood that the terms "first," "second," and the like in the claims, specification, and drawings of the present disclosure are used to distinguish between different objects, rather than to describe a specific order. The terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0040] It should also be understood that the terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the present disclosure. As used in this disclosure and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should further be understood that the term "and / or" as used in this disclosure and the claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0041] As used in this specification and claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0042] The data processing method of the embodiment of the present disclosure can be applied to a processor to improve the processing efficiency of the processor. The processor can be a general-purpose processor, such as a CPU (Central Processing Unit), or it can be an artificial intelligence processor (IPU) for performing artificial intelligence operations. Artificial intelligence operations may include machine learning operations, brain-like operations, etc. Among them, machine learning operations include neural network operations, k-means operations, support vector machine operations, etc. The artificial intelligence processor may, for example, include one or a combination of GPU (Graphics Processing Unit), NPU (Neural-Network Processing Unit), DSP (Digital Signal Process, Digital Signal Processing Unit), and Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) chips. The present disclosure does not limit the specific type of processor.

[0043] In one possible implementation, the processor mentioned in this disclosure may include multiple processing units, each of which can independently execute various assigned tasks, such as convolution tasks, pooling tasks, or fully connected tasks. This disclosure does not limit the processing units or the tasks they execute.

[0044] Figure 1 Schematic diagram of a processor of a data processing method according to an embodiment of the present disclosure. Figure 1 As shown, processor 100 includes multiple processing units 101 and a storage unit 102. Multiple processing units 101 are used to execute instruction sequences, and storage unit 102 is used to store data, which may include random access memory (RAM) and a register file. Multiple processing units 101 in processor 100 can share some storage space, such as some RAM storage space and register file, or have their own storage space.

[0045] Figure 2 FIG. 1 is a flow chart showing a data processing method according to an embodiment of the present disclosure. Figure 2 As shown, the method is applied to the above processor, and the method includes:

[0046] In step S11: according to the types of multiple nodes in the computation graph, determining a first constant node in the computation graph that does not support fusion;

[0047] In step S12: performing constant elimination processing according to the first constant node to obtain an elimination result;

[0048] In step S13: the calculation graph is updated according to the elimination result to obtain an updated calculation graph.

[0049] According to the embodiments of the present disclosure, it is possible to solve the problem that certain types of nodes do not support fusion, which leads to fusion segmentation and poor fusion effect. By eliminating the first constant node that does not support fusion to optimize the calculation graph, the calculation graph segmentation will be reduced, the fusion effect will be improved, and the processing overhead of data call and transmission will be reduced, the computational efficiency of the calculation graph will be improved, and the computing performance of the processor will be brought into play.

[0050] In one possible implementation, the above steps may be stored in the storage unit 102 in the form of computer-executable instructions, and multiple processing units 101 may access the storage unit and execute the instructions to perform the above steps. For example, the processor 100 may include multiple processing units 101, the first processing unit 101 may execute step S11, the second processing unit may execute step S12, and the third processing unit may execute step S13. In another example, more than two steps may be executed by one processing unit 101, for example, the first processing unit 101 executes step S11 and step S12, the second processing unit executes step S13, or step S11, step S12, and step S13 are executed by the same processing unit 101. The present disclosure does not limit the execution method of the steps.

[0051] In this example, the first processing unit 101 may be a CPU, the second processing unit may be a GPU, and the third processing unit may be an IPU. The first CPU may execute step S11, the GPU may execute step S12, and the IPU may execute step S13. The above processors may also execute multiple steps; for example, the GPU may execute steps S11-S13. This disclosure does not limit the type of processor.

[0052] In one possible implementation, the computational graph may be a data relationship diagram representing an information processing flow, for example, a CPU, IPU, GPU, NPU, machine learning processor (Machine Learning Unit, MLU), etc., which may use the information processing flow described in the computational graph to process information and obtain processing results.

[0053] In this example, a computational graph of a neural network can be executed by a GPU, MLU, or the like, and processed on data to be processed, where the data to be processed includes at least one of images, videos, voice, and text. For example, if the data to be processed is an image, the image can be input into the GPU, which can then process the image according to the information processing flow represented by the computational graph to obtain an image processing result.

[0054] In one possible implementation, a computation graph may include nodes and edges. Nodes represent operators that can process input data through operators. For example, nodes may include assertion nodes that can compare and / or judge data through assertion operators. Edges represent the transmission paths of data flows. The input edges of a node represent the data flow input to the node, and the output edges of a node represent the data flow outputting the results of the node's processing of the data. The output edge of a node can be the input edge of another node. For example, if node A transmits its processing results to node B via an output edge, then node A's output edge is the input edge of node B. During the process of processing according to the information processing flow represented by the computation graph, data to be processed can be input into the computation graph through input nodes. The input nodes can represent the input ports of the entire computation graph. Input can enter the computation graph through input nodes and be transmitted to other nodes for processing via the output edges of the input nodes. After processing by these nodes, the processing results are transmitted to the nodes to which they point via the output edges of these nodes. The data to be processed can be processed according to the above process, and the processing results of the data to be processed can be output by the output nodes. The output nodes refer to the output ports of the entire computation graph.

[0055] In one possible implementation, the computational graph may include multiple types of nodes. For example, the types of nodes include computational nodes for computation, such as convolution nodes for computational convolution; logical processing nodes for control, such as nodes where operators such as Switch and Merge are located, or nodes for obtaining input shapes, such as shape nodes (Shape), rank nodes (Rank), size nodes (Size), and other nodes. Optionally, the shape node (Shape) is used to obtain the scale or dimension of the input information (e.g., matrix, image, etc.), the rank node (Rank) is used to obtain the rank of the input information, and the size node (Size) is used to obtain the size of the input information. Optionally, both the image classification and detection networks will perform a Shape operation on the input to obtain its scale, and perform subsequent operations (such as Resize) based on its scale. Since the specific shape of the input is variable, the shapes of most subsequent operations are also variable. The Shape operation or Rank operation used to obtain their shapes cannot be set to constants, and their subsequent operations cannot be optimized with constants. The present disclosure does not limit the types of nodes.

[0056] In one possible implementation, the above-mentioned nodes may obtain attribute parameters of the input information, such as size, dimension, etc. Among the above-mentioned nodes, there are nodes that do not support fusion, such as nodes for obtaining input shapes, etc. Optionally, since the data type of the output data of the above-mentioned nodes (for example, the data type is int64) does not support fusion processing on the processor, and when the attribute parameters of the input information change, the output information of the above-mentioned nodes may also be variables. The present disclosure does not limit the reasons for not supporting fusion. In the example, the neural network can receive input images with different sizes, that is, the length (N), width (W), height (H) and other parameters of each input image (for example, a three-dimensional image) are inconsistent with each other. In the above-mentioned nodes (for example, the shape node (shape)), can placeholders be used? To replace parameters such as length, width, and height, for example, (?, ?, ?, 3), where 3 represents the three hues of RGB. When inputting an image, the node can obtain the actual values ​​of the three placeholders based on the actual size of the image. For example, if the actual size of the input image is 1048×728×256, the output of this node is (1048, 728, 256, 3), that is, the output of this node is the actual dimension of the input image. As the size of the input image changes, the output of this node is also variable. Therefore, the above node is not optimized as a constant node. Therefore, the above node can cause fusion segmentation, which in turn affects the fusion effect.

[0057] However, when using neural networks to process images, multiple inputs can usually be processed in batches, and the attribute parameters of the input data in the same batch can usually remain consistent. For example, multiple video frames of a video can be input, and the sizes of these video frames can remain consistent. For example, if a video includes 10,000 video frames, the sizes of the 10,000 video frames are consistent, and the processing results of the above nodes on the 10,000 video frames remain unchanged. In this case, each input video frame must be calculated through the above nodes, which not only wastes computing resources, but also makes the fusion effect of multiple nodes in the neural network calculation graph poor, resulting in segmented fusion, making it difficult to improve the fusion effect.

[0058] In one possible implementation, to address the above problem, a first constant node whose output information is a constant can be determined among multiple nodes in the computational graph, so that when the attribute parameters of the batch input data remain unchanged, the first constant node whose output information is a constant can be optimized to improve processing efficiency and enhance the fusion effect.

[0059] In a possible implementation, the first constant node refers to a node in the computation graph that does not support fusion and whose output information is a constant.

[0060] In one possible implementation, a first constant node can be determined from multiple nodes in a computation graph. The method further includes: determining a first target node of a preset type from the multiple nodes in the computation graph based on the types of the multiple nodes in the computation graph; performing constant discrimination processing on an output node of the first target node to obtain a second target node whose input information is constant, wherein the output node of the first target node includes a direct output node and an indirect output node, wherein the direct output node includes a node directly pointed to by an output edge of the first target node, and the indirect output node includes a node indirectly pointed to by an output edge of the first target node via other nodes; and obtaining the first constant node based on the first target node and the second target node.

[0061] In one possible implementation, the first target node of the preset type includes a node whose output information can be directly determined to be a constant. As described above, if the attribute parameters of the batch data input into the computation graph are consistent, the output information of the node of the above type is a constant. The first target node of the preset type in the computation graph can be found according to the type of each node in the computation graph. The first target node of the preset type may include the shape node (Shape), rank node (Rank), size node (Size) and a constant node whose output information is constant. It may also include other nodes whose output information is constant when the attribute parameters of the batch data remain unchanged, such as zero-like nodes (setting the elements of the input image or matrix to zero), one-like nodes (setting the elements of the input image or matrix to one), etc. The present disclosure does not limit the first target node of the preset type.

[0062] Figure 3 A schematic diagram illustrating a first constant node according to an embodiment of the present disclosure is shown.

[0063] In one possible implementation, after determining a first target node, constant discrimination processing may be performed on the output nodes of the first target node to obtain a second target node whose input information is constant. This step may include: determining whether the input information of the i-th direct output node of the first target node includes only constant information, where i is a positive integer; if the input information of the i-th direct output node of the first target node includes only constant information, determining the i-th direct output node of the first target node as the second target node; and continuing to determine whether the input information of the j-th indirect output node of the first target node corresponding to the i-th direct output node of the first target node includes only constant information, where j is a positive integer; if the input information of the j-th indirect output node of the first target node includes only constant information, determining the j-th indirect output node of the first target node as the second target node. Figure 3FIG. 1 shows a schematic diagram of a first constant node according to an embodiment of the present disclosure. Figure 3 As shown, node 0 is an input node, which can be used as a port for inputting data such as images and video frames into the computational graph. Nodes 1 and 2 are the first target nodes. For example, node 1 is a shape node (Shape), node 2 is a size node (Size), and node 5 is a constant node, which can respectively obtain the dimensions and sizes of the input image or video frame. It is possible to determine whether the input information of the direct output nodes of the first target node (for example, nodes 1 and 2) only includes constant information. It is possible to determine whether the direct output nodes of the first target node only include constant information. For example, node 3 is the i-th direct output node of node 1, and the input information of node 3 only includes constant information (for example, the output information of node 1). Since the output information of node 1 is a constant, the input information of node 3 is also a constant. That is, after node 3 operates on the constant, the output information is still a constant. Therefore, node 3 is the second target node.

[0064] In one possible implementation, if the i-th direct output node is the second target node, it can be determined whether the indirect output node subsequent to the i-th direct output node is the second target node. For example, if node 7 is the first indirect output node subsequent to the i-th direct output node, it can be determined that the input information of node 7 only includes constant information. For example, the input information of node 7 is the output information of node 3 and the output information of node 6. If the output information of node 3 and the output information of node 6 are both constant, then the output information of node 7 is also constant, that is, node 7 is the second target node. For example, if node 8 is the second indirect output node subsequent to the i-th direct output node, it can be determined that the input information of node 8 only includes constant information. For example, the input information of node 8 is the output information of node 7 and the output information of node 5. If the output information of node 7 and the output information of node 5 are both constant, then the output information of node 8 is also constant, that is, node 8 is the second target node. Through the above method, it can be determined whether multiple indirect output nodes following the i-th direct output node are the second target nodes. If it is determined that a certain indirect output node is not the second target node, the indirect output nodes following the indirect output node will no longer be determined.

[0065] In one possible implementation, constant discrimination processing is performed on the output node of the first target node to obtain a second target node whose input information is a constant, and the method further includes: when the i-th direct output node of the first target node or the j-th indirect output node of the first target node includes non-constant information, stopping constant discrimination processing on the remaining indirect output nodes corresponding to the i-th direct output node of the first target node or the j-th indirect output node of the first target node, wherein the remaining indirect output nodes include indirect output nodes that have not been subjected to constant discrimination processing among the indirect output nodes corresponding to the i-th direct output node of the first target node or the j-th indirect output node of the first target node.

[0066] In this example, the input information of node 8 is the output information of node 7 and node 5. If the output information of node 7 is a constant and the output information of node 5 is not a constant, then node 8 is not the second target node, and the indirect output nodes subsequent to node 8 can be discontinued. The other indirect output nodes subsequent to node 7 can be determined. If all indirect output nodes subsequent to node 7 are not the second target node, the indirect output nodes corresponding to the i-th direct output node can be discontinued, and the determination can begin with the i+1-th direct output node and its corresponding indirect output nodes, for example, node 4 and its corresponding indirect output node. Alternatively, if the i-th direct output node is not the second target node (for example, if the input information of node 3 also includes other non-constant information), the determination can proceed directly to the i+1-th direct output node (for example, node 4) and its corresponding indirect output node.

[0067] In a possible implementation, each first target node and second target node in the computation graph may be determined by the above method, that is, the first constant node in the computation graph may be obtained.

[0068] In this way, the first constant node can be found in the calculation graph, which is conducive to optimizing the calculation graph for the first constant node, reducing fusion segments, and improving the fusion effect.

[0069] In one possible implementation, in step S12, after determining the first constant node in the computation graph, since the output information of the first constant node is a constant, that is, a constant known quantity, calculations can be skipped through the first constant node, thereby reducing computation steps and improving processing efficiency. Step S12 may include removing the first constant node, its input edges, and its output edges to obtain the elimination result.

[0070] In a possible implementation, in a first constant node, it is determined that the nodes pointed to by the output edges are all second constant nodes of the first constant node. Removing the first constant node, the input edges and the output edges of the first constant node to obtain the elimination result may include: directly removing the second constant node, the input edges and the output edges of the second constant node. For example, Figure 3 As shown, nodes 1 and 3 are first constant nodes. When the output information of node 6 is constant, node 7 is also a first constant node. Then node 3 is a second constant node whose nodes pointed to by the output edge are all first constant nodes. Node 3 and its input and output edges can be directly removed.

[0071] In a possible implementation, in the first constant node, it can also be determined that the node pointed to by the output edge contains a third constant node that is not a constant node. Removing the first constant node, the input edge and the output edge of the first constant node to obtain the elimination result may include: obtaining the output information of the third constant node based on the data to be processed input into the calculation graph; saving the output information of the third constant node; removing the third constant node, the input edge and the output edge of the third constant node. The output information of the third constant node can be determined first to be used in the subsequent calculation of the non-constant node. In the example, nodes 1 and 3 are first constant nodes. When the output information of node 6 is non-constant, node 7 is not the first constant node. In this case, the node pointed to by the output edge of node 3 contains a non-constant node, and node 3 is the third constant node. The output information of node 3 is a constant. The value of the constant can be determined first for subsequent calculations, and then node 3 and its input and output edges can be deleted. That is, the input data to be processed is processed through the computational graph until the output information of node 3 is determined. After saving the output information of node 3, node 3 and its input and output edges can be deleted.

[0072] In a possible implementation, the second constant node in the calculation graph can be directly removed in the above manner, and after determining the output information of the third constant node, the third constant node can be removed, and the input and output edges of the second constant node and the third constant node can be removed at the same time to simplify the calculation graph and improve processing efficiency. The removal of the first constant node and the input edge and output edge of the first constant node includes: removing the second constant node and the third constant node at the same time, and the input edge and output edge of the second constant node and the third constant node. In the example, when the second constant node and the third constant node are simultaneously included in the calculation graph, the output information of the third constant node can be first determined and saved, and then the second constant node and the third constant node, as well as the input edge and output edge of the second constant node and the third constant node are removed at the same time, that is, the first constant node is removed, and the input edge and output edge of the first constant node are removed to obtain the elimination result (that is, the calculation graph after removing the first constant node and the input edge and output edge of the first constant node).

[0073] In one possible implementation, in step S13, since the first constant node (including the second constant node that does not support fusion) is deleted, other nodes in the computation graph that support fusion may be fused to obtain a fused node, and the updated computation graph may include at least the fused node. That is, if after deleting the first constant node, the computation graph only includes nodes that support fusion, then after fusing the nodes in the computation graph, the fused node may be obtained, i.e., the updated computation graph. If after deleting the first constant node, the computation graph still includes nodes that do not support fusion, then the updated computation graph may include the fused node and other nodes that do not support fusion.

[0074] In one possible implementation, the information to be processed can be processed through the updated computational graph to obtain the processing results. During the processing, the fusion effect is improved, the overhead of data transmission can be reduced, the computational overhead of constant nodes can be reduced, and the processing efficiency can be improved.

[0075] Figure 4A and Figure 4B Schematic diagram showing an application example of the data processing method according to an embodiment of the present disclosure. Figure 4A As shown, the computation graph may be a computation graph for image processing executed by a GPU, and node 10 is an input node, through which an image may be input into the computation graph for processing.

[0076] In one possible implementation, a first constant node can be determined among multiple nodes in the computational graph. For example, node 11 and node 12 are first target nodes of a preset type. For example, node 11 is a shape node (Shape) and node 12 is a size node (Size). When the image size of the same batch remains unchanged, the output information of node 11 and node 12 is constant.

[0077] In one possible implementation, the input information of node 14 includes only the output information of node 12. Therefore, node 14 is the second target node. The input information of node 17 includes only the output information of node 14. Therefore, node 17 is the second target node. The input information of nodes 13, 15, and 18 may include other non-constants. Therefore, nodes 13 and 18, and nodes 15 and 18 are not second target nodes. The first constant node may include the first target node and the second target node, that is, nodes 11, 12, 14, and 17.

[0078] In one possible implementation, a second constant node can be determined within the first constant node, i.e., a first constant node whose output edges all point to the first constant node. In this example, the second constant nodes may include nodes 11 and 14. The input information of node 17's output node 18 includes non-constant information, and the input information of node 12's output node 13 and node 15 also includes non-constant information. Therefore, nodes 12 and 17 are not second constant nodes, but rather third constant nodes.

[0079] In one possible implementation, the output information of the third constant node can be determined based on the input image. For example, the input image can be processed by each node to obtain the output information of node 12 and node 17, and then saved. Afterwards, the second constant node and the third constant node and their input edges and output edges can be deleted from the computation graph at the same time (e.g., Figure 4B As shown in FIG, the remaining nodes can be further fused to obtain an updated computational graph. The input image and the output information of nodes 12 and 17 are processed using the updated computational graph to obtain a processing result.

[0080] In this example, the remaining nodes (i.e., nodes 10, 13, 16, 15, 18, and other nodes) all support fusion. These nodes can be fused into a fused node, and the fused node can be used to process other images in the same batch and the output information of nodes 12 and 17 to obtain a processing result. Alternatively, if some of the remaining nodes in the updated computational graph still do not support fusion, the nodes that support fusion can be fused into a fused node, and the fused node and the nodes that do not support fusion can be used to process other images in the same batch to obtain a processing result.

[0081] In one possible implementation, the data processing method can remove nodes in the computation graph that do not support fusion, reducing unnecessary computational overhead and improving fusion effects. This reduces the processing overhead of data transfer and calls, improves the computational efficiency of the computation graph, and maximizes the computing performance of the processor. It can be used to optimize the processing of processors such as GPUs and MLUs, for example, optimizing the processing of images through neural networks. This disclosure does not limit the scope of application of the data processing method.

[0082] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0083] It should be further explained that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0084] Figure 5 FIG. 1 is a block diagram of a data processing device according to an embodiment of the present disclosure. Figure 5 As shown, it includes: a determination module 11, which is used to determine the first constant node that does not support fusion in the calculation graph according to the types of multiple nodes in the calculation graph; an elimination module 12, which is used to perform constant elimination processing according to the first constant node to obtain an elimination result; and an update module 13, which is used to update the calculation graph according to the elimination result to obtain an updated calculation graph.

[0085] In a possible implementation, the elimination module is further configured to remove the first constant node, the input edge, and the output edge of the first constant node to obtain the elimination result.

[0086] In one possible implementation, the first constant node includes a third constant node, wherein the nodes pointed to by the output edge of the third constant node include non-constant nodes, and the elimination module is further used to: obtain output information of the third constant node based on the data to be processed input into the calculation graph; save the output information of the third constant node; and remove the third constant node, the input edge, and the output edge of the third constant node.

[0087] In one possible implementation, the first constant node includes a second constant node, wherein the nodes pointed to by the output edges of the second constant node are all the first constant nodes, and the elimination module is further used to: directly remove the second constant node, the input edges and the output edges of the second constant node.

[0088] In a possible implementation, the first constant node includes a second constant node and a third constant node, and the elimination module is further used to: simultaneously remove the second constant node and the third constant node, as well as the input edges and output edges of the second constant node and the third constant node.

[0089] In one possible implementation, the determination module is further used to: determine a first target node of a preset type among multiple nodes in the computational graph based on the types of multiple nodes in the computational graph; perform constant discrimination processing on the output node of the first target node to obtain a second target node whose input information is a constant, the output node of the first target node including a direct output node and an indirect output node, the direct output node including the node directly pointed to by the output edge of the first target node, and the indirect output node including the node indirectly pointed to by the output edge of the first target node via other nodes; obtain the first constant node based on the first target node and the second target node.

[0090] In one possible implementation, the determination module is further used to: determine whether the input information of the i-th direct output node of the first target node includes only constant information, where i is a positive integer; if the input information of the i-th direct output node includes only constant information, determine the i-th direct output node as the second target node, and determine whether the input information of the j-th indirect output node corresponding to the i-th direct output node includes only constant information, where j is a positive integer; if the input information of the j-th indirect output node includes only constant information, determine the j-th indirect output node as the second target node.

[0091] In one possible implementation, the determination module is further used to: when the i-th direct output node or the j-th indirect output node includes non-constant information, stop performing constant discrimination processing on the remaining indirect output nodes corresponding to the i-th direct output node or the j-th indirect output node, and the remaining indirect output nodes include indirect output nodes that have not been subjected to constant discrimination processing among the indirect output nodes corresponding to the i-th direct output node or the j-th indirect output node.

[0092] In a possible implementation, the data to be processed includes at least one of images, videos, voices, and texts.

[0093] Figure 6 FIG. 1 is a structural diagram showing a combined processing device 1200 according to an embodiment of the present disclosure. Figure 6 As shown in FIG, the combined processing device 1200 includes a computing processing device 1202, an interface device 1204, other processing devices 1206, and a storage device 1208. According to different application scenarios, the computing processing device may include one or more computing devices 1210, which may be configured to execute the combined processing device 1202, an interface device 1204, other processing devices 1206, and a storage device 1208. Figure 2 The described operation.

[0094] In different embodiments, the computing and processing device of the present disclosure can be configured to perform user-specified operations. In exemplary applications, the computing and processing device can be implemented as a single-core artificial intelligence processor or a multi-core artificial intelligence processor. Similarly, one or more computing devices included in the computing and processing device can be implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core. When multiple computing devices are implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core, with respect to the computing and processing device of the present disclosure, it can be regarded as having a single-core structure or a homogeneous multi-core structure.

[0095] In exemplary operation, the computing processing device of the present invention can interact with other processing devices through an interface device to jointly complete the operation specified by the user. Depending on the implementation, other processing devices of the present invention may include one or more types of processors in general and / or special processors such as a central processing unit (CPU), a graphics processing unit (GPU), an artificial intelligence processor, etc. These processors may include but are not limited to a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As previously mentioned, only with respect to the computing processing device of the present invention, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing processing device and other processing devices are considered together, the two can be regarded as forming a heterogeneous multi-core structure.

[0096] In one or more embodiments, the other processing device may serve as an interface between the computing device of the present disclosure (which may be embodied as an artificial intelligence computing device such as a neural network computing device) and external data and control, performing basic control including but not limited to data transfer, starting and / or stopping the computing device, and so on. In other embodiments, the other processing device may also collaborate with the computing device to jointly complete computing tasks.

[0097] In one or more embodiments, the interface device can be used to transmit data and control instructions between the computing and processing device and other processing devices. For example, the computing and processing device can obtain input data from other processing devices via the interface device and write it to the storage device (or memory) on the computing and processing device chip. Furthermore, the computing and processing device can obtain control instructions from other processing devices via the interface device and write them to the control cache on the computing and processing device chip. Alternatively or optionally, the interface device can also read data from the storage device of the computing and processing device and transmit it to other processing devices.

[0098] Additionally or optionally, the combined processing device of the present disclosure may further include a storage device. As shown in the figure, the storage device is connected to the computing processing device and the other processing device, respectively. In one or more embodiments, the storage device may be used to store data of the computing processing device and / or the other processing device. For example, the data may be data that cannot be fully stored in the internal or on-chip storage device of the computing processing device or other processing device.

[0099] In some embodiments, the present disclosure also discloses an artificial intelligence chip (e.g. Figure 7 The chip 1302 shown in FIG. 1 includes the above-mentioned data processing device. In one implementation, the chip is a system-on-chip (SoC) and integrates one or more Figure 6 The chip can be connected to the external interface device (such as Figure 7 The external interface device 1306 shown in the figure is connected to other related components. The related components can be, for example, a camera, a display, a mouse, a keyboard, a network card or a wifi interface. In some application scenarios, other processing units (such as video codecs) and / or interface modules (such as DRAM interfaces) can be integrated on the chip. In some embodiments, the present disclosure also discloses a chip packaging structure, which includes the above-mentioned chip. In some embodiments, the present disclosure also discloses a board card, which includes the above-mentioned chip packaging structure. The following will be combined with Figure 7 The board is described in detail.

[0100] Figure 7 FIG. 1 is a schematic diagram showing the structure of a board 1300 according to an embodiment of the present disclosure. Figure 7 As shown in , the board includes a storage device 1304 for storing data, which includes one or more storage units 1310. The storage device can be connected and data can be transmitted with the control device 1308 and the artificial intelligence chip 1302 described above by means of, for example, a bus. Furthermore, the board also includes an external interface device 1306, which is configured for data relay or transfer function between the chip (or the chip in the chip packaging structure) and the external device 1312 (such as a server or computer, etc.). For example, the data to be processed can be passed to the chip by the external device through the external interface device. For another example, the calculation result of the chip can be transmitted back to the external device via the external interface device. Depending on different application scenarios, the external interface device can have different interface forms, for example, it can adopt a standard PCIE interface, etc.

[0101] Each group of storage units is connected to the artificial intelligence chip via a bus. It is understood that each group of storage units can be DDR SDRAM (English: Double Data Rate SDRAM, double data rate synchronous dynamic random access memory).

[0102] DDR can double the speed of SDRAM without increasing the clock frequency. DDR allows data to be read out on the rising and falling edges of the clock pulse. The speed of DDR is twice that of standard SDRAM. In one embodiment, the storage device may include 4 groups of storage units. Each group of storage units may include multiple DDR4 particles (chips). In one embodiment, the artificial intelligence chip may include 4 72-bit DDR4 controllers, and 64 bits of the above 72-bit DDR4 controllers are used for data transmission and 8 bits are used for ECC verification. It can be understood that when DDR4-3200 particles are used in each group of storage units, the theoretical bandwidth of data transmission can reach 25600MB / s.

[0103] In one embodiment, each group of the memory cells includes a plurality of double data rate synchronous dynamic random access memories (DDRs) connected in parallel. DDRs can transmit data twice within one clock cycle. A controller for controlling the DDRs is provided in the chip to control data transmission and data storage in each of the memory cells.

[0104] The interface device is electrically connected to the artificial intelligence chip. The interface device is used to realize data transmission between the artificial intelligence chip and an external device (such as a server or a computer). For example, in one embodiment, the interface device can be a standard PCIE interface. For example, the data to be processed is transmitted to the chip by the server through the standard PCIE interface to realize data transfer. Preferably, when the PCIE 3.0X 16 interface is used for transmission, the theoretical bandwidth can reach 16000MB / s. In another embodiment, the interface device can also be other interfaces. The present disclosure does not limit the specific forms of expression of the above-mentioned other interfaces. The interface unit can realize the switching function. In addition, the calculation results of the artificial intelligence chip are still transmitted back to the external device (such as a server) by the interface device.

[0105] The control device is electrically connected to the artificial intelligence chip. The control device is used to regulate the state of the artificial intelligence chip. Specifically, the artificial intelligence chip and the control device can be electrically connected through an SPI interface. The control device may include a single-chip microcomputer (MCU). For example, the artificial intelligence chip may include multiple processing chips, multiple processing cores or multiple processing circuits, which can drive multiple loads. Therefore, the artificial intelligence chip can be in different working states such as multi-load and light load. The control device can realize the regulation of the working state of multiple processing chips, multiple processing and / or multiple processing circuits in the artificial intelligence chip.

[0106] According to the above combination Figure 6 and Figure 7 Based on the description, those skilled in the art can understand that the present disclosure also discloses an electronic device or apparatus, which may include one or more of the above-mentioned boards, one or more of the above-mentioned chips and / or one or more of the above-mentioned combined processing devices.

[0107] Depending on the application scenario, the electronic devices or apparatuses disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, driving recorders, navigators, sensors, cameras, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, household appliances, and / or medical devices. The vehicles include airplanes, ships, and / or cars; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; and the medical devices include magnetic resonance imaging (MRI) machines, ultrasound machines, and / or electrocardiographs.

[0108] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0109] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0110] The electronic device may be provided as a terminal, a server, or other forms of devices.

[0111] Figure 8FIG1 shows a block diagram of a computer device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Figure 8 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0112] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0113] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0114] It should be noted that, for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will understand that the scheme of the present disclosure is not limited by the order of the actions described. Therefore, based on the disclosure or teachings of the present disclosure, those skilled in the art will understand that some of the steps therein can be performed in other orders or simultaneously. Further, those skilled in the art will understand that the embodiments described in the present disclosure can be regarded as optional embodiments, that is, the actions or modules involved therein are not necessarily necessary for the implementation of one or more schemes of the present disclosure. In addition, depending on the different schemes, the present disclosure also has different emphases on the description of some embodiments. In view of this, those skilled in the art will understand that the parts that are not described in detail in a certain embodiment of the present disclosure may also refer to the relevant descriptions of other embodiments.

[0115] In terms of specific implementation, based on the disclosure and teachings of this disclosure, those skilled in the art can understand that several embodiments disclosed in this disclosure can also be implemented in other ways not disclosed herein. For example, with respect to the various units in the electronic device or device embodiments described above, this document divides them based on the consideration of logical functions, and there may be other ways of division in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. With respect to the connection relationship between different units or components, the connection discussed above in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components.

[0116] In the present disclosure, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network elements. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the solutions described in the embodiments of the present disclosure. In addition, in some scenarios, multiple units in the embodiments of the present disclosure may be integrated into one unit or each unit may exist physically separately.

[0117] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a particular embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined in any way. To keep the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] The electronic device or device disclosed in the present invention can also be applied to the Internet, Internet of Things, data centers, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, medical and other fields. Furthermore, the electronic device or device disclosed in the present invention can also be used in application scenarios related to artificial intelligence, big data and / or cloud computing, such as the cloud, edge, and terminal. In one or more embodiments, the electronic device or device with high computing power according to the solution disclosed in the present invention can be applied to cloud devices (such as cloud servers), while the electronic device or device with low power consumption can be applied to terminal devices and / or edge devices (such as smart phones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-end integration.

[0119] The foregoing content can be better understood in accordance with the following terms:

[0120] For example, Clause A1, a method of data processing, the method comprising determining, based on the types of multiple nodes in the computation graph, a first constant node that does not support fusion in the computation graph; performing constant elimination processing on the first constant node to obtain an elimination result; and updating the computation graph based on the elimination result to obtain an updated computation graph.

[0121] Clause A2, according to the method described in Clause A1, constant elimination processing is performed based on the first constant node to obtain an elimination result, including: removing the first constant node, the input edge and the output edge of the first constant node to obtain the elimination result.

[0122] Clause A3, according to the method described in Clause A2, the first constant node includes a third constant node, wherein the node pointed to by the output edge of the third constant node includes a non-constant node, and the removal of the first constant node and the input edge and output edge of the first constant node includes: obtaining the output information of the third constant node based on the data to be processed input into the computational graph; saving the output information of the third constant node; and removing the third constant node and the input edge and output edge of the third constant node.

[0123] Clause A4, according to the method described in Clause A2, the first constant node includes a second constant node, wherein the nodes pointed to by the output edges of the second constant node are all first constant nodes, and the removal of the first constant node and the input edges and output edges of the first constant node includes: directly removing the second constant node and the input edges and output edges of the second constant node.

[0124] Item A5. According to the method described in Item A2, the first constant node includes a second constant node and a third constant node, and the removal of the first constant node and the input edges and output edges of the first constant node includes: simultaneously removing the second constant node and the third constant node, as well as the input edges and output edges of the second constant node and the third constant node.

[0125] Clause A6, according to the method described in Clause A1, determining the first constant node that does not support fusion in the computation graph based on the types of multiple nodes in the computation graph, including: determining a first target node of a preset type among the multiple nodes in the computation graph based on the types of multiple nodes in the computation graph; performing constant discrimination processing on the output node of the first target node to obtain a second target node whose input information is a constant, the output node of the first target node including a direct output node and an indirect output node, the direct output node including the node directly pointed to by the output edge of the first target node, and the indirect output node including the node indirectly pointed to by the output edge of the first target node via other nodes; obtaining the first constant node based on the first target node and the second target node.

[0126] Clause A7, according to the method described in Clause A6, constant discrimination processing is performed on the output node of the first target node to obtain a second target node whose input information is a constant, including: judging whether the input information of the i-th direct output node of the first target node includes only constant information, i is a positive integer; if the input information of the i-th direct output node includes only constant information, determining the i-th direct output node as the second target node, and judging whether the input information of the j-th indirect output node corresponding to the i-th direct output node includes only constant information, j is a positive integer; if the input information of the j-th indirect output node includes only constant information, determining the j-th indirect output node as the second target node.

[0127] Clause A8, according to the method described in Clause A6, performing constant discrimination processing on the output node of the first target node to obtain a second target node whose input information is a constant, also includes: when the i-th direct output node or the j-th indirect output node includes non-constant information, stopping the constant discrimination processing on the remaining indirect output nodes corresponding to the i-th direct output node or the j-th indirect output node, the remaining indirect output nodes including the indirect output nodes corresponding to the i-th direct output node or the j-th indirect output node that have not undergone constant discrimination processing.

[0128] Item A9, according to the method described in items A1-A8, the data to be processed includes at least one of images, videos, voice and text.

[0129] Item A10, a data processing device, comprising: a determination module, used to determine a first constant node in a computational graph that does not support fusion based on the types of multiple nodes in the computational graph; an elimination module, used to perform constant elimination processing based on the first constant node to obtain an elimination result; and an update module, used to update the computational graph based on the elimination result to obtain an updated computational graph.

[0130] Item A11, according to the device described in Item A10, the elimination module is further used to: remove the first constant node, the input edge and the output edge of the first constant node to obtain the elimination result.

[0131] Item A12, according to the device described in Item A11, the first constant node includes a third constant node, wherein the nodes pointed to by the output edge of the third constant node include non-constant nodes, and the elimination module is further used to: obtain the output information of the third constant node based on the data to be processed input into the calculation graph; save the output information of the third constant node; remove the third constant node, the input edge and the output edge of the third constant node.

[0132] Item A13, according to the device described in Item A11, the first constant node includes a second constant node, wherein the nodes pointed to by the output edges of the second constant node are all first constant nodes, and the elimination module is further used to: directly remove the second constant node, the input edges and the output edges of the second constant node.

[0133] Item A14, according to the device described in Item A11, the first constant node includes a second constant node and a third constant node, and the elimination module is further used to: simultaneously remove the second constant node and the third constant node, as well as the input edges and output edges of the second constant node and the third constant node.

[0134] Item A15, according to the device described in Item A10, the determination module is further used to: determine a first target node of a preset type among multiple nodes in the computational graph based on the types of multiple nodes in the computational graph; perform constant discrimination processing on the output node of the first target node to obtain a second target node whose input information is a constant, the output node of the first target node includes a direct output node and an indirect output node, the direct output node includes a node directly pointed to by the output edge of the first target node, and the indirect output node includes a node indirectly pointed to by the output edge of the first target node via other nodes; obtain the first constant node based on the first target node and the second target node.

[0135] Item A16, according to the device described in Item A15, the determination module is further used to: determine whether the input information of the i-th direct output node of the first target node includes only constant information, i is a positive integer; if the input information of the i-th direct output node includes only constant information, determine the i-th direct output node as the second target node, and determine whether the input information of the j-th indirect output node corresponding to the i-th direct output node includes only constant information, j is a positive integer; if the input information of the j-th indirect output node includes only constant information, determine the j-th indirect output node as the second target node.

[0136] Item A17, according to the device described in Item A15, the determination module is further used to: when the i-th direct output node or the j-th indirect output node includes non-constant information, stop performing constant discrimination processing on the remaining indirect output nodes corresponding to the i-th direct output node or the j-th indirect output node, and the remaining indirect output nodes include the indirect output nodes corresponding to the i-th direct output node or the j-th indirect output node that have not undergone constant discrimination processing.

[0137] Item A18, according to the device described in items A10-A17, the data to be processed includes at least one of images, videos, voice and text.

[0138] Although a number of embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art may conceive of many modifications, changes, and alternatives without departing from the ideas and spirit of the present disclosure. It should be understood that in practicing the present disclosure, various alternatives to the embodiments of the present disclosure described herein may be adopted. The appended claims are intended to define the scope of protection of the present disclosure and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A data processing method, characterized in that: The method is used to process at least one of an image, a video, a voice, and a text, and includes: determining, based on types of a plurality of nodes in a computation graph, a first constant node in the computation graph that does not support fusion; Performing constant elimination processing according to the first constant node to obtain an elimination result; Update the computation graph according to the elimination result to obtain an updated computation graph; The determining, according to the types of the plurality of nodes in the computation graph, a first constant node that does not support fusion in the computation graph includes: Determining, according to the types of the plurality of nodes in the computation graph, a first target node of a preset type among the plurality of nodes in the computation graph; performing constant discrimination processing on the output nodes of the first target node to obtain a second target node whose input information is a constant, wherein the output nodes of the first target node include direct output nodes and indirect output nodes, wherein the direct output nodes include nodes directly pointed to by the output edge of the first target node, and the indirect output nodes include nodes indirectly pointed to by the output edge of the first target node via other nodes; Obtaining the first constant node according to the first target node and the second target node; Performing constant elimination processing according to the first constant node to obtain an elimination result includes: Removing the first constant node, the input edge, and the output edge of the first constant node to obtain the elimination result; The first constant node is a node in the computation graph that does not support fusion and whose output information is a constant.

2. The method according to claim 1, characterized in that The first constant node includes a third constant node, wherein the nodes pointed to by the output edge of the third constant node include non-constant nodes, The removing of the first constant node, the input edge and the output edge of the first constant node comprises: Obtaining output information of a third constant node according to the to-be-processed data input into the computation graph; Saving the output information of the third constant node; Remove the third constant node, its input edges, and its output edges.

3. The method according to claim 1, characterized in that The first constant node includes a second constant node, wherein the nodes pointed to by the output edges of the second constant node are all first constant nodes, The removing of the first constant node, the input edge and the output edge of the first constant node comprises: Directly remove the second constant node, the input edge, and the output edge of the second constant node.

4. The method according to claim 1, wherein The first constant node includes a second constant node and a third constant node, The removing of the first constant node, the input edge and the output edge of the first constant node comprises: Simultaneously removing the second constant node and the third constant node, as well as the input edges and output edges of the second constant node and the third constant node; The first constant node is obtained according to the first target node and the second target node.

5. The method according to claim 1, wherein Performing constant discrimination processing on the output node of the first target node to obtain a second target node whose input information is a constant, comprising: Determine whether the input information of the i-th direct output node of the first target node includes only constant information, where i is a positive integer; In a case where input information of an i-th direct output node of the first target node includes only constant information, determining the i-th direct output node of the first target node as a second target node, and determining whether input information of a j-th indirect output node of the first target node corresponding to the i-th direct output node of the first target node includes only constant information, where j is a positive integer; In a case where the input information of the j-th indirect output node of the first target node includes only constant information, the j-th indirect output node of the first target node is determined as the second target node.

6. The method according to claim 1, characterized in that Performing constant discrimination processing on the output node of the first target node to obtain a second target node whose input information is a constant, further comprising: In a case where the i-th direct output node of the first target node or the j-th indirect output node of the first target node includes non-constant information, constant discrimination processing is stopped on the remaining indirect output nodes corresponding to the i-th direct output node of the first target node or the j-th indirect output node of the first target node, and the remaining indirect output nodes include indirect output nodes that have not been subjected to constant discrimination processing among the indirect output nodes corresponding to the i-th direct output node of the first target node or the j-th indirect output node of the first target node.

7. The method according to any one of claims 2, characterized in that The data to be processed includes at least one of images, videos, voice and text.

8. A data processing device, characterized in that: The device is used to process at least one of an image, video, voice, and text, including: A determination module, configured to determine, based on types of multiple nodes in the computation graph, a first constant node in the computation graph that does not support fusion; an elimination module, configured to perform constant elimination processing according to the first constant node to obtain an elimination result; An updating module, configured to update the computation graph according to the elimination result to obtain an updated computation graph; The determination module is further configured to: determine a first target node of a preset type from among the multiple nodes in the computation graph according to the types of the multiple nodes in the computation graph; perform constant discrimination processing on an output node of the first target node to obtain a second target node whose input information is a constant, wherein the output node of the first target node includes a direct output node and an indirect output node, wherein the direct output node includes a node directly pointed to by an output edge of the first target node, and the indirect output node includes a node indirectly pointed to by an output edge of the first target node via other nodes; and obtain the first constant node according to the first target node and the second target node; The elimination module is further configured to: remove the first constant node, the input edge, and the output edge of the first constant node to obtain the elimination result; The first constant node is a node in the computation graph that does not support fusion and whose output information is a constant.

9. A computer device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images