Data processing method, device and related products

Abstract

The present disclosure relates to a data processing method and device and related products. The product includes a processor, and the processor includes multiple processing units and a storage unit. The multiple processing units are used to execute instruction sequences such as updating calculation graphs and processing data. The storage unit is used to store data and may include a random access memory (RAM) and a register file. The multiple processing units in the processor can share part of the storage space, such as sharing part of the RAM storage space and the register file, and can also have their own storage space at the same time. By using the above processor, the present disclosure can improve the computing efficiency of the related products when performing operations on neural network models.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method and device and related products. Background Art

[0002] With the development of computer technology, dataflow programming has emerged. It can break down complex calculations into simpler ones, for example, into multiple nodes, making calculations simpler and easier to use. However, this can result in additional computational overhead. Related technologies combine multiple simple calculations into a single node before the processing device performs the calculation. This reduces the time it takes to transfer data between nodes and improves computational efficiency. However, some nodes may not support fusion, resulting in fragmented fusion and reduced optimization effectiveness. Summary of the Invention

[0003] Based on this, it is necessary to provide a data processing method, device, computer equipment and storage medium that can improve computing efficiency in response to the above technical problems.

[0004] According to one aspect of the present disclosure, a data processing method is provided, comprising: determining processing conditions of a computational graph; updating the computational graph according to the processing conditions of the computational graph and the types of multiple nodes in the computational graph to obtain an updated computational graph; and inputting data to be processed into the updated computational graph to obtain processing results of the data to be processed.

[0005] According to another aspect of the present disclosure, a data processing device is provided, including: a condition module for determining the processing conditions of a calculation graph; an update module for updating the calculation graph according to the processing conditions of the calculation graph and the types of multiple nodes in the calculation graph to obtain an updated calculation graph; and an acquisition module for inputting data to be processed into the updated calculation graph to obtain processing results of the data to be processed.

[0006] According to another aspect of the present disclosure, an artificial intelligence chip is provided, wherein the chip includes the data processing device.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising the artificial intelligence chip.

[0008] According to another aspect of the present disclosure, a board is provided, comprising: a storage device, an interface device, a control device, and the artificial intelligence chip; wherein the artificial intelligence chip is connected to the storage device, the control device, and the interface device, respectively; the storage device is used to store data; the interface device is used to realize data transmission between the artificial intelligence chip and an external device; and the control device is used to monitor the status of the artificial intelligence chip.

[0009] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the data processing method.

[0010] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the data processing method when executed by a processor.

[0011] According to the data processing method of the embodiment of the present disclosure, the calculation graph can be optimized according to the processing conditions of the calculation graph. In the execution phase, the ineffective assertion nodes, the related nodes of the assertion nodes, the output edges of the related nodes of the assertion nodes, and the output edges of the assertion nodes can be cleared to reduce the fusion segments and improve the fusion effect without affecting the processing results. In addition, when the number of loops is one, the calculation graph can be optimized by the execution nodes in the loop structure subgraph, and the loop control nodes that do not support fusion can be removed to reduce the fusion segments and improve the fusion effect. In addition, the processing overhead of data calls and transfers caused by fusion segments can be reduced, while unnecessary node data processing can be reduced, the computational efficiency of the calculation graph can be improved, and the computing performance of the processor can be brought into play.

[0012] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0014] Figure 1 A schematic diagram illustrating a processor of a data processing method according to an embodiment of the present disclosure;

[0015] Figure 2 A flowchart showing a data processing method according to an embodiment of the present disclosure is shown;

[0016] Figure 3 A schematic diagram illustrating an assertion node according to an embodiment of the present disclosure is shown;

[0017] Figure 4 A schematic diagram showing a cyclic structure subgraph according to an embodiment of the present disclosure;

[0018] Figure 5A and Figure 5B A schematic diagram illustrating an application example of the data processing method according to an embodiment of the present disclosure;

[0019] Figure 6 A block diagram showing a data processing device according to an embodiment of the present disclosure;

[0020] Figure 7 is a structural diagram illustrating a combined processing device according to an embodiment of the present disclosure;

[0021] Figure 8 is a schematic structural diagram showing a board according to an embodiment of the present disclosure;

[0022] Figure 9 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0023] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the embodiments described are part of the embodiments of the present disclosure, not all of them. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0024] It should be understood that the terms "first," "second," and the like in the claims, specification, and drawings of the present disclosure are used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0025] It should also be understood that the terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the present disclosure. As used in this disclosure and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should further be understood that the term "and / or" as used in this disclosure and the claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0026] As used in this specification and claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0027] The data processing method according to the embodiment of the present disclosure can be applied to a processor to improve the processing efficiency of the processor. The processor can be a general-purpose processor, such as a CPU (Central Processing Unit), or an artificial intelligence processor (IPU) for performing artificial intelligence operations. Artificial intelligence operations may include machine learning operations, brain-like operations, etc. Among them, machine learning operations include neural network operations, k-means operations, support vector machine operations, etc. The artificial intelligence processor may, for example, include a GPU (Graphics Processing Unit), an NPU (Neural-Network Processing Unit), a DSP (Digital Signal Process, digital signal processing unit), a field programmable gate array (Field-Programmable Gate Array, FPGA) chip or a combination thereof. The present disclosure does not limit the specific type of processor.

[0028] In one possible implementation, the processor mentioned in this disclosure may include multiple processing units, each of which can independently execute various assigned tasks, such as convolution tasks, pooling tasks, or fully connected tasks. This disclosure does not limit the processing units or the tasks they execute.

[0029] Figure 1 Schematic diagram of a processor of a data processing method according to an embodiment of the present disclosure. Figure 1 As shown, processor 100 includes multiple processing units 101 and a storage unit 102. Multiple processing units 101 are used to execute instruction sequences, and storage unit 102 is used to store data, which may include random access memory (RAM) and a register file. Multiple processing units 101 in processor 100 can share some storage space, such as some RAM storage space and register file, or have their own storage space.

[0030] Figure 2 FIG. 1 is a flow chart showing a data processing method according to an embodiment of the present disclosure. Figure 2 As shown, this method is applied to Figure 1 The processor, the method comprising:

[0031] In step S11: determining the processing conditions of the computation graph;

[0032] In step S12: updating the computation graph according to the processing conditions of the computation graph and the types of the multiple nodes in the computation graph to obtain an updated computation graph;

[0033] In step S13: the data to be processed is input into the updated calculation graph to obtain the processing result of the data to be processed.

[0034] According to the embodiments of the present disclosure, it is possible to solve the problem that certain types of nodes cannot be segmented during the fusion process, which causes fusion segmentation and poor fusion effect. By optimizing the calculation graph through the processing conditions of the calculation graph, the calculation graph segmentation can be reduced, the fusion effect can be improved, the processing overhead of data call and transmission can be reduced, the computational efficiency of the calculation graph can be improved, and the computing performance of the processor can be brought into play.

[0035] In one possible implementation, the above steps may be stored in the storage unit 102 in the form of computer-executable instructions, and multiple processing units 101 may access the storage unit and execute the instructions to perform the above steps. For example, the processor 100 may include multiple processing units 101, the first processing unit 101 may execute step S11, the second processing unit may execute step S12, and the third processing unit may execute step S13. In another example, more than two steps may be executed by one processing unit 101, for example, the first processing unit 101 executes step S11 and step S12, the second processing unit executes step S13, or step S11, step S12, and step S13 are executed by the same processing unit 101. The present disclosure does not limit the execution method of the steps.

[0036] In this example, the first processing unit 101 may be a CPU, the second processing unit may be a GPU, and the third processing unit may be an IPU. The first CPU may execute step S11, the GPU may execute step S12, and the IPU may execute step S13. The above processors may also execute multiple steps; for example, the GPU may execute steps S11-S13. This disclosure does not limit the type of processor.

[0037] In one possible implementation, the computational graph may be a data relationship diagram representing an information processing flow, for example, a CPU, IPU, GPU, NPU, machine learning processor (Machine Learning Unit, MLU), etc., which may use the information processing flow described in the computational graph to process information and obtain processing results.

[0038] In this example, a computational graph of a neural network can be executed by a GPU, MLU, or the like, and processed on data to be processed, where the data to be processed includes at least one of images, videos, voice, and text. For example, if the data to be processed is an image, the image can be input into the GPU, which can then process the image according to the information processing flow represented by the computational graph to obtain an image processing result.

[0039] In one possible implementation, a computation graph may include nodes and edges. Nodes represent operators that can process input data through operators. For example, nodes may include assertion nodes that can compare and / or judge data through assertion operators. Edges represent the transmission paths of data flows. The input edges of a node represent the data flow input to the node, and the output edges of a node represent the data flow that outputs the results of the node's processing of the data. The output edge of a node can be the input edge of another node. For example, if node A transmits its processing results to node B via an output edge, then node A's output edge is the input edge of node B. During the process of processing according to the information processing flow represented by the computation graph, data to be processed can be input into the computation graph through input nodes. The input nodes can represent the input ports of the entire computation graph. Input can enter the computation graph through input nodes and be transmitted to other nodes for processing via the output edges of the input nodes. After processing by these nodes, the processing results are transmitted to the nodes to which they point via the output edges of these nodes. The data to be processed can be processed according to the above process, and the processing results of the data to be processed can be output by the output nodes. The output nodes refer to the output ports of the entire computation graph.

[0040] In one possible implementation, the computation graph may include multiple types of nodes, for example, an execution node (Operater) for executing data processing, such as a computation node, an assertion node (Assert) for comparing and / or judging information, and a loop control node in loop processing (including a selection node Switch, a merge node Merge, an input node Enter, an output node Exit, and a continuation iteration node NextIteration), etc. The present disclosure does not impose any restrictions on the types of nodes.

[0041] In one possible implementation, the assertion node can compare and / or judge the input information to determine whether the comparison result or judgment result meets expectations. If it meets expectations, the calculation graph can continue to be executed, that is, the node pointed to by the output edge of the assertion node can continue to be executed. If it does not meet expectations, the processing can be interrupted and an error can be reported.

[0042] In one possible implementation, the assertion node can be used in the debugging phase (for example, the training phase of a neural network) to compare and / or judge the input information or the intermediate information output by other nodes in the computational graph. For example, the computational graph is a computational graph of a neural network. During the training process of the neural network, the assertion node can be used to judge whether the input sample meets the requirements. The assertion node usually performs judgment and other processing in the debugging phase, but in the execution phase (for example, after the training of the neural network is completed, the trained neural network is used to process the data to be processed), the input of the computational graph is usually information that meets the requirements and can be processed directly without judgment. The processing performed by the assertion node and the node that transmits the processing results to the assertion node is usually meaningless and consumes computing resources.

[0043] In the example, the sample can be an image. After the image is input into the neural network, the size of the image can be determined by the node before the assertion node, and the size can be transmitted to the assertion node through the output edge. The assertion node can determine whether the size of the image meets the requirements. If it meets the requirements, subsequent processing can be performed. If it does not meet the requirements, the processing is interrupted and an error is reported. In the example, during the execution phase (for example, the neural network training is completed and used to process images of a specific size), the GPU can be used to process images that meet the size requirements. That is, the images input into the neural network are all images that meet the size requirements. Then the assertion node and the node that determines the image size (the node that transmits the size to the assertion node through the output edge) have no effect during the execution phase. However, these nodes will still perform processing such as determining the size and making judgments, which consumes computing resources.

[0044] In one possible implementation, the assertion node and the node that transmits the processing result to the assertion node can be optimized in the execution phase. In step S11, the processing conditions of the calculation graph can be judged, and the processing conditions include the processing phase of the calculation graph, and the processing phase includes the debugging phase and the execution phase. That is, in step S11, it can be judged whether the processing performed by the calculation graph is the processing of the debugging phase or the processing of the execution phase. If it is the processing of the execution phase, the calculation graph can be optimized, the assertion node and the node that transmits the processing result to the assertion node can be deleted to reduce unnecessary graph segmentation and save computing resources. Moreover, before optimizing the calculation graph, since the data flow programming platform (Tensorflow) reasoning process is adapted to the GPU, the numerous nodes in the calculation graph are fused into one node mode before calculation, and the GPU or IPU compiles the calculation instructions as a whole, which can reduce unnecessary data transfer time between nodes. The GPU will put the nodes that support fusion into the fusion segment, and the nodes that do not support it are the most direct cause of graph segmentation. However, the GPU or IPU's graph fusion strategy only considers whether the device supports the compute nodes, and does not consider the fusion of non-compute nodes. For example, the GPU or IPU cannot support the fusion of logical nodes such as assertion nodes in the graph, so these operators will cause graph segmentation and increase data transmission time. Therefore, after optimizing the computation graph for the assertion nodes, the assertion nodes that cause graph fusion segmentation can be deleted, but this does not affect the entire computation process. Therefore, the above optimization of the computation graph reduces the impact of graph fusion segmentation on computation. When adapting to the IPU, TensorFlow inference makes specific optimizations for assertion nodes, which can minimize the segmentation of the computation graph on the IPU, reduce performance overhead outside of computation, and maximize the computational advantages.

[0045] In one possible implementation, step S12 may include: when the processing stage of the computation graph is the execution stage, determining an assertion node from the multiple nodes according to the types of the multiple nodes in the computation graph; traversing the input nodes of the assertion node to obtain the related nodes of the assertion node; clearing the assertion node, the related nodes of the assertion node, the input edges and output edges of the related nodes of the assertion node, and the input edges and output edges of the assertion node to obtain the updated computation graph.

[0046] In one possible implementation, as described above, assertion nodes and nodes that transmit processing results to assertion nodes have no effect during the execution phase. Therefore, assertion nodes and nodes that transmit processing results to assertion nodes in a computation graph can be identified and optimized. Assertion nodes can be identified based on the types of multiple nodes in the computation graph. For example, if a computation graph node includes execution nodes, assertion nodes, loop control nodes, and so on, assertion nodes can be identified.

[0047] In a possible implementation, after the assertion node is determined, a node that transmits the processing result to the assertion node, that is, a node related to the assertion node, may be determined.

[0048] Figure 3 FIG. 1 shows a schematic diagram of an assertion node according to an embodiment of the present disclosure. Figure 3 As shown, after determining the assertion node, the node that transmits the processing result to the assertion node can be determined, for example, nodes op1 and op2. In addition, in addition to the nodes that directly transmit the processing result to the assertion node (op1 and op2), the node that indirectly transmits the processing result to the assertion node can also be determined, that is, the processing result is finally transmitted to the assertion node after being processed by other nodes, for example, nodes op3, op4 and op5. For example, the processing result of op3 is transmitted to op1, and after being processed by op1, the processing result of op1 is obtained and transmitted to the assertion node, that is, the processing result of op3 is indirectly transmitted to the assertion node. Since the assertion node has no effect during the execution phase, the input information of the assertion node (for example, the processing results of op1 and op2) has no effect, that is, there is no need to input information to the assertion node, so nodes op1 and op2 have no effect. Furthermore, the nodes that transmit the processing results to op1 and op2 (nodes whose processing results are indirectly transmitted to the assertion node) have no effect.

[0049] In one possible implementation, traversing the input nodes of the assertion node to obtain related nodes of the assertion node includes: performing deep traversal processing on the input nodes of the assertion node to obtain a first node, wherein the output edge of the first node directly or indirectly points to the assertion node; determining in the first node whether there is a first node whose output edge directly or indirectly points to a second node, wherein the second node includes any node in the computation graph other than the assertion node and the first node; if there is a first node whose output edge directly or indirectly points to the second node in the first node, excluding the first node whose output edge directly or indirectly points to the second node, wherein the second node includes any node in the computation graph other than the assertion node and the first node; and determining the retained first node as the related node of the assertion node.

[0050] In one possible implementation, a deep traversal process can be performed on the input nodes of the assertion node. That is, the information source of the input information of the assertion node and the nodes passed through during the information transmission process (that is, the nodes whose output edges directly point to the assertion node, that is, the nodes whose processing results are directly transmitted to the assertion node, and the nodes whose output edges indirectly point to the assertion node, that is, the nodes whose processing results are indirectly transmitted to the assertion node) can be traced to obtain the first node that may not be effective during the execution phase. In this example, the deep traversal process can obtain nodes op1 and op2 whose output edges directly point to the assertion node, and nodes op3, op4, and op5 whose output edges indirectly point to the assertion node.

[0051] In one possible implementation, although the assertion node and the information source of the input information of the assertion node obtained by tracing may not play a role in the execution phase, the nodes passed through during the information transmission process may transmit the processing results to other nodes (second nodes), that is, not only the processing results are transmitted to the assertion node or the nodes passed through during the information transmission process (first nodes), but other nodes may play a role in the execution phase. Therefore, in the process of optimizing the calculation graph in the execution phase, the first node that transmits the processing results to the second node should not be optimized, that is, in the process of determining the nodes to be optimized, it can be determined whether there is an output edge in the first node that directly or indirectly points to the first node of other nodes (second nodes). If so, the first node that transmits the processing results to the second node is excluded. The retained first node is the relevant node of the assertion node. Optionally, if there is no second node that needs to be excluded, the excluded first node is empty, that is, all first nodes are relevant nodes of the assertion node. Optionally, if the excluded first nodes are all first nodes, the retained first node is empty, that is, the relevant nodes are empty.

[0052] For example, the processing result of node op4 is not only transmitted to op2, but also transmitted to the assertion node after being processed by op2. The processing result of node op4 is also transmitted directly to other nodes, and the processing result of op5 is transmitted indirectly to other nodes. These other nodes may have an impact during the execution phase. Therefore, nodes op4 and op5 may also have an impact during the execution phase. Therefore, when optimizing the computation graph during the execution phase, nodes op4 and op5 should not be optimized. In other words, nodes op4 and op5 can be excluded from the first nodes op1, op2, op3, op4, and op5, leaving op1, op2, and op3 as the relevant nodes of the assertion node.

[0053] In one possible implementation, assertion nodes that are ineffective during execution, their associated nodes, their input edges and output edges, and their associated nodes' input edges and output edges can be removed. That is, these graph elements are removed from the computation graph to obtain an updated computation graph. In this example, the assertion node and its input edges and output edges, nodes op1, op2, and op3, and their input edges and output edges can be removed from the computation graph to obtain an updated computation graph. For example, the updated computation graph may include subsequent nodes such as op6 and their output edges, as well as other nodes and their output edges.

[0054] In this way, ineffective assertion nodes, related nodes of assertion nodes, input edges and output edges of related nodes of assertion nodes, and input edges and output edges of the assertion nodes can be cleared during the execution phase to reduce unnecessary computing overhead without affecting the processing results and improve computing efficiency. Since the assertion nodes that cause fusion segments are deleted, the fusion segments can be reduced during fusion, the data transfer can be reduced, and the computing efficiency can be improved.

[0055] In one possible implementation, after the above optimization, in step S13, the data to be processed can be input into an updated computation graph, i.e., the computation graph is stripped of the assertion node, its related nodes, its related nodes' output edges, and the output edges of the assertion node, and processed by the updated computation graph. Step S13 may include: fusing multiple operators in the updated computation graph that support fusion to obtain a fused operator; and processing the data to be processed based on at least the fused operator to obtain the processing result.

[0056] In one possible implementation, since the computation graph is updated, the assertion nodes that cannot be fused, the related nodes of the assertion nodes, the input edges and output edges of the related nodes of the assertion nodes, and the input edges and output edges of the assertion nodes are deleted. Therefore, during the fusion process, the fusion segments can be reduced, the processing overhead of data calls and transfers can be reduced, and since unnecessary computing overhead is reduced, the processor (for example, GPU, MLU, etc.) can obtain processing results with higher processing efficiency.

[0057] In one possible implementation, after deleting the assertion nodes that do not support fusion, the nodes that support fusion can be fused to obtain a fused node. If there are no nodes in the updated computation graph that do not support fusion, all nodes can be fused into a fused node, and the fused node can be used to process the pending data to obtain a processing result. If other nodes that do not support fusion still exist in the updated computation graph, the fused node and the nodes that do not support fusion can be used to process the pending data to obtain a processing result.

[0058] In one possible implementation, the computation graph may perform a loop process, for example, the computation graph may use the processing result of the loop process as the output result of the computation graph, or as the intermediate result of the information processing in the computation graph. In an example, the computation graph may include a loop structure subgraph, wherein the loop structure subgraph includes nodes for performing the loop process, and the loop structure subgraph may be part or all of the computation graph. In an example, the loop structure subgraph may include a loop control node (for example, a selection node Switch, a merge node Merge, an input node Enter, an output node Exit, and a continue iteration node NextIteration) and an execution node (Operater). The loop control node may be used to determine the loop condition (for example, the number of loops). If the loop condition is met, the input information may be processed by the execution node Operator. If the loop condition is not met, the loop may be ended and the processing result of the previous loop may be output.

[0059] Figure 4 FIG. 1 is a schematic diagram showing a cyclic structure subgraph according to an embodiment of the present disclosure. Figure 4 As shown, the input information of the loop structure subgraph can enter the loop structure subgraph through the input node Enter, and the input information of the current loop and the processing result of the previous loop are merged through the merge node Merge (for example, the processing result of the previous loop is added to the list to replace the input information of the previous loop, and the input information of the current loop is added to the list). The selection node Switch can be used to determine whether the current loop count is less than the preset loop count. If it is less than the preset loop count, further loop processing can be performed. For example, if the current loop count is less than the preset loop count, the selection node Switch can transmit the input information in the list to the execution node Operator for processing. If the current loop count is equal to the preset loop count, the loop can be stopped after the execution node Operator processes the input information and adds the processing result to the list. The list is then transmitted to the output node Exit to output the list as the processing result of the loop processing to the loop structure subgraph. If the current loop count is greater than the preset loop count, the loop can be stopped. If the input information meets the loop condition, the execution node Operator can process the input information and input the processing result to the next iteration node NextIteration for the next loop. The above loop process can be iteratively executed until the number of loops is greater than the preset number of loops, then the loop ends and the list is output as a loop structure subgraph.

[0060] In the example, the above-mentioned loop structure subgraph can be executed by the GPU, the input information may include multiple images, the preset number of loops may be set to the number of images, the image may be input into the node Enter, and when the number of loops is less than or equal to the number of images, the image processing (for example, resampling, scaling, etc.) may be performed through the node Operator, and the processed image may be added to a list, and when the number of loops is equal to the number of images, the image of the current loop may be processed through the node Operator, and after the processed image is added to the list, the list may be output as the processing result of the loop processing to the loop structure subgraph.

[0061] In one possible implementation, during the debugging phase, a loop structure subgraph can be used to batch process multiple inputs to debug the computation graph (for example, to update the parameters of a neural network). During the execution phase, the computation graph can process the inputs. If a single input is processed (for example, only one image is processed at a time), no loop is required and execution can be performed directly through the execution node. The loop control nodes in the loop structure subgraph have no effect during the execution phase and can be optimized to reduce computational overhead. Before the computation graph is optimized, since loop control nodes are not compute nodes and cannot be fused on the GPU or IPU, these operators will cause graph segmentation, increasing data transfer time, etc. Therefore, the computation graph can be optimized for loop control nodes, removing loop control nodes that cause graph fusion segmentation. However, this does not affect the overall computation process. Therefore, the above optimization of the computation graph reduces the impact of graph fusion segmentation on computation. When adapting TensorFlow inference to the IPU, specific optimizations are made for these loop control operators, which can minimize the segmentation of the computation graph on the IPU, reduce performance overhead other than computation, and maximize the computational advantages. The present disclosure does not impose any restrictions on the processing stage of the computation graph. If the number of loops can be determined to be one during the debugging stage, the loop control nodes can also be optimized.

[0062] In one possible implementation, the processing conditions include the number of loops. In step S11, the processing conditions of the computational graph may be determined, that is, the number of loops may be determined (for example, it may be determined by the amount of input data to be processed, for example, in the inference stage, only one picture is required). When the number of loops is one, step S12 may include: determining a loop structure subgraph in the computational graph according to the types of multiple nodes in the computational graph, wherein the loop structure subgraph includes a loop control node and an execution node; when the number of loops in the loop structure subgraph is one, it is not necessary to determine whether to loop based on the loop control node, and it is only necessary to execute the operation of the execution node once. Therefore, the loop structure subgraph can be optimized at this time; according to the position information of the loop structure subgraph in the computational graph, the loop structure subgraph is replaced with the execution node to obtain the updated computational graph.

[0063] In one possible implementation, the processing condition includes a loop count. If the loop count is one, a loop structure subgraph (i.e., a portion of the computation graph used for loop processing) may be determined in the computation graph. Location information of the loop structure subgraph in the computation graph may also be determined.

[0064] In one possible implementation, the position information of the cyclic structure subgraph in the computational graph includes the input edge and output edge of the cyclic structure subgraph. In this example, the cyclic structure subgraph can be regarded as an integral module, and the position of the integral module in the computational graph can be determined by its input edge and output edge. For example, the output edge of node A points to the cyclic structure subgraph, and the output edge of the cyclic structure subgraph points to node B. Then, the input edge of the cyclic structure subgraph is the output edge of node A, and the output edge of the cyclic structure subgraph is the input edge of node B. The position of the cyclic structure subgraph is between nodes A and B. Optionally, the input edge pointing to the Enter node is the input edge of the cyclic structure subgraph, and the node pointed out by the Exit node is the output edge of the cyclic structure subgraph.

[0065] In one possible implementation, when the number of loops of the loop structure subgraph is one, the loop structure subgraph is replaced with the execution node according to the position information of the loop structure subgraph in the computational graph, including: when the number of loops of the loop structure subgraph is one, the input edge of the loop structure subgraph is determined as the input edge of the execution node, and the output edge of the loop structure subgraph is determined as the output edge of the execution node.

[0066] In one possible implementation, after determining the position of the loop structure subgraph in the computation graph, the execution node in the loop structure subgraph can be used to replace the loop structure subgraph, that is, the loop control node is cleared to reduce unnecessary computational overhead and improve computational efficiency. Specifically, the input edge of the loop structure subgraph can be determined as the input edge of the execution node, and the output edge of the loop structure subgraph can be used as the output edge of the execution node. That is, the execution node is used to replace the loop structure subgraph to obtain an updated computation graph. In step S13, the data to be processed is processed using the updated computation graph to obtain a processing result. That is, after reducing the loop control nodes that do not support fusion, the nodes in the updated computation graph can be fused, and the fused operator is used to process the data to be processed to obtain a processing result.

[0067] In the example, if the number of loops is one, then Figure 4 The input edge of the input node Enter is used as the input edge of the execution node Operator, and the output edge of the output node Exit is used as the output edge of the execution node Operator. That is, the loop structure subgraph is replaced by the execution node Operator to obtain the updated computation graph, thereby reducing unnecessary computational overhead.

[0068] In this way, when the number of loops is one, the computational graph can be optimized through the execution nodes in the loop structure subgraph to reduce unnecessary computational overhead. In addition, the number of loop control nodes that do not support fusion can be reduced, thereby improving the fusion effect, reducing the processing overhead of data calls and transfers, and improving computational efficiency.

[0069] According to the data processing method of the embodiment of the present disclosure, the computational graph can be optimized based on the processing conditions of the computational graph. In the execution phase, ineffective assertion nodes, related nodes of assertion nodes, output edges of related nodes of assertion nodes, and output edges of the assertion nodes can be cleared to reduce unnecessary computational overhead without affecting the processing results. In addition, when the number of loops is one, the computational graph can be optimized by executing nodes in the loop structure subgraph to reduce unnecessary computational overhead. Furthermore, assertion nodes and loop control nodes that do not support fusion can be reduced, improving the fusion effect, reducing the processing overhead of data calls and transfers, improving the computational efficiency of the computational graph, and giving full play to the computing performance of the processor.

[0070] Figure 5A and Figure 5B Schematic diagram showing an application example of the data processing method according to an embodiment of the present disclosure. Figure 5AAs shown, the computation graph can be a computation graph for image processing executed by a GPU. During the debugging phase, an image can be input into node op3 and processed sequentially through op2 and op1 to obtain image quality parameters (e.g., clarity, etc.). The image is then input into an assertion node, which determines whether the image meets the quality conditions. If the quality conditions are met, the loop structure subgraph can be entered to further process the image that meets the quality conditions. If the quality conditions are not met, the processing is interrupted and no further processing is performed. If the quality conditions are met, the image can be processed (e.g., scaled) by executing the operator node in the loop structure subgraph. The preset number of loops can be set to N (N is a positive integer). When the number of images input into the loop structure subgraph reaches N, a list of N images scaled by the operator node is output to the loop structure subgraph as the processing result of the computation graph. The processing result can be used to debug the computation graph. In this example, the computation graph can be a neural network. The processing result and image annotation information can be used to determine the network loss and update the network parameters. For example, the neural network can be batch trained with N output results as a batch.

[0071] In one possible implementation, during the execution phase, for example, after the neural network training is completed, an image that meets the quality conditions can be input into the neural network for processing by the neural network. In this case, it can be determined that the image meets the quality conditions, and the number of loops is one, so there is no need to determine whether the quality conditions are met, nor is there any need to perform loop processing. Therefore, ineffective nodes can be optimized to obtain an updated computational graph.

[0072] In one possible implementation, the input nodes of the assertion node can be deeply traversed to trace the information source of the input information of the assertion node, obtain the three nodes op3, op2 and op1, and exclude op2 and op1 whose output edges directly or indirectly point to other nodes. That is, the nodes to be optimized are op3, the assertion node and its output edges.

[0073] In one possible implementation, the loop structure subgraph can be optimized and replaced by the execution node Operator, that is, the input edge of the input node Enter is used as the input edge of the execution node Operator, and the output edge of the output node Exit is used as the output edge of the execution node Operator.

[0074] In one possible implementation, op3, the assertion node and its output edge can be cleared, and the loop structure subgraph can be replaced by the execution node Operator to obtain the updated computation graph, such as Figure 5BAs shown in the computation graph, an image can be input into node op3 and then transferred to other nodes after being processed by op3 and op2. An image can also be directly input into the execution node Operator, where it is processed to obtain the result.

[0075] In one possible implementation, the data processing method can optimize assertion nodes and loop structure subgraphs, reducing unnecessary computational overhead and improving processing efficiency. It can be used to optimize processing processes on processors such as GPUs and MLUs, for example, optimizing image processing via neural networks. This disclosure does not limit the scope of application of the data processing method.

[0076] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0077] It should be further explained that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0078] Figure 6 FIG. 1 is a block diagram of a data processing device according to an embodiment of the present disclosure. Figure 6 As shown, the device includes: a condition module 11, which is used to determine the processing conditions of the calculation graph; an update module 12, which is used to update the calculation graph according to the processing conditions of the calculation graph and the types of multiple nodes in the calculation graph to obtain an updated calculation graph; an acquisition module 13, which is used to input the data to be processed into the updated calculation graph to obtain the processing results of the data to be processed.

[0079] In one possible implementation, the processing condition includes a processing stage of a computational graph, and the processing stage includes a debugging stage and an execution stage, wherein the update module is further configured to: when the processing stage of the computational graph is an execution stage, determine an assertion node from the multiple nodes according to the types of the multiple nodes in the computational graph; traverse the input nodes of the assertion node to obtain the related nodes of the assertion node; clear the assertion node, the related nodes of the assertion node, the input edges and output edges of the related nodes of the assertion node, and the input edges and output edges of the assertion node to obtain the updated computational graph.

[0080] In one possible implementation, the update module is further configured to: perform a deep traversal process on the input nodes of the assertion node to obtain a first node, wherein the output edge of the first node directly or indirectly points to the assertion node; determine in the first node whether there is a first node whose output edge directly or indirectly points to a second node, wherein the second node includes any node in the computational graph other than the assertion node and the first node; if there is a first node whose output edge directly or indirectly points to a second node in the first node, exclude the first node whose output edge directly or indirectly points to the second node; and determine the retained first node as a related node of the assertion node.

[0081] In one possible implementation, the processing condition includes the number of loops, wherein the update module is further configured to: determine a loop structure subgraph in the computation graph according to the types of multiple nodes in the computation graph, wherein the loop structure subgraph includes a loop control node and an execution node; when the number of loops of the loop structure subgraph is one, replace the loop structure subgraph with the execution node according to the position information of the loop structure subgraph in the computation graph to obtain the updated computation graph.

[0082] In one possible implementation, the position information of the loop structure subgraph in the computational graph includes the input edge and output edge of the loop structure subgraph, wherein the update module is further configured to: when the number of loops of the loop structure subgraph is one, determine the input edge of the loop structure subgraph as the input edge of the execution node, and determine the output edge of the loop structure subgraph as the output edge of the execution node.

[0083] In a possible implementation, the acquisition module is further configured to fuse the multiple operators supporting fusion of the updated computation graph to obtain a fusion operator; and to process the data to be processed at least according to the fusion operator to obtain the processing result.

[0084] In a possible implementation, the data to be processed includes at least one of images, videos, voices, and texts.

[0085] Figure 7 FIG. 1 is a structural diagram showing a combined processing device 1200 according to an embodiment of the present disclosure. Figure 7 As shown in FIG, the combined processing device 1200 includes a computing processing device 1202, an interface device 1204, other processing devices 1206, and a storage device 1208. According to different application scenarios, the computing processing device may include one or more computing devices 1210, which may be configured to execute the combined processing device 1202, an interface device 1204, other processing devices 1206, and a storage device 1208. Figure 2 The described operation.

[0086] In different embodiments, the computing and processing device of the present disclosure can be configured to perform user-specified operations. In exemplary applications, the computing and processing device can be implemented as a single-core artificial intelligence processor or a multi-core artificial intelligence processor. Similarly, one or more computing devices included in the computing and processing device can be implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core. When multiple computing devices are implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core, with respect to the computing and processing device of the present disclosure, it can be regarded as having a single-core structure or a homogeneous multi-core structure.

[0087] In exemplary operation, the computing processing device of the present invention can interact with other processing devices through an interface device to jointly complete the operation specified by the user. Depending on the implementation, other processing devices of the present invention may include one or more types of processors in general and / or special processors such as a central processing unit (CPU), a graphics processing unit (GPU), an artificial intelligence processor, etc. These processors may include but are not limited to a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As previously mentioned, only with respect to the computing processing device of the present invention, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing processing device and other processing devices are considered together, the two can be regarded as forming a heterogeneous multi-core structure.

[0088] In one or more embodiments, the other processing device may serve as an interface between the computing device of the present disclosure (which may be embodied as an artificial intelligence computing device such as a neural network computing device) and external data and control, performing basic control including but not limited to data transfer, starting and / or stopping the computing device, and so on. In other embodiments, the other processing device may also collaborate with the computing device to jointly complete computing tasks.

[0089] In one or more embodiments, the interface device can be used to transmit data and control instructions between the computing and processing device and other processing devices. For example, the computing and processing device can obtain input data from other processing devices via the interface device and write it to the storage device (or memory) on the computing and processing device chip. Furthermore, the computing and processing device can obtain control instructions from other processing devices via the interface device and write them to the control cache on the computing and processing device chip. Alternatively or optionally, the interface device can also read data from the storage device of the computing and processing device and transmit it to other processing devices.

[0090] Additionally or optionally, the combined processing device of the present disclosure may further include a storage device. As shown in the figure, the storage device is connected to the computing processing device and the other processing device, respectively. In one or more embodiments, the storage device may be used to store data of the computing processing device and / or the other processing device. For example, the data may be data that cannot be fully stored in the internal or on-chip storage device of the computing processing device or other processing device.

[0091] In some embodiments, the present disclosure also discloses an artificial intelligence chip (e.g. Figure 8 The chip 1302 shown in FIG. 1 includes the above-mentioned data processing device. In one implementation, the chip is a system-on-chip (SoC) and integrates one or more combined processing devices. The chip can be connected to the external interface device (such as Figure 8 The external interface device 1306 shown in the figure is connected to other related components. The related components can be, for example, a camera, a display, a mouse, a keyboard, a network card or a wifi interface. In some application scenarios, other processing units (such as video codecs) and / or interface modules (such as DRAM interfaces) can be integrated on the chip. In some embodiments, the present disclosure also discloses a chip packaging structure, which includes the above-mentioned chip. In some embodiments, the present disclosure also discloses a board card, which includes the above-mentioned chip packaging structure. The following will be combined with Figure 8 The board is described in detail.

[0092] Figure 8 FIG. 1 is a schematic diagram showing the structure of a board 1300 according to an embodiment of the present disclosure. Figure 8 As shown in , the board includes a storage device 1304 for storing data, which includes one or more storage units 1310. The storage device can be connected and data can be transmitted with the control device 1308 and the artificial intelligence chip 1302 described above by means of, for example, a bus. Furthermore, the board also includes an external interface device 1306, which is configured for data relay or transfer function between the chip (or the chip in the chip packaging structure) and the external device 1312 (such as a server or computer, etc.). For example, the data to be processed can be passed to the chip by the external device through the external interface device. For another example, the calculation result of the chip can be transmitted back to the external device via the external interface device. Depending on different application scenarios, the external interface device can have different interface forms, for example, it can adopt a standard PCIE interface, etc.

[0093] Each group of storage units is connected to the artificial intelligence chip via a bus. It is understood that each group of storage units can be DDR SDRAM (English: Double Data Rate SDRAM, double data rate synchronous dynamic random access memory).

[0094] DDR can double the speed of SDRAM without increasing the clock frequency. DDR allows data to be read out on the rising and falling edges of the clock pulse. The speed of DDR is twice that of standard SDRAM. In one embodiment, the storage device may include 4 groups of storage units. Each group of storage units may include multiple DDR4 particles (chips). In one embodiment, the artificial intelligence chip may include 4 72-bit DDR4 controllers, and 64 bits of the above 72-bit DDR4 controllers are used for data transmission and 8 bits are used for ECC verification. It can be understood that when DDR4-3200 particles are used in each group of storage units, the theoretical bandwidth of data transmission can reach 25600MB / s.

[0095] In one embodiment, each group of the memory cells includes a plurality of double data rate synchronous dynamic random access memories (DDRs) connected in parallel. DDRs can transmit data twice within one clock cycle. A controller for controlling the DDRs is provided in the chip to control data transmission and data storage in each of the memory cells.

[0096] The interface device is electrically connected to the artificial intelligence chip. The interface device is used to realize data transmission between the artificial intelligence chip and an external device (such as a server or a computer). For example, in one embodiment, the interface device can be a standard PCIE interface. For example, the data to be processed is transmitted to the chip by the server through the standard PCIE interface to realize data transfer. Preferably, when the PCIE 3.0X 16 interface is used for transmission, the theoretical bandwidth can reach 16000MB / s. In another embodiment, the interface device can also be other interfaces. The present disclosure does not limit the specific forms of expression of the above-mentioned other interfaces. The interface unit can realize the switching function. In addition, the calculation results of the artificial intelligence chip are still transmitted back to the external device (such as a server) by the interface device.

[0097] The control device is electrically connected to the artificial intelligence chip. The control device is used to regulate the state of the artificial intelligence chip. Specifically, the artificial intelligence chip and the control device can be electrically connected through an SPI interface. The control device may include a single-chip microcomputer (MCU). For example, the artificial intelligence chip may include multiple processing chips, multiple processing cores or multiple processing circuits, which can drive multiple loads. Therefore, the artificial intelligence chip can be in different working states such as multi-load and light load. The control device can realize the regulation of the working state of multiple processing chips, multiple processing and / or multiple processing circuits in the artificial intelligence chip.

[0098] According to the above combination Figure 7 and Figure 8 Based on the description, those skilled in the art can understand that the present disclosure also discloses an electronic device or apparatus, which may include one or more of the above-mentioned boards, one or more of the above-mentioned chips and / or one or more of the above-mentioned combined processing devices.

[0099] Depending on the application scenario, the electronic devices or apparatuses disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, driving recorders, navigators, sensors, cameras, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, household appliances, and / or medical devices. The vehicles include airplanes, ships, and / or cars; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; and the medical devices include magnetic resonance imaging (MRI) machines, ultrasound machines, and / or electrocardiographs.

[0100] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0101] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0102] The electronic device may be provided as a terminal, a server, or other forms of devices.

[0103] Figure 9 FIG1 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Figure 9 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0104] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0105] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0106] It should be noted that, for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will understand that the scheme of the present disclosure is not limited by the order of the actions described. Therefore, based on the disclosure or teachings of the present disclosure, those skilled in the art will understand that some of the steps therein can be performed in other orders or simultaneously. Further, those skilled in the art will understand that the embodiments described in the present disclosure can be regarded as optional embodiments, that is, the actions or modules involved therein are not necessarily necessary for the implementation of one or more schemes of the present disclosure. In addition, depending on the different schemes, the present disclosure also has different emphases on the description of some embodiments. In view of this, those skilled in the art will understand that the parts that are not described in detail in a certain embodiment of the present disclosure may also refer to the relevant descriptions of other embodiments.

[0107] In terms of specific implementation, based on the disclosure and teachings of this disclosure, those skilled in the art can understand that several embodiments disclosed in this disclosure can also be implemented in other ways not disclosed herein. For example, with respect to the various units in the electronic device or device embodiments described above, this document divides them based on the consideration of logical functions, and there may be other ways of division in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. With respect to the connection relationship between different units or components, the connection discussed above in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components.

[0108] In the present disclosure, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network elements. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the solutions described in the embodiments of the present disclosure. In addition, in some scenarios, multiple units in the embodiments of the present disclosure may be integrated into one unit or each unit may exist physically separately.

[0109] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a particular embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined in any way. To keep the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] The electronic device or device disclosed in the present invention can also be applied to the Internet, Internet of Things, data centers, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, medical and other fields. Furthermore, the electronic device or device disclosed in the present invention can also be used in application scenarios related to artificial intelligence, big data and / or cloud computing, such as the cloud, edge, and terminal. In one or more embodiments, the electronic device or device with high computing power according to the solution disclosed in the present invention can be applied to cloud devices (such as cloud servers), while the electronic device or device with low power consumption can be applied to terminal devices and / or edge devices (such as smart phones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-end integration.

[0111] The foregoing content can be better understood in accordance with the following terms:

[0112] For example, Clause A1, a method of data processing, the method comprising determining the processing conditions of a computational graph; updating the computational graph according to the processing conditions of the computational graph and the types of multiple nodes in the computational graph to obtain an updated computational graph; inputting the data to be processed into the updated computational graph to obtain the processing results of the data to be processed.

[0113] Clause A2, according to the method described in Clause A1, the processing conditions include the processing stage of the computational graph, the processing stage includes the debugging stage and the execution stage, and the operation of updating the computational graph according to the processing conditions of the computational graph and the types of multiple nodes in the computational graph to obtain the updated computational graph includes: when the processing stage of the computational graph is the execution stage, according to the types of multiple nodes in the computational graph, determining an assertion node among the multiple nodes; traversing the input nodes of the assertion node to obtain the related nodes of the assertion node; clearing the assertion node, the related nodes of the assertion node, the input edges and output edges of the related nodes of the assertion node, and the input edges and output edges of the assertion node to obtain the updated computational graph.

[0114] Clause A3, according to the method described in Clause A2, the operation of traversing the input nodes of the assertion node to obtain the related nodes of the assertion node includes: performing deep traversal processing on the input nodes of the assertion node to obtain a first node, wherein the output edge of the first node directly or indirectly points to the assertion node; excluding the first node in the first node whose output edge directly or indirectly points to the second node, wherein the second node includes any node in the computation graph other than the assertion node and the first node; and determining the retained first node as the related node of the assertion node.

[0115] Clause A4, according to the method described in Clause A1, the processing conditions include the number of loops, and the operation of updating the computational graph according to the processing conditions of the computational graph and the types of multiple nodes in the computational graph to obtain an updated computational graph includes: determining a loop structure subgraph in the computational graph according to the types of multiple nodes in the computational graph, wherein the loop structure subgraph includes a loop control node and an execution node; when the number of loops of the loop structure subgraph is one, updating the loop structure subgraph to the execution node according to the position information of the loop structure subgraph in the computational graph to obtain the updated computational graph.

[0116] Clause A5, according to the method described in Clause A4, the position information of the loop structure subgraph in the computational graph includes the input edge and output edge of the loop structure subgraph, and the operation of updating the loop structure subgraph to the execution node according to the position information of the loop structure subgraph in the computational graph when the number of loops of the loop structure subgraph is one, includes: when the number of loops of the loop structure subgraph is one, determining the input edge of the loop structure subgraph as the input edge of the execution node, and determining the output edge of the loop structure subgraph as the output edge of the execution node.

[0117] Clause A6, according to the method described in Clause A1, inputting the data to be processed into the updated computational graph to obtain the processing result of the data to be processed, including: fusing multiple operators that support fusion of the updated computational graph to obtain a fusion operator; processing the data to be processed at least according to the fusion operator to obtain the processing result.

[0118] Item A7, according to the method described in items A1-A7, the data to be processed includes at least one of images, videos, voice and text.

[0119] Item A8, a data processing device, comprising: a condition module, used to determine the processing conditions of a calculation graph; an update module, used to update the calculation graph according to the processing conditions of the calculation graph and the types of multiple nodes in the calculation graph to obtain an updated calculation graph; an acquisition module, used to input the data to be processed into the updated calculation graph to obtain the processing results of the data to be processed.

[0120] Item A9, according to the device described in Item A8, the processing conditions include the processing stage of the computation graph, the processing stage includes the debugging stage and the execution stage, wherein the update module is further configured to: when the processing stage of the computation graph is the execution stage, determine the assertion node among the multiple nodes according to the types of the multiple nodes in the computation graph; traverse the input nodes of the assertion node to obtain the related nodes of the assertion node; clear the assertion node, the related nodes of the assertion node, the input edges and output edges of the related nodes of the assertion node, and the input edges and output edges of the assertion node to obtain the updated computation graph.

[0121] Item A10, according to the device described in Item A9, the update module is further configured to: perform deep traversal processing on the input nodes of the assertion node to obtain a first node, wherein the output edge of the first node directly or indirectly points to the assertion node; determine in the first node whether there is a first node whose output edge directly or indirectly points to a second node, wherein the second node includes any node in the computational graph other than the assertion node and the first node; if there is a first node whose output edge directly or indirectly points to a second node in the first node, exclude the first node whose output edge directly or indirectly points to the second node; determine the retained first node as a related node of the assertion node.

[0122] Item A11, according to the device described in Item A8, the processing condition includes the number of loops, wherein the update module is further configured to: determine a loop structure subgraph in the calculation graph based on the types of multiple nodes in the calculation graph, wherein the loop structure subgraph includes a loop control node and an execution node; when the number of loops of the loop structure subgraph is one, replace the loop structure subgraph with the execution node based on the position information of the loop structure subgraph in the calculation graph to obtain the updated calculation graph.

[0123] Clause A12, according to the device described in Clause A11, the position information of the loop structure subgraph in the computational graph includes the input edge and output edge of the loop structure subgraph, wherein the update module is further configured to: when the number of loops of the loop structure subgraph is one, determine the input edge of the loop structure subgraph as the input edge of the execution node, and determine the output edge of the loop structure subgraph as the output edge of the execution node.

[0124] Item A13, according to the device described in Item A8, the acquisition module is further configured to fuse the multiple operators supporting fusion of the updated computation graph to obtain a fusion operator; and process the data to be processed at least according to the fusion operator to obtain the processing result.

[0125] Item A14, according to the device described in items A8-A13, the data to be processed includes at least one of images, videos, voice and text.

[0126] Although a number of embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art may conceive of many modifications, changes, and alternatives without departing from the ideas and spirit of the present disclosure. It should be understood that in practicing the present disclosure, various alternatives to the embodiments of the present disclosure described herein may be adopted. The appended claims are intended to define the scope of protection of the present disclosure and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A data processing method, characterized in that: include: determining a processing condition for a computation graph, wherein the computation graph is a computation graph of a neural network; updating the computation graph according to the processing conditions of the computation graph and the types of the plurality of nodes in the computation graph to obtain an updated computation graph; Inputting the data to be processed into the updated calculation graph to obtain a processing result of the data to be processed; The processing conditions include a processing stage of the computation graph, the processing stage includes a debugging stage and an execution stage, the debugging stage is a training stage of the neural network, and the computation graph is updated according to the processing conditions of the computation graph and the types of multiple nodes in the computation graph to obtain an updated computation graph, including: When the processing phase of the computation graph is an execution phase, determining an assertion node from among the multiple nodes according to types of the multiple nodes in the computation graph; Traversing the input nodes of the assertion node to obtain related nodes of the assertion node; Clearing the assertion node, related nodes of the assertion node, input edges and output edges of related nodes of the assertion node, and input edges and output edges of the assertion node to obtain the updated computation graph; The step of inputting the data to be processed into the updated computation graph to obtain a processing result of the data to be processed comprises: fusing a plurality of operators supporting fusion in the updated computation graph to obtain a fusion operator; and processing the data to be processed at least according to the fusion operator to obtain the processing result. The data to be processed includes at least one of images, videos, voices and texts.

2. The method according to claim 1, characterized in that Traversing the input nodes of the assertion node to obtain related nodes of the assertion node includes: Performing a deep traversal process on the input nodes of the assertion node to obtain a first node, wherein an output edge of the first node directly or indirectly points to the assertion node; Determine, in the first node, whether there is a first node with an output edge directly or indirectly pointing to a second node, wherein the second node includes any node in the computation graph except the assertion node and the first node; If there is a first node in the first node where an output edge directly or indirectly points to a second node, the first node where the output edge directly or indirectly points to the second node is excluded; The first retained node is determined as a related node of the assertion node.

3. The method according to claim 1, characterized in that The treatment conditions include the number of cycles, Updating the computation graph according to the processing conditions of the computation graph and the types of multiple nodes in the computation graph to obtain an updated computation graph includes: Determining a loop structure subgraph in the computation graph according to types of multiple nodes in the computation graph, wherein the loop structure subgraph includes a loop control node and an execution node; When the number of loops of the loop structure subgraph is one, the loop structure subgraph is replaced with the execution node according to position information of the loop structure subgraph in the computation graph to obtain the updated computation graph.

4. The method according to claim 3, characterized in that The position information of the cyclic structure subgraph in the computation graph includes the input edge and the output edge of the cyclic structure subgraph, When the number of loops of the loop structure subgraph is one, replacing the loop structure subgraph with the execution node according to position information of the loop structure subgraph in the computation graph includes: When the number of cycles of the cyclic structure subgraph is one, the input edge of the cyclic structure subgraph is determined as the input edge of the execution node, and the output edge of the cyclic structure subgraph is determined as the output edge of the execution node.

5. A data processing device, characterized in that: include a condition module for determining a processing condition of a computation graph, wherein the computation graph is a computation graph of a neural network; An updating module, configured to update the computation graph according to the processing conditions of the computation graph and the types of the plurality of nodes in the computation graph, to obtain an updated computation graph; An acquisition module, configured to input the data to be processed into the updated computation graph to obtain a processing result of the data to be processed; The processing condition includes a processing stage of the computation graph, the processing stage includes a debugging stage and an execution stage, the debugging stage is a training stage of the neural network, and the update module is further configured to: when the processing stage of the computation graph is the execution stage, determine an assertion node from the multiple nodes according to the types of the multiple nodes in the computation graph; traverse the input nodes of the assertion node to obtain the related nodes of the assertion node; clear the assertion node, the related nodes of the assertion node, the input edges and output edges of the related nodes of the assertion node, and the input edges and output edges of the assertion node to obtain the updated computation graph; The obtaining module is further configured to fuse the multiple operators supporting fusion of the updated computation graph to obtain a fused operator; and process the data to be processed at least according to the fused operator to obtain the processing result; The data to be processed includes at least one of images, videos, voices and texts.

6. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images