Data processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a data processing method and device, electronic equipment and a storage medium. The data processing method comprises the following steps: acquiring a computational graph; determining at least one core operator based on the computational graph; determining arrangement modes corresponding to the plurality of operators through at least one round of iterative operation; wherein in each round of iterative operation, one or more core operators are selected from at least one core operator to serve as initial operators; determining an arrangement mode corresponding to each initial operator according to the read-write overhead of each initial operator in a plurality of preset arrangement modes; the initial operator is taken as a starting point, forward and backward propagation is carried out according to the dependency relationship in combination with the arrangement mode corresponding to the initial operator, the arrangement mode corresponding to each operator is obtained in sequence, and in the process of obtaining the arrangement mode corresponding to each operator in sequence, the arrangement mode corresponding to the current operator is obtained in combination with the arrangement mode corresponding to the adjacent operator; the adjacent operators are operators which are adjacent to the current operator according to the dependency relationship and have determined arrangement modes.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a data processing method, a data processing apparatus, an electronic device, and a non-transitory computer-readable storage medium. Background Art

[0002] A tensor is a multilinear map defined on the Cartesian product of some vector space and some dual space. For example, a scalar can be viewed as a 0-dimensional tensor, a vector can be viewed as a 1-dimensional tensor, and a matrix can be viewed as a 2-dimensional tensor. A tensor can have any number of dimensions. Tensor operations are widely used in processors such as parallel processors.

[0003] With the development of artificial intelligence and machine learning, new requirements have been put forward for many parallel processor devices represented by parallel processors (for example, multi-core processors, digital signal processors, etc.). In general computing, the computing unit of a parallel processor requires a large amount of data, and this data is generally stored in the storage component of the parallel processor, such as a memory. There are many data arrangement methods for placing the elements of a tensor in the memory, and the processor provides a variety of memory access methods for tensors. The data arrangement method and memory access method have an important impact on the data handling efficiency of the tensor. Summary of the invention

[0004] At least one embodiment of the present disclosure provides a data processing method, comprising: obtaining a computational graph, wherein the computational graph includes a plurality of operators and dependencies between the plurality of operators; determining at least one core operator based on the computational graph; determining, through at least one round of iterative operations, the arrangement modes corresponding to the plurality of operators, respectively, wherein the arrangement mode corresponding to each operator includes the arrangement mode of the input tensor and the arrangement mode of the output tensor of the operator, and the arrangement mode is used to indicate the layout mode of the tensor in the storage component; wherein, in each round of iterative operations: selecting one or more of the at least one core operator; A plurality of core operators are used as starting operators; the arrangement mode corresponding to each starting operator is determined according to the read and write overhead of each starting operator under a plurality of preset arrangement modes; taking the starting operator as the starting point, in combination with the arrangement mode corresponding to the starting operator, the arrangement mode corresponding to each operator is propagated forward and backward according to the dependency relationship, and the arrangement mode corresponding to each operator is obtained in turn, wherein in the process of obtaining the arrangement mode corresponding to each operator in turn, the arrangement mode corresponding to the current operator is obtained in combination with the arrangement mode corresponding to the adjacent operator, and the adjacent operator is an operator that is adjacent to the current operator according to the dependency relationship and has a determined arrangement mode.

[0005] For example, in the data processing method provided by at least one embodiment of the present disclosure, determining at least one core operator based on the computational graph may include: selecting the at least one core operator from the multiple operators based on predetermined conditions; wherein the predetermined conditions include at least one of the following: a fusion operator, an operator with a higher computational amount when the multiple operators are sorted from high to low in terms of computational amount, and an operator of a predetermined functional type, wherein the fusion operator is an operator obtained by fusing multiple operators through an optimization strategy.

[0006] For example, in the data processing method provided by at least one embodiment of the present disclosure, one or more core operators are selected from the at least one core operator as the starting operator, including: dividing the at least one core operator into at least one priority level according to importance; selecting one or more core operators belonging to the same priority level as the starting operator in each round of iterative operation; wherein the at least one round of iterative operation is executed in sequence from high to low according to the importance of the at least one priority level, the propagation stops when the core operator is reached, and the current round of iterative operation is ended when both the forward propagation and the backward propagation starting with the at least one starting operator reach the core operator.

[0007] For example, in the data processing method provided by at least one embodiment of the present disclosure, the arrangement mode corresponding to each starting operator is determined according to the read and write overhead of each starting operator under a plurality of preset arrangement modes, including: for a target tensor whose arrangement mode has not yet been determined in the starting operator: when performing an iterative operation for the first time, the preset arrangement mode with the smallest read and write overhead is selected as the arrangement mode of the target tensor from the read and write overheads of the target tensor under the plurality of preset arrangement modes, wherein the target tensor includes an input tensor or an output tensor; when performing an iterative operation other than the first time, in response to the tensor corresponding to the target tensor having an undetermined arrangement mode, the preset arrangement mode with the smallest read and write overhead is selected from the read and write overheads of the target tensor under the plurality of preset arrangement modes. As the arrangement mode of the target tensor, the corresponding tensor is loaded as the target tensor, or the target tensor is loaded as the corresponding tensor; in response to the tensor corresponding to the target tensor having a determined arrangement mode, a preset arrangement mode with the smallest read / write overhead is selected from a plurality of read / write overheads corresponding to the target tensor as the arrangement mode of the target tensor, wherein the plurality of read / write overheads include the read / write overhead of the target tensor under a first preset arrangement mode, and the read / write overhead of the target tensor when converted to a plurality of arrangement modes, the first preset arrangement mode is the arrangement mode determined to be used by the corresponding tensor, and the conversion of the plurality of arrangement modes includes converting the first preset arrangement mode into a plurality of other preset arrangement modes.

[0008] For example, in the data processing method provided by at least one embodiment of the present disclosure, a preset arrangement method with the smallest read / write overhead is selected as the arrangement method of the target tensor from the read / write overhead of the target tensor under the multiple preset arrangement methods, including: excluding at least one preset arrangement method from the multiple preset arrangement methods according to the operator characteristics of the starting operator; and selecting the preset arrangement method with the smallest read / write overhead as the arrangement method of the target tensor from the read / write overhead of the starting operator under the remaining preset arrangement methods.

[0009] For example, in the data processing method provided by at least one embodiment of the present disclosure, taking the starting operator as the starting point, in combination with the arrangement mode corresponding to the starting operator, propagating forward and backward according to the dependency relationship, and obtaining the arrangement mode corresponding to each operator in turn, including: in combination with the dependency relationship, topologically sorting the calculation graph according to a preset topological sorting type; taking the starting operator as the starting point, performing propagation operations on each operator in turn forward and backward according to the result of the topological sorting; wherein the propagation operation includes: determining the current operator, wherein, when propagating backward with the starting operator as the starting point, the current operator is an operator that uses the output tensor of the adjacent operator as the input tensor of the current operator, and when propagating forward with the starting operator as the starting point, the current operator is an operator that outputs the output tensor to the adjacent operator as the input tensor of the adjacent operator; in response to the current operator not being a core operator, determining the arrangement mode corresponding to the current operator based on the arrangement mode corresponding to the adjacent operator; in response to the current operator being a core operator, stopping this round of iterative operations.

[0010] For example, in the data processing method provided by at least one embodiment of the present disclosure, the arrangement mode corresponding to the current operator is determined based on the arrangement mode corresponding to the adjacent operator, including: in response to taking the output tensor of the adjacent operator as the input tensor of the current operator: from the first read-write cost corresponding to the input tensor of the current operator and the second read-write cost corresponding to the input tensor of the current operator, select the preset arrangement mode with the smallest read-write cost as the arrangement mode of the input tensor of the current operator, wherein the first read-write cost is the read-write cost under the first preset arrangement mode, the output tensor of the adjacent operator is determined to use the first preset arrangement mode, and the second read-write cost is the read-write cost when converting the first preset arrangement mode into at least one other preset arrangement mode; from the read-write costs of the output tensor of the current operator under the multiple preset arrangements, select the preset arrangement mode with the smallest read-write cost as the arrangement mode of the output tensor of the current operator.

[0011] For example, in the data processing method provided by at least one embodiment of the present disclosure, the arrangement mode corresponding to the current operator is determined based on the arrangement mode corresponding to the adjacent operator, including: in response to the output tensor of the current operator as the input tensor of the adjacent operator: from the third read-write cost corresponding to the output tensor of the current operator and the fourth read-write cost corresponding to the output tensor of the current operator, select the preset arrangement mode with the smallest read-write cost as the arrangement mode of the output tensor of the current operator, wherein the third read-write cost is the read-write cost under the second preset arrangement mode, the input tensor of the adjacent operator is determined to use the second preset arrangement mode, and the fourth read-write cost is the read-write cost when converting at least one other preset arrangement mode to the second preset arrangement mode; from the read-write costs of the input tensor of the current operator under the multiple preset arrangements, select the preset arrangement mode with the smallest read-write cost as the arrangement mode of the input tensor of the current operator.

[0012] For example, in the data processing method provided by at least one embodiment of the present disclosure, in response to the number of executions of the iterative operation reaching a preset threshold and the presence of at least one operator in the calculation graph whose arrangement method has not yet been determined, the arrangement method of the at least one operator is determined based on a predetermined rule, wherein the predetermined rule specifies conditional requirements and corresponding arrangement methods, and the operators that meet the corresponding conditional requirements use the corresponding arrangement method.

[0013] For example, in the data processing method provided by at least one embodiment of the present disclosure, before determining the arrangement methods corresponding to the multiple operators respectively through at least one round of iterative operations, the data processing method also includes: determining the shape size of the tensors related to the multiple operators; for each operator, according to the shape size of the tensor related to the operator, obtaining the read and write overhead of the related tensor under the multiple preset arrangement methods.

[0014] For example, in the data processing method provided by at least one embodiment of the present disclosure, the related tensors include input tensors and output tensors, and determining the shape size of the tensors related to the multiple operators includes: deriving the shape size of the input tensor and output tensor of each operator according to the shape size of the computation graph and the input activation tensor of the computation graph.

[0015] For example, in the data processing method provided by at least one embodiment of the present disclosure, the read and write overheads of the related tensors in the multiple preset arrangements are obtained according to the shape size of the tensors related to the operator, including: determining the amount of data that needs to be transmitted for the related tensors in each preset arrangement according to the shape size of the tensors related to the operator; and obtaining the read and write overheads of the related tensors in each preset arrangement based on the amount of data that needs to be transmitted and hardware information, wherein the hardware information includes the data transmission bandwidth in each preset arrangement.

[0016] For example, in the data processing method provided by at least one embodiment of the present disclosure, before determining the arrangement modes corresponding to the multiple operators respectively through at least one round of iterative operations, the data processing method also includes: according to the shape size of the tensor related to the operator, obtaining the read and write overhead of the related tensor when the arrangement mode conversion occurs, wherein the arrangement mode conversion includes storing the input tensor of the operator in a first preset arrangement mode in a storage component, and loading the input tensor from the storage component in a second preset arrangement mode, and the first preset arrangement mode and the second preset arrangement mode are different.

[0017] For example, in the data processing method provided in at least one embodiment of the present disclosure, the arrangement method includes at least one of a memory access method and a data arrangement method, and the data arrangement method indicates the storage order and dimensional arrangement of data in the storage component.

[0018] At least one embodiment of the present disclosure provides a data processing device, including: an acquisition unit, configured to acquire a calculation graph, wherein the calculation graph includes multiple operators and dependencies between the multiple operators; a core operator determination unit, configured to determine at least one core operator based on the calculation graph; an iterative operation unit, configured to determine the arrangement modes corresponding to the multiple operators through at least one round of iterative operation, wherein the arrangement mode corresponding to each operator includes the arrangement mode of the input tensor and the arrangement mode of the output tensor of the operator, and the arrangement mode includes at least one of a memory access mode and a data arrangement mode, and the data arrangement mode indicates the storage order and dimension arrangement of data in the storage component. Column; wherein, in each round of iterative operation: one or more core operators are selected from the at least one core operator as the starting operator; according to the read and write overhead of each starting operator under a plurality of preset arrangements, the arrangement mode corresponding to each starting operator is determined; starting from the starting operator, in combination with the arrangement mode corresponding to the starting operator, forward and backward propagation is performed according to the dependency relationship to obtain the arrangement mode corresponding to each operator in turn, wherein, in the process of obtaining the arrangement mode corresponding to each operator in turn, the arrangement mode corresponding to the current operator is obtained in combination with the arrangement mode corresponding to the adjacent operator, and the adjacent operator is an operator that is adjacent to the current operator according to the dependency relationship and has a determined arrangement mode.

[0019] At least one embodiment of the present disclosure provides an electronic device, comprising: a memory, which non-transitorily stores computer-executable instructions; and a processor, which is configured to execute the computer-executable instructions, wherein the computer-executable instructions, when executed by the processor, implement the data processing method according to at least one embodiment of the present disclosure.

[0020] At least one embodiment of the present disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the data processing method according to at least one embodiment of the present disclosure is implemented.

[0021] In the data processing method provided by at least one embodiment of the present disclosure, a starting operator is first selected from the core operators, and the arrangement mode corresponding to the starting operator is determined. Then, the arrangement mode is propagated with the starting operator as the starting point. In addition, when determining the arrangement mode corresponding to each operator, the arrangement mode of adjacent operators is considered in combination with the calculation graph. That is, when determining the arrangement mode of each operator, not only the operator itself is considered, but also the dependency relationship between the operators and the arrangement mode of adjacent operators are combined to determine the arrangement mode of the current operator. The arrangement mode of each operator obtained is not only suitable for the operator itself, but the optimal arrangement mode of each operator for the entire calculation graph can be obtained, thereby improving data processing efficiency and system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure.

[0023] Figure 1 A schematic diagram of the structure of a general-purpose graphics processing unit (GPGPU); Figure 2A is a schematic structure of a tensor; Figure 2B This is a schematic diagram of the storage format of NCHW; Figure 2C This is a schematic diagram of the storage format of NHWC; Figure 2D This is a schematic diagram of the storage format of NC / 32HW32; Figure 3 A schematic flow chart of a data processing method provided for at least one embodiment of the present disclosure; Figure 4A A schematic diagram of a directed acyclic graph of a neural network is shown; Figure 4B Shown according to Figure 4A The implementation process of the directed acyclic graph shown in the figure is to fuse operators to obtain the computational graph; Figure 5 A schematic flowchart of an iterative operation provided for at least one embodiment of the present disclosure; Figure 6 A schematic diagram of a sorting result after a preset topological sorting is shown in an embodiment of the present disclosure; Figure 7 A schematic diagram of the execution process of a data processing method provided by at least one embodiment of the present disclosure; Figure 8 A schematic block diagram of a data processing device provided for at least one embodiment of the present disclosure; Fig. 9 A schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure; Fig.10 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0025] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits the detailed description of some known functions and known components.

[0026] Figure 1 A schematic structural diagram of a general-purpose graphics processing unit (GPGPU).

[0027] like Figure 1 As shown, the general-purpose graphics processor is actually an array of programmable multiprocessors. For example, the programmable multiprocessor can be a streaming processor cluster (SPC), for example, including Figure 1 The stream processor clusters 1, ..., and stream processor cluster M are shown, where M is a positive integer greater than 1. In a general-purpose graphics processor, one stream processor cluster processes one computing task, or multiple stream processor clusters process one computing task. Multiple stream processor clusters share data through a global cache or a global memory.

[0028] like Figure 1 As shown, taking stream processor cluster 1 as an example, one stream processor cluster includes multiple computing units, such as Figure 1In the figure, there are computing units 1, 2, ..., N, where N is a positive integer. Each computing unit (CU) is used to perform arithmetic and logical operations, such as accumulation, reduction, conventional addition, subtraction, multiplication, and division. A computing unit includes multiple cores (also called computing cores or computing cores), each of which includes an arithmetic logic unit (ALU), a floating-point computing unit, etc. The computing core is used to perform specific computing tasks. In addition, the computing unit also includes registers (e.g. Figure 1 The register file in the computing unit and the shared memory are used to hierarchically store source data and destination data related to computing tasks. The shared memory in a computing unit is used to share data between the cores of the computing unit.

[0029] In parallel computing, computing tasks are generally performed by multiple threads. These threads are divided into multiple thread blocks before being executed in a general-purpose graphics processor (or parallel computing processor), and then distributed through the thread block distribution module ( Figure 1 The CPU (not shown) distributes multiple thread blocks to each computing unit. All threads in a thread block must be assigned to the same computing unit for execution. At the same time, the thread block is split into minimum execution thread bundles (or simply thread bundles, warps), each of which contains a fixed number of threads (or less than this fixed number), for example, 32 threads. Multiple thread blocks can be executed in the same computing unit or in different computing units.

[0030] In each compute unit, the warp scheduling / dispatching module ( Figure 1 The thread warps are scheduled and allocated (not shown) so that multiple computing cores of the computing unit can run the thread warps. Depending on the number of computing cores in the computing unit, multiple thread warps in a thread block can be executed simultaneously or in time-sharing. Multiple threads in each thread warp will execute the same instruction. The memory execution instruction will be emitted to the shared memory in the computing unit or further emitted to the intermediate level cache or global cache or global memory for read and write operations, etc.

[0031] The elements of a tensor are stored in memory (e.g. Figure 1 There are many formats for placing data in memory.

[0032] For example, Figure 2A is a schematic structure of a tensor. Assume that a tensor can be represented as N×C×H×W, where N is the batch size, for example, 1, C is the number of feature maps (i.e., the number of channels), for example, 64, H is the image height, for example, 5, and W is the image width, for example, 4. The pixel elements of a tensor are represented as 0, 1, 2, 3, ..., and so on. Figure 2AThe tensors shown describe different layout formats.

[0033] For example, the data arrangement format may include NCHW (channel-height-width). Figure 2B This is a schematic diagram of the storage format of NCHW.

[0034] For example, for NCHW, refer to Figure 2B , starting from the first channel (c=0), the elements in this channel are arranged consecutively in row-major order, that is, in the order of 0, 1, 2, 3, ..., 19, and then continue with the second channel (c=1) and subsequent channels until the elements of all channels are arranged. If N>1, continue to the next batch.

[0035] For example, the data arrangement format may also include NHWC. Figure 2C This is a schematic diagram of the storage format of NHWC.

[0036] For example, for NHWC (height-width-channel), Figure 2C As shown, from the first element of the first channel (c=0) Figure 2C ), and then stores the first element of the second channel (c=1) ( Figure 2C 20), and so on, until the first elements of all channels are laid out, for example, until the first element of the 64th channel (c=63) ( Figure 2C After the element 1260 in , select the second element of the first channel (c=0) ( Figure 2C 1), and then store the second element of the second channel (c=1) ( Figure 2C , element 21 in ), and so on, until the second elements of all channels are laid out, and so on.

[0037] For example, the data arrangement format may also include NC / xHWx (interleaved mode), where x may be 8, 16, 32, etc. as required.

[0038] NC / xHWx is similar to NHWC, but with one key difference: in the memory layout of NC / xHWx, the C channels are divided into C / x groups, each with x channels: the first group consists of channels c=0 to c=x-1, and the second group consists of channels c=x to c=2x-1, and each group is arranged in NHWC format.

[0039] Figure 2D This is a schematic diagram of the storage format of NC / 32HW32.

[0040] like Figure 2DAs shown, the 64 channels are divided into two groups of 32 channels each. The first group consists of channels c=0 to c=31, and the second group consists of channels c=32 to c=63. Each group is then arranged in NHWC format.

[0041] Of course, there may be many other data arrangement formats, such as BLOCK_ROW_MAJOR (block-row major order), BLOCK_COL_MAJOR (block-column major order), etc., which are not listed one by one in this disclosure.

[0042] The processor can also have multiple memory access methods. For example, typical memory access methods include unified memory access (UMA) and non-uniform memory access (NUMA). Figure 1 Taking the graphics processor architecture shown as an example, assuming that the streaming processor cluster is used as the computing core, if unified memory access is used, each streaming processor cluster can access all storage spaces; if non-uniform memory access is used, each streaming processor cluster only accesses the storage space closest to it, so that data transfer efficiency is higher.

[0043] Different types of memory, such as registers, cache, main memory, and secondary storage, have different access speeds. Registers and cache are fast but have small capacity, while secondary storage, such as hard disks, are slow but have large capacity. A balance can be achieved between speed and capacity through a hierarchy of storage.

[0044] The convolution kernel tensor is expressed as N×Cin×Kh×Kw, where N represents the number of batches, Cin represents the number of input channels, Kh represents the height of the convolution kernel, and Kw represents the width of the convolution kernel. Taking conventional convolution calculation as an example, convolution usually performs reduction operations (reduce) in three dimensions: Cin, Kh, and Kw. Assuming that the input activation tensor follows the data arrangement of NCHW, the data required for a convolution may not be continuous, especially when the input activation tensor W is very large, which can easily cause cache misses. In addition, if SIMD (Single Instruction Multiple Data) instructions are used for calculation, assuming that SIMD instructions can process 4 data at a time, due to the discontinuity of the data, the SIMD instruction acceleration effect cannot be achieved. However, if the NC / 4HW4 layout is used, SIMD instructions can be used for acceleration. Therefore, the accurate selection of data arrangement and memory access methods has a crucial impact on data processing efficiency and data handling efficiency.

[0045] A neural network can be viewed as a directed acyclic graph (DAG) consisting of many computing nodes, each of which corresponds to an operator. Operators in a neural network usually refer to the basic mathematical operations or operations used in the network layer of the model. These operators are used to build the various layers and components of the network to achieve data transmission, conversion, and calculation. They are the basic building blocks of the network model, defining the structure and operation flow of the model, including input, output, and intermediate calculations. In the model, the connection relationship between operators forms a directed graph that reflects the calculation order of different operations in the model. By combining these operators, complex and powerful neural network models can be constructed to handle a variety of complex tasks and data. For example, the following are some common neural network operators: linear layer, convolutional layer, pooling layer, recurrent layer, activation function layer, etc.

[0046] The inventors have found that currently, when setting the arrangement of operators, only a single consideration is given to the impact of a certain data arrangement or memory arrangement on computing efficiency. For example, the arrangement of a certain operator is manually specified based on experience, which does not consider the impact of the relationship between operators on the setting of data arrangement or memory access. In fact, a neural network is often composed of a variety of operators, and different operators have different requirements for data arrangement or memory access. This setting method will result in the inability to obtain the optimal data arrangement or memory access method for the operator for the entire calculation graph.

[0047] In addition, when tensors flow between operators, conversion due to different layouts will incur additional overhead, such as additional data movement. However, it is very likely that the layout conversion of a certain operator is the best choice for the entire computational graph. The current operator layout setting cannot automatically determine whether and how to convert the layout, nor can it obtain the optimal operator layout selection for the entire computational graph.

[0048] At least one embodiment of the present disclosure provides a data processing method, a data processing device, an electronic device, and a non-transitory computer-readable storage medium. The data processing method comprises: obtaining a computational graph, wherein the computational graph comprises a plurality of operators and dependencies between the plurality of operators; determining at least one core operator based on the computational graph; determining the arrangement modes corresponding to the plurality of operators respectively through at least one round of iterative operations, wherein the arrangement mode corresponding to each operator comprises the arrangement mode of the input tensor and the arrangement mode of the output tensor of the operator, and the arrangement mode is used to indicate the layout mode of the tensor in the storage component; wherein, in each round of iterative operations: selecting one or more core operators from at least one core operator as starting operators; determining the arrangement mode corresponding to each starting operator according to the read and write overhead of each starting operator under a plurality of preset arrangement modes; taking the starting operator as the starting point, combining the arrangement mode corresponding to the starting operator, propagating forward and backward according to the dependency relationship, and obtaining the arrangement mode corresponding to each operator in turn, wherein, in the process of obtaining the arrangement mode corresponding to each operator in turn, obtaining the arrangement mode corresponding to the current operator in combination with the arrangement mode corresponding to the adjacent operator, and the adjacent operator is an operator that is adjacent to the current operator according to the dependency relationship and has a determined arrangement mode.

[0049] In the data processing method provided by at least one embodiment of the present disclosure, a starting operator is first selected from the core operators, and the arrangement mode corresponding to the starting operator is determined. Then, the arrangement mode is propagated with the starting operator as the starting point. In addition, when determining the arrangement mode corresponding to each operator, the arrangement mode of adjacent operators is considered in combination with the dependency relationship between operators provided by the calculation graph. That is, when determining the arrangement mode of each operator, not only the operator itself is considered, but also the dependency relationship between operators and the arrangement mode of adjacent operators are combined to determine the arrangement mode of the current operator. The arrangement mode of each operator thus obtained is not only suitable for the operator itself, but also is the optimal arrangement mode for the overall operating efficiency of the entire calculation graph, thereby effectively improving data processing efficiency and system performance.

[0050] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

[0051] Figure 3 A schematic flowchart of a data processing method provided for at least one embodiment of the present disclosure.

[0052] like Figure 3 As shown, the data processing method provided by at least one embodiment of the present disclosure includes at least steps S10-S30.

[0053] First, in step S10, a computation graph is obtained.

[0054] For example, a computation graph includes multiple operators and dependencies between the multiple operators.

[0055] A neural network can be viewed as a directed acyclic graph (DAG) consisting of many computing nodes, each of which corresponds to an operator. Specifically, Figure 4A A schematic diagram of a directed acyclic graph of a neural network is shown. Figure 4A As shown, a directed acyclic graph consisting of 13 computing nodes is shown, that is, the corresponding neural network includes 13 operators. In addition, the nodes are connected by lines, which represent the data dependency and data flow between the computing nodes. For example, the output data of node 1 flows to node 2, and the output data of node 2 flows to node 3 and node 6, and so on. It can be determined that node 1 has a data dependency relationship with node 2, and node 2 has a data dependency relationship with node 6 and node 3. It can be understood that Figure 4A The network structure shown in the figure is only illustrative, and the embodiments of the present disclosure do not impose any specific limitation on the structure of the neural network.

[0056] Next, according to Figure 4A The directed acyclic graph shown can be subjected to operator fusion to obtain a fused computational graph. Figure 4B Shown according to Figure 4A The directed acyclic graph shown in the figure is used to perform operator fusion to obtain the implementation process of the computational graph.

[0057] In the related art, for the calculation of neural networks, in order to more efficiently execute the calculation process of operators in the network, operators can be fused according to some rules and patterns, that is, multiple operators are fused into one fusion operator, such as Figure 4B As shown, operator 1 and operator 2 can be fused into a fused operator A. Similarly, operator 3, operator 4, and operator 5 can be fused into a fused operator C. Operator 6, operator 7, operator 8, and operator 9 can be fused into a fused operator B, and so on. In addition, Figure 4B The computation graph in also includes a single operator that is not fused with other operators, for example, operator D and operator E. According to the above operator fusion rules, a computation graph consisting of operators, fused operators, and the lines between them can be obtained. It can be understood that the embodiments of the present disclosure do not limit the specific rules for performing operator fusion, and can be performed in a conventional manner in the relevant technology.

[0058] As an implementation method, the above-mentioned computational graph can be generated based on a directed acyclic graph of a neural network. It is understandable that other methods can also be used to obtain the computational graph according to the embodiments of the present disclosure, as long as the computational graph can characterize the data dependency relationship and execution order between multiple operators in the neural network, and there is no limitation here.

[0059] Of course, in the present invention, the computation graph may be a computation graph after operator fusion, for example Figure 4B The computational graph shown can also be a computational graph without operator fusion, for example Figure 4A The calculation diagram shown is not specifically limited in the present disclosure.

[0060] The data processing method of the present invention can be applied to a variety of application fields. Depending on the application field, the calculation graph can be used to implement different functions.

[0061] For example, the data processing method can be used in computer vision, and the computational graph can be a computational graph obtained based on a neural network in the fields of image recognition, video analysis, image segmentation, target detection, facial recognition, medical image analysis, etc.

[0062] For example, the data processing method can be used for computer vision and natural language processing, and the computational graph can be a computational graph obtained based on a neural network in the fields of machine translation, sentiment analysis, speech recognition, text generation, etc.

[0063] For example, the data processing method can be used for speech recognition and synthesis, and the computational graph can be a computational graph obtained based on a neural network in the fields of speech enhancement, speech recognition, etc.

[0064] For example, the data processing method can also be applied to recommendation systems, medical diagnosis, finance, autonomous driving, robotics and other fields, and the computational graph can be a computational graph obtained based on a neural network in the corresponding technical field, which will not be described in detail here. Step S20, based on the computational graph, determine at least one core operator.

[0065] For example, in some embodiments, step S20 may include: based on predetermined conditions, selecting at least one core operator from multiple operators; wherein the predetermined conditions include at least one of the following: a fusion operator, an operator with a higher computational amount when multiple operators are sorted from high to low in terms of computational amount, and an operator of a predetermined functional type, wherein the fusion operator is an operator obtained by fusing multiple operators through an optimization strategy.

[0066] In the present disclosure, operators with large computational load and complex calculations can be selected as core operators. The arrangement of core operators has a more important impact on the overall operating efficiency of the entire computational graph. For example, core operators can be selected based on the degree of influence of the arrangement of operators on the overall operating efficiency of the entire computational graph. For example, operators with the largest computational load can be selected as core operators, for example, fusion operators can be selected as core operators, and the number of core operators is not specifically limited.

[0067] For example, a fusion operator may be selected as a core operator. The fusion operator itself is obtained by fusing multiple operators and is a complex operator with a large amount of calculation or a complex structure.

[0068] For example, an operator with a larger amount of computation may be selected, such as an operator with a higher amount of computation when multiple operators are sorted from high to low in terms of computational amount, such as the top 30% of operators, the top 50% of operators, etc. The present disclosure does not impose any specific restrictions on this.

[0069] For example, an operator of a predetermined function type may be selected. For example, the operator of the predetermined function type may be an operator of another function type except a RELU (activation function) operator and a POINTWISE (pointwise operation) operator.

[0070] Of course, those skilled in the art may also set other appropriate conditions to select the core operator according to practical experience, and the present disclosure does not impose any specific limitation on this.

[0071] Step S30, determining the arrangement modes corresponding to the multiple operators respectively through at least one round of iterative operation.

[0072] For example, the arrangement method corresponding to each operator includes the arrangement method of the input tensor and the arrangement method of the output tensor of the operator.

[0073] For example, the arrangement mode here includes at least one of a memory access mode and a data arrangement mode. For example, the data arrangement mode indicates the storage order and dimension arrangement of data in the storage component, such as NCHW, NHWC, etc. listed in this disclosure. The memory access mode is such as UMA or NUMA, etc.

[0074] For example, before the iterative operation, the read and write overhead of the tensors (input tensors and output tensors) related to each operator in a variety of preset arrangements can be obtained in advance, so that there is no need to perform real-time calculations during the iterative operation, avoiding repeated calculations and speeding up the iteration process. The read and write overhead includes the time and resources consumed by reading data from the memory to the computing unit and writing the calculation results back to the memory. The arrangement of operators is determined based on the read and write overhead. Read and write operations may become a performance bottleneck. The read and write overhead directly indicates the computing efficiency. The arrangement method with the best computing efficiency can be obtained by determining the arrangement method based on the read and write overhead.

[0075] For example, the preset arrangement method may be a combination of a memory access method and a data arrangement method. For example, one preset arrangement method may be NCHW+UMA, and another preset arrangement method may be NHWC+UMA, etc. Those skilled in the art may set it as needed, and the present disclosure does not impose any specific limitations on this.

[0076] For example, the preset arrangement mode may be only a data arrangement mode or only a memory access mode, and the present disclosure does not impose any specific limitation on this.

[0077] For example, before step S30, the data processing method provided by at least one embodiment of the present disclosure also includes: determining the shape size of tensors related to multiple operators; for each operator, according to the shape size of the tensors related to the operator, obtaining the read and write overhead of the related tensors under multiple preset arrangements.

[0078] For example, determining the shape size of tensors associated with multiple operators may include: deriving the shape size of input tensors and output tensors of each operator based on the shape size of the computation graph and the input activation tensor of the input computation graph.

[0079] For example, by traversing the computation graph and combining the inference function of each operator, the shape and size of the tensors flowing in the computation graph can be inferred.

[0080] For example, for a matrix multiplication and accumulation operator, it is usually expressed as [M,K]×[K,N]=[M,N], where M, N and K are positive integers representing the dimension sizes, and the shape size of the output tensor can be derived from the shape size of the input tensor.

[0081] For example, for a UNARY POINTWISE operator, the shape of the output tensor is the same as the shape of the input tensor.

[0082] For example, for a BINARY POINTWISE operator, the shape of the output tensor is related to the broadcast operation along a certain dimension.

[0083] For example, for a reduction operator, the shape of the output tensor is obtained according to the dimension specified during the reduction operation (reduction axis).

[0084] For example, for a convolution operator, the shape and size of the output tensor are obtained according to the convolution calculation formula.

[0085] Other types of operators can be derived in a similar manner, and are not listed one by one in this disclosure.

[0086] Therefore, according to the shape and size of the input activation tensor of the computation graph and the input computation graph, the shape and size of the input tensor and output tensor of each operator can be derived in turn by traversing each operator.

[0087] After obtaining the shape and size of the input tensor and output tensor of each operator, the read and write overhead of the input tensor and output tensor of each operator under various preset arrangements can be obtained.

[0088] For example, the multiple preset arrangement modes may include multiple data arrangement modes, such as NCHW, NHWC, N(C / x)HWCx, BLOCK_COL_MAJOR, COL_MAJOR, etc. The multiple preset arrangement modes may also include memory access modes, such as UMA or NUMA, etc. For example, the multiple preset arrangement modes may include a combination of multiple data arrangement modes and memory access modes, such as NCHW+UMA, NCHW+NUMA, etc.

[0089] For example, in some embodiments, based on the shape size of the tensor related to the operator, the read and write overhead of the related tensor in multiple preset arrangements is obtained, which may include: determining the amount of data that needs to be transmitted for the related tensor in each preset arrangement according to the shape size of the tensor related to the operator; based on the amount of data that needs to be transmitted and the hardware information, obtaining the read and write overhead of the related tensor in each preset arrangement, wherein the hardware information includes the data transmission bandwidth in each preset arrangement.

[0090] For example, the following formula can be used to calculate the read and write overhead of each operator layout: Cost = M / B Among them, M is the data volume of the tensor in a certain preset arrangement, B is the data transfer bandwidth in the preset arrangement, and Cost represents the reading and writing overhead of the tensor in the preset arrangement.

[0091] Even if the shape and size of the tensors are the same, the amount of data in different preset arrangements may be different. For example, BLOCK_COL_MAJOR has a larger amount of data than COL_MAJOR. This is because padding will be performed due to alignment requirements.

[0092] The data transfer bandwidth of different preset arrangements is also different. For example, the data transfer bandwidth of BLOCK_COL_MAJOR is higher than that of COL_MAJOR.

[0093] For example, in other embodiments, the read and write overhead of an operator can be modeled, the model input is information such as the shape and size of the operator, and the model output is the read and write overhead caused by data handling. For example, machine learning, neural networks, etc. can be used for modeling, and the present disclosure does not specifically limit the model and modeling process. For example, complex operators, such as fusion operators, can be modeled separately, and the above calculation formula can be used to obtain the read and write overhead for other operators.

[0094] For example, in some embodiments, the data processing method provided by at least one embodiment of the present disclosure also includes: according to the shape and size of the tensor related to the operator, obtaining the read and write overhead of the related tensor when the arrangement mode conversion occurs, wherein the arrangement mode conversion includes storing the input tensor of the operator in a first preset arrangement mode in the storage component, and loading the input tensor from the storage component in a second preset arrangement mode, and the first preset arrangement mode and the second preset arrangement mode are different.

[0095] For example, the following formula can be used to calculate the read and write overhead when the operator layout is changed: Cost = M1 / B1+M2 / B2 Among them, M1 is the data volume of the tensor in the first preset arrangement, B1 is the data handling bandwidth in the first preset arrangement, M2 is the data volume of the tensor in the second preset arrangement, B2 is the data handling bandwidth in the second preset arrangement, and Cost represents the read and write overhead when the tensor is converted from the first preset arrangement to the second preset arrangement (or from the second preset arrangement to the first preset arrangement).

[0096] For example, usually the input tensor will undergo a layout conversion. For the input tensor, the read and write overhead with and without layout conversion can be calculated. For the output tensor, only the read and write overhead under each preset layout can be calculated to reduce the amount of calculation.

[0097] Therefore, through the above process, the read and write overhead of the input tensor and output tensor of each operator under different preset arrangement modes and arrangement conversion modes can be obtained.

[0098] For example, in some other embodiments, after the shape and size of each operator are derived, the read and write overheads of some operators may be calculated before a round of iterative operations begins, and then the read and write overheads of some operators may be calculated again before the next round of iterative operations begins. The present disclosure does not impose any specific restrictions on this.

[0099] Figure 5 A schematic flow chart of an iterative operation provided for at least one embodiment of the present disclosure. Figure 5 As shown, the iterative operation may include steps S301-S303.

[0100] In step S301, one or more core operators are selected from at least one core operator as starting operators.

[0101] In step S302, the arrangement mode corresponding to each starting operator is determined according to the read and write overhead of each starting operator under multiple preset arrangement modes.

[0102] In step S303, starting from the initial operator and combining the arrangement mode corresponding to the initial operator, the process is propagated forward and backward according to the dependency relationship to obtain the arrangement mode corresponding to each operator in turn.

[0103] For example, in the process of obtaining the arrangement mode corresponding to each operator in turn, the arrangement mode corresponding to the current operator is obtained by combining the arrangement mode corresponding to the adjacent operator, where the adjacent operator is an operator that is adjacent to the current operator according to a dependency relationship and has a determined arrangement mode.

[0104] Here, for forward propagation, the current operator is an operator that is before and adjacent to the adjacent operator according to the dependency relationship, for example, the output tensor of the current operator is output to the adjacent operator as the input tensor. For backward propagation, the current operator is an operator that is after and adjacent to the adjacent operator according to the dependency relationship, for example, the output tensor of the adjacent operator is output to the current operator as the input tensor. Figure 4A For example, assuming that the starting operator is operator 7, the propagation direction of forward propagation is operator 7->operator 6->operator 2..., and the reverse direction of backward propagation is operator 7->operator 9->operator 10...

[0105] Therefore, when determining the arrangement method corresponding to each operator, the arrangement method of adjacent operators is considered in combination with the calculation graph. That is, when determining the arrangement method of each operator, not only the operator itself is considered, but also the dependency relationship between operators and the arrangement method of adjacent operators are combined to determine the arrangement method of the current operator. The arrangement method of each operator obtained is not only suitable for the operator itself, but the optimal arrangement method of each operator for the entire calculation graph can be obtained, thereby improving data processing efficiency and system performance.

[0106] For example, the propagation stops when it reaches the core operator, and the current round of iterative operation ends when the forward propagation and backward propagation starting from at least one starting operator both reach the core operator. For example, when there are multiple starting points, the forward propagation and backward propagation starting from each starting point can reach the core operator, end the current round of iterative operation, and start the next round of iterative operation. For example, the propagation reaching the core operator here means that if the current operator is the core operator, the propagation is considered to have reached the core operator, and the current round of iterative operation ends.

[0107] For example, the propagation here can be understood as propagating the arrangement mode corresponding to the starting operator to other operators. Since in the process of obtaining the arrangement mode corresponding to each operator in turn, the arrangement mode corresponding to the current operator is obtained by combining the arrangement mode corresponding to the adjacent operator, each operator actually combines and refers to the arrangement mode of the starting operator when determining its own arrangement mode, thereby realizing the propagation of the arrangement mode corresponding to the starting operator.

[0108] For example, in response to multiple operators all determining corresponding arrangements, the iteration operation ends.

[0109] For example, the arrangement mode corresponding to each operator can be determined through one or more rounds of iterative operations. For example, if the setting of the arrangement mode corresponding to all operators has not been completed, and the number of iterations does not exceed the preset threshold, the iterative operation continues to be performed. For example, if the number of executions of the iterative operation reaches the preset threshold and there is at least one operator in the calculation graph whose arrangement mode has not yet been determined, the arrangement mode of at least one operator is determined based on a predetermined rule. For example, the predetermined rule can be a heuristic rule, which specifies the condition requirements and the corresponding arrangement mode, and the operator that meets the corresponding condition requirements uses the corresponding arrangement mode.

[0110] Some possible predetermined rules are listed below in a non-limiting manner, and the present disclosure is not limited thereto.

[0111] For example, taking a graphics processor as an example, it is more efficient to use a block arrangement.

[0112] For example, for a three-dimensional tensor, the arrangement is set to BLOCK_COL_MAJOR+NUMA; for a linear operator, the second input tensor is the weight and needs to be set to UMA.

[0113] For the four-dimensional activation tensor, it is set to BLOCK_NCHW (the layout is data blocks organized in NCHW format) + NUMA. For the four-dimensional convolution kernel tensor, it is set to BLOCK_CONV_WEIGHT (the layout is block-convolution weight) + UMA.

[0114] The predetermined rules can be set according to the hardware, the shape and size of the tensor, the operator type, etc. The predetermined rules change with the hardware update. Those skilled in the art can set them according to actual needs, and this disclosure does not make any specific restrictions.

[0115] The specific process of each round of iterative operation is described in detail below.

[0116] For example, in some embodiments, step S301 may include: dividing at least one core operator into at least one priority level according to importance; selecting one or more core operators belonging to the same priority level as starting operators in each round of iterative operation; wherein at least one round of iterative operation is executed in sequence from high to low importance of at least one priority level, and the propagation stops when the core operator is reached, and the round of iterative operation is ended when both the forward propagation and the backward propagation starting with at least one starting operator reach the core operator.

[0117] For example, core operators may be divided into one or more priorities according to their importance, such as the amount of computation, and in each round of iteration, all core operators in a certain priority level are used as starting operators for this round of iteration.

[0118] For example, the order of iterative operations can be determined from high to low priority. For example, in the first iterative operation, one or more core operators with the highest priority are selected as the starting operators to perform this round of iterative operations, and this round of iterative operations ends when the propagation reaches the core operator; in the second iterative operation, one or more core operators with the second highest priority are selected as the starting operators to perform this round of iterative operations, and this round of iterative operations ends when the propagation reaches the core operator, and so on.

[0119] Therefore, the arrangement of operators with high importance is determined first, such as operators with large computational complexity or complex structure, which have a greater impact on the overall operating efficiency of the calculation graph. The arrangement of these operators can be determined first. The arrangement of these operators has a greater impact on the overall operating efficiency of the calculation graph. Determining the arrangement of these operators first can more accurately obtain the optimal arrangement of each operator for the entire calculation graph.

[0120] For example, in other embodiments, the core operator may be set to have only one priority level, such as when the number of core operators is small. In this case, the arrangement of each operator may be obtained through one round of iterative operations.

[0121] For example, in other embodiments, core operators may be divided into different priorities according to other conditions besides the importance, and the present disclosure does not impose any specific limitation on this.

[0122] For example, in some other embodiments, the iterative operation may select a core operator of any priority level as the starting operator each time, and the present disclosure does not impose any specific limitation on this.

[0123] For example, in some embodiments, step S302 may include: in response to an undetermined arrangement of the tensor corresponding to the target tensor, selecting a preset arrangement with the smallest read / write overhead from the read / write overheads of the target tensor under multiple preset arrangement modes as the arrangement of the target tensor, wherein the corresponding tensor is loaded as the target tensor, or the target tensor is loaded as the corresponding tensor; in response to an determined arrangement of the tensor corresponding to the target tensor, selecting a preset arrangement with the smallest read / write overhead from the multiple read / write overheads corresponding to the target tensor as the arrangement of the target tensor, wherein the multiple read / write overheads include the read / write overhead of the target tensor under a first preset arrangement mode, and the read / write overhead of the target tensor when converting to multiple arrangement modes, the first preset arrangement mode is the arrangement mode determined to be used by the corresponding tensor, and the conversion of multiple arrangement modes includes converting the first preset arrangement mode into multiple other preset arrangement modes.

[0124] For example, when executing the first iteration, all operators have not been arranged, and the input tensors and output tensors of the starting operator can be determined in the same way.

[0125] For example, taking the input tensor as an example, the preset arrangement with the lowest read / write overhead among the read / write overhead of the input tensor of the starting operator under multiple preset arrangements can be selected as the arrangement of the input tensor of the starting operator. The read / write overhead of the input tensor under multiple preset arrangements can be determined by referring to the above content, which will not be repeated here.

[0126] For example, when performing iterative operations for the second time and thereafter, for a target tensor whose arrangement has not yet been determined in the starting operator, for example, the target tensor may be an input tensor or output tensor whose arrangement has not yet been determined in the starting operator, the arrangement of other input tensors or output tensors whose arrangements have been determined must also be considered, for example, the read and write overhead caused by the arrangement conversion must also be considered.

[0127] For example, if the arrangement of the tensor corresponding to the target tensor is not determined, the preset arrangement with the lowest read / write overhead is selected as the arrangement of the target tensor from the read / write overhead of the target tensor under multiple preset arrangements. For example, assuming that the target tensor is the input tensor of operator 1, the output tensor 1 of operator 2 is output to operator 1 as the target tensor, and the corresponding tensor is output tensor 1. For example, assuming that the target tensor is the output tensor of operator 1, the output tensor of operator 1 is output to operator 3 as input tensor 1, and the corresponding tensor is input tensor 1.

[0128] For example, if the tensor corresponding to the target tensor has a determined arrangement, the preset arrangement with the smallest read / write overhead is selected as the arrangement of the target tensor from the multiple read / write overheads corresponding to the target tensor. Here, the multiple read / write overheads include the read / write overhead of the target tensor under the first preset arrangement, and the read / write overhead of the target tensor when converted to multiple arrangement modes. The first preset arrangement is the arrangement determined to be used by the corresponding tensor, and the conversion of multiple arrangement modes includes the conversion of the first preset arrangement to multiple other preset arrangement modes.

[0129] For example, assuming that the read and write overhead under the first preset arrangement is the smallest, it is determined that the target tensor uses the first preset arrangement. For example, assuming that the read and write overhead under the first preset arrangement is the smallest when converted to the second preset arrangement, it is determined that the target tensor uses the second preset arrangement.

[0130] Therefore, in the present disclosure, for core operators with complex performance, such as matrix multiplication operators, it is necessary to comprehensively consider the arrangement combinations of multiple input and output tensors to obtain a more accurate arrangement.

[0131] For example, in some embodiments, some preset arrangements that necessarily have large read and write overheads may be excluded based on the operator characteristics of the starting operator, thereby reducing the comparison scope and speeding up the execution of iterative operations.

[0132] For example, from the read and write costs of the target tensor under multiple preset arrangements, selecting the preset arrangement with the smallest read and write cost as the arrangement of the target tensor can include: excluding at least one preset arrangement from the multiple preset arrangements according to the operator characteristics of the starting operator; and selecting the preset arrangement with the smallest read and write cost as the arrangement of the target tensor from the read and write costs of the starting operator under the remaining preset arrangements.

[0133] For example, for the convolution kernel tensor, the memory access method uses UMA. For example, for the tensors of the matrix multiplication operator, one of the tensors must be UMA. For example, for the reduction operator, if a reduction operation is performed between multiple computing cores, the output tensor must use UMA. After that, you can choose from the remaining preset arrangements. For example, taking the convolution kernel tensor as an example, there is no need to consider the case where the memory access method is UMA. In this way, the number of read and write overheads that need to be compared can be reduced, the process of iterative operations can be accelerated, and efficiency can be improved.

[0134] For example, in some embodiments, step S303 may include: topologically sorting the computation graph according to a preset topological sorting type in combination with dependencies; starting from a starting operator, performing propagation operations on each operator in turn forward and backward according to the result of the topological sorting; wherein the propagation operation includes: determining a current operator, wherein when propagating backward from the starting operator, the current operator is an operator that uses the output tensor of an adjacent operator as the input tensor of the current operator, and when propagating forward from the starting operator, the current operator is an operator that outputs the output tensor to an adjacent operator as the input tensor of the adjacent operator; in response to the current operator not being a core operator, determining an arrangement method corresponding to the current operator based on an arrangement method corresponding to adjacent operators; in response to the current operator being a core operator, stopping this round of iterative operations.

[0135] For example, the preset topological sorting type may be breadth-first traversal. For example, other feasible topological sorting methods may also be used.

[0136] For example, determining the arrangement mode corresponding to the current operator based on the arrangement mode corresponding to the adjacent operator may include: in response to taking the output tensor of the adjacent operator as the input tensor of the current operator: selecting the preset arrangement mode with the smallest read-write cost as the arrangement mode of the input tensor of the current operator from the first read-write cost corresponding to the input tensor of the current operator and the second read-write cost corresponding to the input tensor of the current operator, wherein the first read-write cost is the read-write cost under the first preset arrangement mode, the output tensor of the adjacent operator is determined to use the first preset arrangement mode, and the second read-write cost is the read-write cost when converting the first preset arrangement mode into at least one other preset arrangement mode; selecting the preset arrangement mode with the smallest read-write cost as the arrangement mode of the output tensor of the current operator from the read-write costs of the output tensor of the current operator under multiple preset arrangements.

[0137] For example, determining the arrangement mode corresponding to the current operator based on the arrangement modes corresponding to adjacent operators may include: in response to the output tensor of the current operator being the input tensor of the adjacent operator: selecting the preset arrangement mode with the smallest read-write cost as the arrangement mode of the output tensor of the current operator from the third read-write cost corresponding to the output tensor of the current operator and the fourth read-write cost corresponding to the output tensor of the current operator, wherein the third read-write cost is the read-write cost under the second preset arrangement mode, the input tensor of the adjacent operator is determined to use the second preset arrangement mode, and the fourth read-write cost is the read-write cost when converting at least one other preset arrangement mode to the second preset arrangement mode; selecting the preset arrangement mode with the smallest read-write cost as the arrangement mode of the input tensor of the current operator from the read-write costs of the input tensor of the current operator under multiple preset arrangement modes.

[0138] For example, the propagation process starts from the core operator and ends when the propagation reaches the core operator, for example, starting from the start operator, and performing the propagation operation on each operator in turn forward and backward according to the result of the topological sorting.

[0139] Figure 6 This is a schematic diagram of the sorting result after the preset topological sorting according to an embodiment of the present disclosure. It should be noted that: Figure 6 Only three operators are shown in the figure, but in fact the sorting result may also include more operators, which will not be described again here.

[0140] For example, assuming that operator OP2 is the starting operator, operator OP3 uses the output tensor t3 of operator OP2 as the input tensor t4, and the output tensor t1 of operator OP1 is output to operator OP2 as the input tensor t2. Here, tensor t1 and tensor t2 are actually the same tensor, and tensor t3 and tensor t4 are actually the same tensor, but operator OP1 may store output tensor t1 in a storage component in a certain data arrangement format, and operator OP2 loads input tensor t2 from the storage component in a certain data arrangement format.

[0141] In step S302, the arrangement of the input tensor t2 and the output tensor t3 of the operator OP2 can be obtained. The specific process can refer to the description of step S302, which will not be repeated here.

[0142] In step S303, when propagating backward starting from the operator OP2, it is determined that the current operator is the operator OP3.

[0143] Assuming that operator OP3 is the core operator, it will no longer propagate backward and this round of iteration can be stopped. The specific stopping time is when the core operator is also encountered during forward propagation.

[0144] Assuming that the operator OP3 is not a core operator, for example, a RELU type operator, the arrangement mode corresponding to the operator OP3 is determined based on the arrangement mode corresponding to the starting operator OP2.

[0145] Specifically, in step S302, the arrangement mode used by the output tensor t3 of the operator OP2 has been determined, for example, the preset arrangement mode 1, and the preset arrangement mode with the smallest read-write cost is selected as the preset arrangement mode of the input tensor t4 of the operator OP3 from the first read-write cost cost1 corresponding to the input tensor t4 of the operator OP3 and the second read-write cost cost2 corresponding to the input tensor t4 of the operator OP3. Here, the first read-write cost cost1 is the read-write cost when the input tensor t4 of the operator OP3 uses the preset arrangement mode 1, and the second read-write cost is the read-write cost when the preset arrangement mode 1 is converted to other preset arrangements.

[0146] For example, assuming that the first read-write cost cost1 is the smallest, it is determined that the input tensor t4 of operator OP3 uses the preset arrangement method 1. Assuming that the read-write cost of converting the preset arrangement method 1 to the preset arrangement method 2 is the smallest, it is determined that the input tensor t4 of operator OP3 uses the preset arrangement method 2.

[0147] For example, for the output tensor t5 of the operator OP3, the preset arrangement method with the smallest read / write overhead can be selected from the read / write overhead of the output tensor t5 of the operator OP3 under multiple preset arrangement methods as the arrangement method of the output tensor t5 of the operator OP3.

[0148] In step S303, when propagating forward starting from the operator OP2, it is determined that the current operator is the operator OP1.

[0149] Assuming that operator OP1 is the core operator, forward propagation will no longer occur and this round of iteration can be stopped. The specific stopping time is when the core operator is also encountered during backward propagation.

[0150] Assuming that the operator OP1 is not a core operator, for example, a POINTWISE type operator, the arrangement mode corresponding to the operator OP1 is determined based on the arrangement mode corresponding to the starting operator OP2.

[0151] Specifically, in step S302, the arrangement mode used by the input tensor t2 of the operator OP2 has been determined, for example, the preset arrangement mode 3, and the preset arrangement mode with the smallest read-write cost is selected as the arrangement mode of the output tensor t1 of the operator OP1 from the third read-write cost cost3 corresponding to the output tensor t1 of the operator OP1 and the fourth read-write cost cost4 corresponding to the output tensor t1 of the operator OP1. Here, the third read-write cost cost3 is the read-write cost when the output tensor t1 of the operator OP1 uses the preset arrangement mode 3, and the second read-write cost is the read-write cost when other preset arrangements are converted to the preset arrangement mode 3.

[0152] For example, assuming that the third read-write cost cost3 is the smallest, it is determined that the output tensor t1 of the operator OP1 uses the preset arrangement method 3. Assuming that the read-write cost of converting the preset arrangement method 4 to the preset arrangement method 3 is the smallest, it is determined that the output tensor t1 of the operator OP1 uses the preset arrangement method 4.

[0153] For example, for the input tensor t0 of operator OP1, the preset arrangement mode with the lowest read / write overhead can be selected from the read / write overhead of the input tensor t0 of operator OP1 under multiple preset arrangements as the arrangement mode of the input tensor t0 of operator OP1. Of course, if there is an operator OP0 before operator OP1, the output tensor t' of operator OP0 is output to operator OP1 as the input tensor t0, and if the arrangement mode of the output tensor t' of operator OP1 has been determined, the arrangement mode of the input tensor t0 of operator OP1 can be determined by referring to the similar process mentioned above, taking into account the read / write overhead of the arrangement mode conversion, which will not be described in detail here.

[0154] Afterwards, the operator OP1 may be continuously propagated forward as an adjacent operator to determine the current operator, and the operator OP3 may be propagated backward as an adjacent operator to determine the current operator. The specific process will not be described in detail.

[0155] Therefore, when determining the arrangement of each operator, the corresponding arrangement of adjacent operators is considered, and the predetermined arrangement of core operators that have a greater impact on the overall efficiency of the calculation graph is propagated through the above process. When determining the arrangement of each operator, the arrangement of the core operators and the dependency relationship between the operators are combined to obtain the optimal operator arrangement for the entire calculation graph.

[0156] Figure 7 A schematic diagram of the execution process of a data processing method provided in at least one embodiment of the present disclosure.

[0157] like Figure 7 As shown, first, a calculation graph is obtained. For the relevant process of obtaining the calculation graph, reference may be made to the relevant description of the aforementioned step S10, which will not be repeated here.

[0158] After obtaining the computation graph, the shape and size of each operator are derived one by one according to the shape and size of the computation graph and the input activation tensor of the input computation graph. The specific process can be referred to the above content and will not be repeated here.

[0159] Afterwards, the core operator is determined based on the computation graph. The specific process can refer to the related description of the aforementioned step S20, which will not be repeated here.

[0160] After that, at least one round of iteration is started, such as Figure 7 Shown in the dotted box.

[0161] In each round of iterative operations, the read and write overhead of some operators is first determined. The specific process can be referred to the aforementioned related content and will not be repeated here.

[0162] Afterwards, a starting operator is selected, for example, the starting operator of the first round of iteration is the core operator with the highest priority, the starting operator of the second round of iteration is the core operator with the second highest priority, and so on. For details, please refer to the relevant description in the above step S301, which will not be repeated here.

[0163] Afterwards, the arrangement mode corresponding to the starting operator is determined according to the read and write overhead of the starting operator under multiple preset arrangements. The specific process can refer to the relevant description in the aforementioned step S302, which will not be repeated here.

[0164] Afterwards, the arrangement mode is propagated starting from the starting operator, and the arrangement mode corresponding to the starting operator is propagated forward and backward according to the dependency relationship to obtain the arrangement mode corresponding to each operator in turn. The specific process can refer to the relevant description in the above step S303, which will not be repeated here.

[0165] Afterwards, if both forward and backward propagation reach the core operator, the current round of iteration ends. If the number of iterations does not exceed the preset threshold, the next round of iterations is continued, and the specific process is as described above. If the number of iterations exceeds the preset threshold, it is determined whether the arrangement of all tensors is completed. If so, it ends. If not, the arrangement of the remaining tensors is set according to the predetermined rules, and then it ends.

[0166] In this data processing method, the core operators and the arrangement of the core operators are first determined, and the core operators are also prioritized, so when determining the arrangement of each operator, the arrangement of some operators can be set according to the limited importance, and the arrangement of other operators is determined in sequence according to the arrangement of these operators. In addition, when determining the arrangement of each operator, the arrangement corresponding to the adjacent operators is considered, and through the above process, the arrangement of the core operators that have a greater impact on the overall efficiency of the calculation graph is propagated. When determining the arrangement of each operator, the arrangement of the core operators and the dependency relationship between the operators are combined, and the optimal operator arrangement for the entire calculation graph can be obtained.

[0167] Figure 8 A schematic block diagram of a data processing device provided for at least one embodiment of the present disclosure.

[0168] like Figure 8 As shown, the data processing device 100 includes an acquisition unit 101 , a core operator determination unit 102 and an iterative operation unit 103 .

[0169] The acquisition unit 101 is configured to acquire a computation graph, wherein the computation graph includes multiple operators and dependencies between the multiple operators.

[0170] The core operator determination unit 102 is configured to determine at least one core operator based on the computation graph.

[0171] The iterative operation unit 103 is configured to determine the arrangement modes corresponding to multiple operators respectively through at least one round of iterative operation, wherein the arrangement mode corresponding to each operator includes the arrangement mode of the input tensor and the arrangement mode of the output tensor of the operator, and the arrangement mode includes at least one of a memory access mode and a data arrangement mode, and the data arrangement mode indicates the storage order and dimensional arrangement of the data in the storage component.

[0172] For example, in each round of iterative operations: one or more core operators are selected from at least one core operator as the starting operator; the arrangement mode corresponding to each starting operator is determined according to the read and write overhead of each starting operator under multiple preset arrangements; starting from the starting operator, combined with the arrangement mode corresponding to the starting operator, propagating forward and backward according to the dependency relationship, and obtaining the arrangement mode corresponding to each operator in turn, wherein, in the process of obtaining the arrangement mode corresponding to each operator in turn, the arrangement mode corresponding to the current operator is obtained in combination with the arrangement mode corresponding to the adjacent operator, and the adjacent operator is an operator that is adjacent to the current operator according to the dependency relationship and has a determined arrangement mode.

[0173] For example, the acquisition unit 101, the core operator determination unit 102, and the iterative operation unit 103 include codes and programs stored in the memory, and the acquisition unit 101, the core operator determination unit 102, and the iterative operation unit 103 are implemented as a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities. The processing unit can be a general-purpose processor, and can also be a single-chip microcomputer, a microprocessor, a digital signal processor, a dedicated image processing chip, or a field programmable logic array, etc. The acquisition unit 101, the core operator determination unit 102, and the iterative operation unit 103 execute the code and program to implement some or all of the functions of the acquisition unit 101, the core operator determination unit 102, and the iterative operation unit 103 as described above. For example, the acquisition unit 101, the core operator determination unit 102, and the iterative operation unit 103 can be a circuit board or a combination of multiple circuit boards, used to implement the functions as described above. In an embodiment of the present application, the circuit board or the combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-temporary memories connected to the processors; and (3) firmware stored in the memories that is executable by the processors.

[0174] It should be noted that the acquisition unit 101 can be used to implement Figure 3 Step S10 shown; the core operator determination unit 102 can be used to implement Figure 3 Step S20 shown; the iterative operation unit 103 can be used to implement Figure 3Step S30 shown. Therefore, for the specific description of the functions that can be implemented by the acquisition unit 101, reference can be made to the relevant description of step S10 in the embodiment of the above-mentioned data processing method. For the specific description of the functions that can be implemented by the core operator determination unit 102, reference can be made to the relevant description of step S20 in the embodiment of the above-mentioned data processing method. For the specific description of the functions that can be implemented by the iteration operation unit 103, reference can be made to the relevant description of step S30 in the embodiment of the above-mentioned data processing method. The repeated parts will not be repeated here. In addition, the data processing device 100 can achieve technical effects similar to those of the aforementioned data processing method, which will not be repeated here.

[0175] It should be noted that in at least one embodiment of the present disclosure, the data processing device 100 may include more or fewer circuits or units, and the connection relationship between the various circuits or units is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited and can be composed of analog devices according to circuit principles, or can be composed of digital chips, or can be composed in other applicable ways.

[0176] For example, the data processing device 100 may be implemented in hardware, software, or a combination of hardware and software, and the present disclosure does not impose any specific limitations on this.

[0177] The data processing method provided by at least one embodiment of the present disclosure can achieve a technical effect similar to the data processing method described above, and will not be described in detail here.

[0178] The data processing method and data processing device provided in at least one embodiment of the present disclosure can be applied to different systems or devices, such as Fig. 9 The electronic device 300 shown. The electronic device 300 can be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer, an AR device, a VR device, a vehicle terminal, etc., or a server, etc. For example, the hardware architecture of the electronic device 300 may include a graphics processor. The data processing method provided in at least one embodiment of the present disclosure can be applied to scenarios involving CPU, high performance computing (HPC) and artificial intelligence (AI) in the electronic device 300. Of course, the present disclosure is not limited to this, and any scenario, device, apparatus, etc. involving calculation graphs and model operations can adopt the data processing method or data processing apparatus provided in at least one embodiment of the present disclosure.

[0179] In some embodiments, the data processing device provided by at least one embodiment of the present disclosure may be a chip, for example, the chip is a system-on-a-chip (SoC). The system-on-a-chip includes a processor, which may be a single-core processor or a multi-core processor, a memory, an I / O interface, and the like.

[0180] Fig. 9 The following is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Fig. 9 As shown, the electronic device 300 is suitable for implementing the data processing method provided by the embodiment of the present disclosure. It should be noted that Fig. 9 The components of the electronic device 300 shown are merely exemplary and non-limiting. The electronic device 300 may also have other components according to actual application requirements.

[0181] like Fig. 9 As shown, the electronic device 300 may include a processing device 301 (eg, a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in a memory to implement various functions.

[0182] For example, when the computer readable instructions are executed by the processing device 301, one or more steps in the data processing method described in any of the above embodiments may be executed. It should be noted that for a detailed description of the processing process of the data processing method, reference may be made to the relevant description in the above data processing method embodiments.

[0183] For example, the memory may include any combination of one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory (cache), etc. For example, computer-readable instructions may be loaded from storage device 308 into random access memory (RAM) 303 to run computer-readable instructions. Non-volatile memory may include, for example, read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, flash memory, etc. Various applications and various data, such as style images, and various data used and / or generated by applications, etc., may also be stored in the computer-readable storage medium.

[0184] For example, the processing device 301, the read only memory (ROM) 302, and the random access memory (RAM) 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0185] Typically, the following devices may be connected to the input / output (I / O) interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, a flash memory, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other electronic devices wirelessly or by wire to exchange data. Although Fig. 9 An electronic device 300 having various devices is shown, but it should be understood that it is not required to implement or have all the devices shown, and the electronic device 300 may alternatively implement or have more or fewer devices. For example, the processing device 301 can control other components in the electronic device 300 to perform the desired functions. The processing device 301 can be a device with data processing capabilities and / or program execution capabilities such as a central processing unit (CPU), a tensor processing unit (TPU) or a graphics processing unit GPU. The central processing unit (CPU) can be an X86, ARM, RISC-V architecture, etc. The GPU can be directly integrated into the SOC, directly integrated into the motherboard, or built into the north bridge chip of the motherboard.

[0186] Fig.10 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Fig.10 As shown, the storage medium 400 may be a non-transitory computer-readable storage medium, and one or more computer-readable instructions 401 may be non-transitory stored on the storage medium 400. For example, when the computer-readable instructions 401 are executed by a processor, one or more steps in the data processing method described above may be executed.

[0187] For example, the storage medium 400 may be applied in the electronic device 300 . For example, the storage medium 400 may include the storage device 308 in the electronic device 300 .

[0188] For example, the storage device may include any combination of one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and the processor may run the computer-readable instructions to implement various functions of the processor. Various applications and various data may also be stored in the storage medium.

[0189] For example, the storage medium may include a memory card of a smart phone, a cache component of a tablet computer, a hard disk of a personal computer, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other applicable storage media.

[0190] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0191] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0192] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0193] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0194] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0195] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

[0196] There are a few points to note about this disclosure: (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure. Other structures may refer to the general design.

[0197] (2) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to obtain new embodiments.

[0198] The above description is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A data processing method, comprising: Obtaining a computation graph, wherein the computation graph includes a plurality of operators and dependencies between the plurality of operators; Based on the computation graph, determining at least one core operator; Determine, through at least one round of iterative operations, the arrangement modes corresponding to the multiple operators, respectively, wherein the arrangement mode corresponding to each operator includes the arrangement mode of the input tensor of the operator and the arrangement mode of the output tensor of the operator, and the arrangement mode is used to indicate the layout mode of the tensor in the storage component; Among them, in each round of iteration operation: Selecting one or more core operators from the at least one core operator as a starting operator; Determine the arrangement mode corresponding to each starting operator according to the read and write overhead of each starting operator under multiple preset arrangement modes; Taking the starting operator as the starting point, combined with the arrangement mode corresponding to the starting operator, propagate forward and backward according to the dependency relationship, and obtain the arrangement mode corresponding to each operator in turn, wherein, in the process of obtaining the arrangement mode corresponding to each operator in turn, the arrangement mode corresponding to the current operator is obtained in combination with the arrangement mode corresponding to the adjacent operator, and the adjacent operator is an operator that is adjacent to the current operator according to the dependency relationship and has a determined arrangement mode.

2. The data processing method according to claim 1, wherein: Based on the computation graph, determining at least one core operator includes: Selecting the at least one core operator from the plurality of operators based on a predetermined condition; The predetermined condition includes at least one of the following: a fusion operator, an operator with a higher computational amount when the plurality of operators are sorted from high to low in terms of computational amount, an operator of a predetermined functional type, The fusion operator is an operator obtained by fusing multiple operators through an optimization strategy.

3. The data processing method according to claim 1, wherein: Selecting one or more core operators from the at least one core operator as a starting operator comprises: Dividing the at least one core operator into at least one priority level according to the importance; In each round of iterative operation, one or more core operators of the same priority are selected as the starting operators; Among them, the at least one round of iterative operation is executed in descending order according to the importance of the at least one priority level, the forward and backward propagation stops when reaching the core operator, and the current round of iterative operation ends when the forward and backward propagation starting from the starting operator both reach the core operator.

4. The data processing method according to claim 1, wherein: According to the read and write overhead of each starting operator under multiple preset arrangements, the arrangement method corresponding to each starting operator is determined, including: For the target tensor whose arrangement has not been determined in the starting operator: When performing the first iteration operation, selecting a preset arrangement mode with the smallest read / write overhead from the read / write overhead of the target tensor under the multiple preset arrangement modes as the arrangement mode of the target tensor, wherein the target tensor includes an input tensor or an output tensor; When executing non-first iteration operations, In response to an undetermined arrangement of the tensor corresponding to the target tensor, a preset arrangement with the smallest read / write overhead is selected from the read / write overhead of the target tensor under the multiple preset arrangement modes as the arrangement mode of the target tensor, wherein in response to the target tensor being the input tensor, the corresponding tensor is loaded as the target tensor, or in response to the target tensor being the output tensor, the target tensor is loaded as the corresponding tensor; In response to the determined arrangement of the tensor corresponding to the target tensor, a preset arrangement with the smallest read / write overhead is selected as the arrangement of the target tensor from the multiple read / write overheads corresponding to the target tensor, wherein the multiple read / write overheads corresponding to the target tensor include the read / write overhead of the target tensor under a first preset arrangement and the read / write overhead of the target tensor when converted to multiple arrangement modes, the first preset arrangement being the arrangement determined to be used by the corresponding tensor, and the multiple arrangement conversions including the conversion of the first preset arrangement into multiple other preset arrangement modes.

5. The data processing method according to claim 4, wherein: From the read and write costs of the target tensor under the multiple preset arrangements, selecting a preset arrangement with the smallest read and write cost as the arrangement of the target tensor includes: Excluding at least one preset arrangement from the plurality of preset arrangements according to the operator characteristics of the starting operator; From the read and write costs of the starting operator under the remaining preset arrangements, select the preset arrangement with the smallest read and write cost as the arrangement of the target tensor.

6. The data processing method according to claim 1, wherein: Taking the starting operator as the starting point, combined with the arrangement mode corresponding to the starting operator, propagating forward and backward according to the dependency relationship, the arrangement modes corresponding to each operator are obtained in turn, including: In combination with the dependency relationship, topologically sorting the computation graph according to a preset topological sorting type; Taking the starting operator as the starting point, performing propagation operations on each operator forward and backward in sequence according to the result of the topological sorting; The propagation operation includes: Determine the current operator, wherein, when propagating backward from the starting operator, the current operator is an operator that uses the output tensor of the adjacent operator as the input tensor of the current operator, and when propagating forward from the starting operator, the current operator is an operator that outputs the output tensor to the adjacent operator as the input tensor of the adjacent operator; In response to the current operator not being the core operator, determining an arrangement mode corresponding to the current operator based on arrangement modes corresponding to the adjacent operators; In response to the current operator being the core operator, stopping the current round of iteration operation.

7. The data processing method according to claim 6, wherein: Determining the arrangement mode corresponding to the current operator based on the arrangement mode corresponding to the adjacent operators includes: In response to the output tensor of the adjacent operator being output to the current operator as the input tensor of the current operator: From the first read-write cost corresponding to the input tensor of the current operator and the second read-write cost corresponding to the input tensor of the current operator, select the preset arrangement mode with the smallest read-write cost as the arrangement mode of the input tensor of the current operator, wherein the first read-write cost is the read-write cost under the first preset arrangement mode, the output tensor of the adjacent operator is determined to use the first preset arrangement mode, and the second read-write cost is the read-write cost when converting the first preset arrangement mode to at least one other preset arrangement mode; From the read and write costs of the output tensor of the current operator under the multiple preset arrangements, select a preset arrangement with the smallest read and write cost as the arrangement of the output tensor of the current operator.

8. The data processing method according to claim 6, wherein: Determining the arrangement mode corresponding to the current operator based on the arrangement mode corresponding to the adjacent operators includes: In response to outputting the output tensor of the current operator to the adjacent operator as the input tensor of the adjacent operator: From the third read-write cost corresponding to the output tensor of the current operator and the fourth read-write cost corresponding to the output tensor of the current operator, select the preset arrangement mode with the smallest read-write cost as the arrangement mode of the output tensor of the current operator, wherein the third read-write cost is the read-write cost under the second preset arrangement mode, the input tensor of the adjacent operator is determined to use the second preset arrangement mode, and the fourth read-write cost is the read-write cost when converting at least one other preset arrangement mode to the second preset arrangement mode; From the read and write costs of the input tensor of the current operator under the multiple preset arrangements, select a preset arrangement with the smallest read and write cost as the arrangement of the input tensor of the current operator.

9. The data processing method according to claim 1, wherein: In response to the number of executions of the iterative operation reaching a preset threshold and there being at least one operator in the computation graph whose arrangement has not yet been determined, determining the arrangement of the at least one operator based on a predetermined rule, The predetermined rule specifies the condition requirements and the arrangement methods corresponding to the condition requirements, and the operators that meet the corresponding condition requirements use the arrangement methods corresponding to the corresponding condition requirements.

10. The data processing method according to any one of claims 1 to 9, wherein: Before determining the arrangement modes respectively corresponding to the multiple operators through at least one round of iterative operation, the data processing method further includes: Determine the shape and size of the tensors associated with the multiple operators; For each operator, according to the shape and size of the tensor associated with the operator, the read and write overhead of the tensor associated with the operator in the plurality of preset arrangements is obtained.

11. The data processing method according to claim 10, wherein: The operator-related tensors include the operator's input tensor and the operator's output tensor. Determining the shape and size of the tensors associated with the multiple operators includes: The shape size of the input tensor and the output tensor of each operator is derived according to the computation graph and the shape size of the input activation tensor input into the computation graph.

12. The data processing method according to claim 10, wherein: According to the shape and size of the tensor associated with the operator, the read and write overhead of the tensor associated with the operator in the plurality of preset arrangements is obtained, including: Determine, according to the shape and size of the tensor associated with the operator, the amount of data that needs to be transmitted for the tensor associated with the operator in each preset arrangement mode; Based on the amount of data to be transmitted and the hardware information, the read and write overhead of the operator-related tensors in each preset arrangement is obtained, wherein the hardware information includes the data transmission bandwidth in each preset arrangement.

13. The data processing method according to claim 10, wherein: Before determining the arrangement modes respectively corresponding to the multiple operators through at least one round of iterative operation, the data processing method further includes: According to the shape and size of the tensor associated with the operator, the read and write overhead of the tensor associated with the operator when an arrangement conversion occurs is obtained, wherein the arrangement conversion includes storing the input tensor of the operator in a first preset arrangement in the storage component, and loading the input tensor from the storage component in a second preset arrangement, and the first preset arrangement is different from the second preset arrangement.

14. The data processing method according to any one of claims 1 to 9, wherein: The arrangement method includes at least one of a memory access method and a data arrangement method. The data arrangement method indicates the storage order and dimensional arrangement of data in the storage component, and the memory access method indicates the memory organization method and the rules for accessing the memory.

15. A data processing device, comprising: An acquisition unit configured to acquire a computation graph, wherein the computation graph includes a plurality of operators and dependencies between the plurality of operators; A core operator determination unit, configured to determine at least one core operator based on the computation graph; an iterative operation unit, configured to determine, through at least one round of iterative operation, the arrangement modes corresponding to the plurality of operators, respectively, wherein the arrangement mode corresponding to each operator includes an arrangement mode of the input tensor of the operator and an arrangement mode of the output tensor of the operator, and the arrangement mode is used to indicate a layout mode of the tensor in the storage component; Among them, in each round of iteration operation: Selecting one or more core operators from the at least one core operator as a starting operator; Determine the arrangement mode corresponding to each starting operator according to the read and write overhead of each starting operator under multiple preset arrangement modes; Taking the starting operator as the starting point, combined with the arrangement mode corresponding to the starting operator, propagate forward and backward according to the dependency relationship, and obtain the arrangement mode corresponding to each operator in turn, wherein, in the process of obtaining the arrangement mode corresponding to each operator in turn, the arrangement mode corresponding to the current operator is obtained in combination with the arrangement mode corresponding to the adjacent operator, and the adjacent operator is an operator that is adjacent to the current operator according to the dependency relationship and has a determined arrangement mode.

16. An electronic device, comprising: A memory non-transitorily stores computer executable instructions; a processor configured to execute the computer executable instructions, Wherein, when the computer executable instructions are executed by the processor, the data processing method according to any one of claims 1-14 is implemented.

17. A non-transitory computer-readable storage medium, wherein: The non-transitory computer-readable storage medium stores computer-executable instructions, When the computer executable instructions are executed by a processor, the data processing method according to any one of claims 1 to 14 is implemented.

Images