Data processing method and device, electronic equipment and computer readable storage medium

Abstract

This application provides a data processing method, apparatus, electronic device, and computer-readable storage medium, relating to the field of data processing technology. The method includes: if a neural network model is determined to include a candidate operator set, the candidate operator set sequentially including a first convolution operator, a reshaping operator, and a transpose operator, and the candidate operator set satisfies preset conditions, then a target operator set corresponding to the candidate operator set is determined; the target operator set sequentially includes a second convolution operator and a depth-to-breadth operator; the last dimension attribute of the output tensor of the first convolution operator represents the block size, and the last dimension attribute of the output tensor of the second convolution operator represents the number of output channels; the candidate operator set in the neural network model is replaced with the target operator set. This application uses the number of output channels represented by the last dimension attribute of the output tensor of the second convolution operator as the basic unit for data transfer, improving the continuity and efficiency of data transfer.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to the field of neural network technology. More specifically, this application relates to a data processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Neural network models are deployed in AI accelerators. The number and type of operators in the neural network model affect the efficiency of data transfer in the AI ​​accelerator. Some neural network models may have convolution (Conv), reshape, and transpose operators. Conv is a computationally intensive operator that requires computation regardless of the operation. Reshape and transpose are memory-intensive operators that only rearrange the data without performing any data computation. Operators that do not perform any data computation are more time-consuming for AI accelerators.

[0003] Furthermore, during the data transfer process, when executing the aforementioned convolution operator, reshaping operator, and transpose operator, the data transfer is performed based on the last dimension attribute of the output tensor of the convolution operator. This last dimension attribute represents the block size, which is usually 2, a relatively small value. The AI ​​accelerator's data transfer based on the block size results in low continuity between data, affecting data transfer efficiency and consequently impacting the inference performance of the neural network model. Summary of the Invention

[0004] This application provides a data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product to solve the technical problems in the background art.

[0005] According to a first aspect of the embodiments of this application, a data processing method is provided, the method comprising:

[0006] The input samples and the neural network model to be deployed are determined. The neural network model includes multiple operators, including convolution operators, reshaping operators and transpose operators.

[0007] Input samples are fed into a neural network model to obtain the input tensors and output tensors of each operator in the neural network model.

[0008] If it is determined that the neural network model includes at least one candidate operator set, the candidate operator set includes a first convolution operator, a reshaping operator and a transpose operator in sequence, and each operator in the candidate operator set satisfies a preset condition, then the target operator set corresponding to the candidate operator set is determined; the target operator set includes a second convolution operator and a depth-to-breadth operator in sequence.

[0009] Among them, the second convolution operator and the first convolution operator have the same input tensor but different output tensors. The last dimension of the output tensor of the first convolution operator represents the block size, and the last dimension of the output tensor of the second convolution operator represents the number of output channels. The depth-to-breadth operator and the transpose operator have the same output tensor.

[0010] Replace the candidate operator set in the neural network model with the target operator set, and then deploy the replaced neural network model on the AI ​​accelerator.

[0011] According to a second aspect of the embodiments of this application, a data processing apparatus is provided, the apparatus comprising:

[0012] The first determining module is used to determine the input samples and the neural network model to be deployed. The neural network model includes multiple operators, and the types of multiple operators include convolution operators, reshaping operators and transpose operators.

[0013] The tensor acquisition module is used to input input samples into the neural network model and obtain the input tensors and output tensors of each operator in the neural network model.

[0014] The target operator set determination module is used to determine the target operator set corresponding to the candidate operator set if the neural network model includes at least one candidate operator set, the candidate operator set includes a first convolution operator, a reshaping operator and a transpose operator in sequence, and each operator in the candidate operator set satisfies a preset condition; the target operator set includes a second convolution operator and a depth-to-breadth operator in sequence.

[0015] Among them, the second convolution operator and the first convolution operator have the same input tensor but different output tensors. The last dimension of the output tensor of the first convolution operator represents the block size, and the last dimension of the output tensor of the second convolution operator represents the number of output channels. The depth-to-breadth operator and the transpose operator have the same output tensor.

[0016] The deployment module is used to replace the candidate operator set in the neural network model with the target operator set, and then deploy the replaced neural network model on the AI ​​accelerator.

[0017] According to a third aspect of the embodiments of this application, an electronic device is provided, the electronic device including a memory, a processor and a computer program stored in the memory, wherein the processor executes the program to implement the steps of the method provided in the first aspect.

[0018] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method provided in the first aspect.

[0019] According to a fifth aspect of the present application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium, wherein when a processor of a computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, causing the computer device to perform steps implementing the method provided in the first aspect.

[0020] The beneficial effects of the technical solutions provided in this application are:

[0021] In this embodiment, the candidate operator set in the neural network model is replaced with a target operator set. The candidate operator set includes a first convolution operator, a reshaping operator, and a transpose operator, while the target operator set includes a second convolution operator and a depth-to-breadth operator. This allows the AI ​​accelerator to perform data transfer based on the target operator set, rather than the candidate operator set, after deploying the neural network model on the AI ​​accelerator. When the AI ​​accelerator performs data transfer based on the candidate operator set, it uses the last dimension attribute of the output tensor of the first convolution operator as the basic unit. The last dimension attribute of the output tensor of the first convolution operator represents the block size, which is typically 2. In this embodiment, the last dimension attribute of the output tensor of the second convolution operator represents the number of output channels, which is typically 16 or 32, much larger than the block size of 2. This allows the AI ​​accelerator to perform data transfer based on the number of output channels, greatly improving the continuity and efficiency of data transfer. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0023] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0024] Figure 2A flowchart illustrating a method for determining whether a set of candidate operators satisfies preset conditions, provided in an embodiment of this application;

[0025] Figure 3 This is a schematic diagram illustrating the conversion of a candidate operator into a target operator, provided in an embodiment of this application.

[0026] Figure 4 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0029] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.”

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0031] First, let's introduce and explain several terms used in this application:

[0032] An operator is a mapping O from one function space to another: X → X. In a generalized sense, operators can be extended to any space, such as the inner product space. Any operation performed on any function can be considered an operator. Operations such as exponentiation, square root, convolution, and transpose are all operators.

[0033] A tensor is a quantity with size and multiple directions. The direction refers to the tensor's order, also known as dimension. A tensor containing only a single number is called a scalar (also called a zero-dimensional tensor or 0D tensor). An array of numbers is called a vector or a one-dimensional tensor (1D tensor), which has only one axis. An array of vectors is called a matrix or a two-dimensional tensor (2D tensor), which has two axes (usually called rows and columns). Combining multiple matrices into a new array results in a 3D tensor, which visually resembles a cube. Similarly, combining multiple 3D tensors into an array creates a 4D tensor, and combining multiple 4D tensors results in a 5D tensor. Combining nD tensors results in an (n+1)D tensor. An operator's tensor includes an input tensor and an output tensor.

[0034] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0035] This application provides a data processing method, such as... Figure 1 As shown, the method includes:

[0036] Step S101: Determine the input samples and the neural network model to be deployed. The neural network model includes multiple operators, and the types of multiple operators include convolution operators, reshaping operators and transpose operators.

[0037] The input samples in this application embodiment are training samples. These training samples can be any image samples, text samples, video samples, audio samples, etc., and this application embodiment does not impose any restrictions on them.

[0038] The neural network model in this embodiment is to be deployed in an AI accelerator. The neural network model can be any type of neural network model, such as a convolutional neural network. The neural network model can include multiple operators, including convolution operators, reshaping operators, and transpose operators. Of course, it can also include other types of operators, such as activation operators and pooling operators.

[0039] Typically, each layer of a neural network model corresponds to at least one operator; for example, a convolutional layer corresponds to the convolution operator, and a pooling layer corresponds to the pooling operator.

[0040] Step S102: Input the input samples into the neural network model to obtain the input tensors and output tensors of each operator in the neural network model.

[0041] The embodiments of this application can output input samples to a neural network model, thereby obtaining the input tensor and output tensor of each operator in the neural network model. For two consecutive operators, the output tensor of the first operator is the input tensor of the second operator.

[0042] In the embodiments of this application, each operator's input tensor and output tensor contain at least one dimension attribute. The number of dimensions of the input tensor and the number of dimensions of the output tensor of each operator are not necessarily the same. For example, the number of dimensions of the input tensor of the reshaping operator is 4, and the number of dimensions of the output tensor is 6. The dimensions of the two are different.

[0043] Step S103: If it is determined that the neural network model includes at least one candidate operator set, the candidate operator set includes a first convolution operator, a reshaping operator and a transpose operator in sequence, and each operator in the candidate operator set satisfies a preset condition, then the target operator set corresponding to the candidate operator set is determined; the target operator set includes a second convolution operator and a depth-to-breadth operator in sequence.

[0044] The second convolution operator and the first convolution operator have the same input tensor but different output tensors. The last dimension of the output tensor of the first convolution operator represents the block size, while the last dimension of the output tensor of the second convolution operator represents the number of output channels. The depth-to-breadth operator and the transpose operator have the same output tensor.

[0045] The neural network model in this application includes at least one set of candidate operators. The set of candidate operators includes, in sequence, a first convolution conv1 operator, a reshape operator, and a transpose operator reshape. The set of candidate operators can be a collection. From the perspective of the computation graph (each node in the computation graph represents an operator, the directed edge pointing to each node represents the input tensor of the operator corresponding to the node, and the directed edge emanating from the node represents the output tensor of the operator corresponding to the node), the structure of the subgraph corresponding to the set of candidate operators is conv1 operator + reshape operator + transpose operator.

[0046] The Conv1 operator + reshape operator + transpose operator are continuous. Among them, the conv1 operator is a computationally intensive operator that requires computation regardless of the operation. The reshape and transpose operators are memory-intensive operators that only rearrange the data. Moving data without any computation is very time-consuming for AI accelerators. The embodiments of this application aim to convert candidate operators that meet preset conditions into a target operator set. The target operator set does not include the reshape operator + transpose operator. The target operator set includes a second convolution operator conv2 and a depth to space operator in sequence. That is, the subgraph structure corresponding to the target operator set is conv2 + depth to space. The depth to space operator has more continuous data, better locality, and higher performance than the original transpose operator.

[0047] In this embodiment of the application, for each candidate operator set, it is necessary to determine whether each candidate operator set meets the preset conditions. For candidate operator sets that meet the preset conditions, the candidate operator set can be converted into the target operator set.

[0048] Specifically, such as Figure 2 As shown, it exemplarily illustrates a flowchart for determining whether a set of candidate operators meets preset conditions, including the following steps:

[0049] Step S201: Determine whether the output tensor of the first convolution operator is the output tensor of the last operator of the neural network model. If not, proceed to step S202; if yes, determine that the candidate operator set does not meet the preset conditions.

[0050] In this context, the output tensor of the last operator in the neural network model is also the output tensor of the overall computation graph of the neural network model. For a neural network model, the overall computation graph of the neural network is determined, and the operators corresponding to the overall computation graph are also determined. The output tensor of the overall computation graph of the neural network model is the output tensor of the last operator in the neural network model. If the output tensor of the convolution operator is the output tensor of the last operator in the neural network model, then the three operators in the candidate operator set cannot be equivalently transformed because the output of the second convolution operator after the transformation is changed, which will lead to a change in the output of the overall computation graph. Therefore, one of the presuppositions is that the output tensor of the convolution operator is not the output tensor of the last operator in the neural network model.

[0051] S202: Determine whether the output tensors of the reshaping operators are all the output tensors of the last operator of the neural network model; if not, proceed to step S203; if yes, determine that the candidate operator set does not meet the preset conditions.

[0052] As explained above, if the output tensor of the reshaping operator is the output tensor of the last operator in the neural network model, then the three operators in the candidate operator set cannot be equivalently transformed because the target operator set does not include the reshaping operator. If the output tensor of the reshaping operator is the output tensor of the last operator in the neural network model, this will lead to a change in the output tensor of the overall computation graph. Therefore, one of the presuppositions is that the output tensor of the reshaping operator is not the output tensor of the last operator in the neural network model, that is, the reshaping operator is not the last operator in the neural network model.

[0053] S203: Determine whether the first dimension attribute of the input tensor of the reshaping operator is the same as the first dimension attribute of the output tensor of the reshaping operator; if not, determine that the candidate operator set does not meet the preset conditions; if so, proceed to step S204.

[0054] Step S204: Determine whether the second dimension attribute in the input tensor of the reshaping operator is the same as the second dimension attribute in the output tensor of the reshaping operator; if not, determine that the candidate operator set does not meet the preset conditions; if yes, proceed to step S205.

[0055] Step S205 determines whether the third dimension attribute in the input tensor of the reshaping operator is the same as the third dimension attribute in the output tensor of the reshaping operator; if not, it is determined that the candidate operator set does not meet the preset conditions; if so, step S206 is executed.

[0056] The input tensor of the reshaping operator in this embodiment has four dimensions. The first dimension of the input tensor of the reshaping operator can be recorded using reshape.in.shape[0], the second dimension of the input tensor of the reshaping operator can be recorded using reshape.in.shape[1], the third dimension of the input tensor of the reshaping operator can be recorded using reshape.in.shape[2], and the fourth dimension of the input tensor of the reshaping operator can be recorded using reshape.in.shape[3].

[0057] Wherein, reshape.in.shape[0] = N1, N1 represents the batch size, reshape.in.shape[1] = H1, H1 represents the image height, reshape.in.shape[2] = W1, W represents the image width, reshape.in.shape[3] = C1, C1 represents the number of input channels, and the dimension shape of the input tensor of this reshaping operator is [N1, H1, W1, C1].

[0058] The output tensor of the reshaping operator in this embodiment has four dimensions. The first dimension of the output tensor of the reshaping operator can be recorded using reshape.out.shape[0]. The second dimension of the input tensor of the output operator can be recorded using reshape.out.shape[1]. The third dimension of the output tensor of the reshaping operator can be recorded using reshape.out.shape[2]. The fourth dimension of the output tensor of the reshaping operator can be recorded using reshape.out.shape[3]. The fifth dimension of the output tensor of the reshaping operator can be recorded using reshape.out.shape[4]. The sixth dimension of the input tensor of the output operator can be recorded using reshape.out.shape[5].

[0059] Wherein, reshape.out.shape[0] = N2, N2 represents the batch size, reshape.out.shape[1] = H2, H2 represents the image height, reshape.out.shape[2] = W2, W2 represents the image width, reshape.out.shape[3] = C2, C2 represents the number of output channels, reshape.out.shape[4] = B, B represents the block size, reshape.out.shape[5] = B, B represents the block size, and the dimension shape of the input tensor of the reshaping operator is [N2, H2, W2, C2, B, B].

[0060] For the Reshape operator, it is necessary to determine whether the N, H, and W dimensions of the Reshape operator remain unchanged, that is, to determine whether reshape.out.shape[0] = reshape.in.shape[0] (i.e., whether N1 = N2), reshape.out.shape[1] = reshape.in.shape[1] (i.e., whether H1 = H2), and reshape.out.shape[2] = reshape.in.shape[2] (i.e., whether W1 = W2). If yes, then step S206 is executed; if no, then it is determined that the candidate operator set does not meet the preset conditions. Therefore, one of the preset conditions is that the first dimension attribute in the input tensor of the reshape operator is the same as the first dimension attribute in the output tensor of the reshape operator; another preset condition is that the second dimension attribute in the input tensor of the reshape operator is the same as the second dimension attribute in the output tensor of the reshape operator; and yet another preset condition is that the third dimension attribute in the input tensor of the reshape operator is the same as the third dimension attribute in the output tensor of the reshape operator.

[0061] S206: Determine whether the fifth and sixth dimension attributes of the output tensor of the reshaping operator are the same, and whether the fourth dimension attribute in the input tensor of the reshaping operator is equal to the product of the fourth, fifth, and sixth dimension attributes of the output tensor of the reshaping operator. If yes, proceed to step S15; otherwise, end the execution.

[0062] Continuing from the previous example, for the Reshape operator, it is also necessary to determine whether the Reshape operator changes the C1 dimension to the (C2, B, B) pattern, that is, to determine whether the fifth and sixth dimension attributes of the output tensor of the reshape operator are the same: reshape.out.shape[4] = reshape.out.shape[5]. In addition, it is also necessary to determine whether the fourth dimension attribute in the input tensor of the reshape operator is equal to the product of the fourth, fifth, and sixth dimension attributes of the output tensor of the reshape operator: reshape.in.shape[3] = reshape.out.shape[3] * reshape.out.shape[4] * reshape.out.shape[5]. If so, then step S207 is executed.

[0063] In general, the reshape operator needs to satisfy the pattern reshape(N1, H1, W1, C1)->(N2, H2, W2, C2, B, B), that is, it needs to satisfy N1=N2, H1=H2, W1=W2, C1=C2*B*B. Specifically, suppose a certain reshape operator 1 satisfies (1, 20, 20, 128)---->(1, 20, 20, 8, 4, 4). Obviously, reshape operator 1 satisfies reshape.out.shape[0]=reshape.in.shape[0], reshape.out.shape[1]=reshape.in.shape[1], reshape.out.shape[2]=reshape.in.shape[2], and satisfies reshape.out.shape[4]=reshape.out.shape[5] and reshape. in.shape[3] = reshape.out.shape[3] * reshape.out.shape[4] * reshape.out.shape[5], that is, this reshape operator 1 satisfies the reshape(N1, H1, W1, C1) -> (N2, H2, W2, C2, B, B) pattern required by our equivalent transformation. A certain reshape operator 2 satisfies reshape(1, 20, 20, 128) -> (1, 20, 20, 16, 2, 4). The fifth dimension attribute of the output tensor of this reshape operator 2 is not equal to the sixth dimension attribute of the output tensor. A certain reshape operator 3 satisfies reshape(1, 20, 20, 128) -> (1, 20, 20, 16, 8). The output tensor of this reshape operator 3 does not have a sixth dimension attribute. This reshape operator 2 and reshape operator 3 do not meet the preset conditions. One of the preset conditions is that the fourth dimension attribute of the input tensor of the reshaping operator is equal to the product of the fourth, fifth, and sixth dimension attributes of the output tensor of the reshaping operator.

[0064] S207: Based on the order of the attributes of each dimension in the input and output tensors of the transpose operator, determine the rearrangement parameters of the transpose operator, and determine whether the rearrangement parameters meet the preset conditions. If yes, determine that the candidate operator set meets the preset fusion conditions; otherwise, determine that the candidate operator set does not meet the preset conditions.

[0065] In this embodiment, the transpose operator needs to determine the rearrangement parameter (perm) of the transpose operator based on the order of the attributes of each dimension in the input tensor and the output tensor. The transpose operator modifies the order of the attributes of each dimension in the input tensor based on the rearrangement parameter to obtain the output tensor of the transpose operator.

[0066] Specifically, the output tensor of the reshaping operator is the input tensor of the transpose operator, both being (N2, H2, W2, C2, B, B). The sorting parameter of each dimension attribute in the input tensor (N2, H2, W2, C2, B, B) of the transpose operator is (0, 1, 2, 3, 4, 5). It is necessary to determine whether the rearrangement parameter perm of the transpose operator is equal to (0, 1, 4, 2, 5, 3), that is, to determine whether the transpose operator rearranges (N2, H2, W2, C2, B, B) into (N2, H2, B, W2, B, C2). If yes, it is determined that the candidate operator set meets the preset conditions, and the target operator set corresponding to the candidate operator set can be generated. If no, it is determined that the subsequent operator set does not meet the preset conditions.

[0067] In this embodiment, the candidate operator set and the target operator set are equivalent, meaning the input tensors of the candidate operator set and the target operator set are the same, and the output tensors of the candidate operator set and the target operator set are the same. Specifically, the first operator in the candidate operator set is the first convolution operator, and the first operator in the target operator set is the second convolution operator; that is, the input tensors of the first and second convolution operators are the same. The last operator in the candidate operator set is the transpose operator, and the last operator in the target operator set is the depth-to-breadth operator; that is, the output tensor of the transpose operator is the same as the output tensor of the depth-to-breadth operator.

[0068] Furthermore, since the output tensor of the first convolution operator is (N2, H2, W2, C2, B, B), during subsequent data transfer, the AI ​​accelerator uses the last dimension attribute B as the basic unit for data transfer. Typically, B is 2. This results in low continuity between the data transferred by the AI ​​accelerator each time, leading to low data transfer efficiency. To avoid this situation, in this embodiment, the last dimension attribute of the output tensor of the second convolution operator is not B, but C2, that is, data transfer is performed using the number of output channels as the basic unit. C2 is generally 16 or 32. The output tensor of the second convolution operator is (N2, H2, W2, B, B, C2).

[0069] It is worth noting that the output tensor of the first convolution operator is (N1, H1, W1, C1), and the matrix shape of the output tensor of the first convolution operator is [N1, H1, W1, C1]. The matrix shape of the output tensor of the second convolution operator (N2, H2, W2, B, B, C2) is different from that of the output tensor of the first convolution operator [N1, H1, W1, C1]. Therefore, the matrix shape of the output tensor of the second convolution operator (N2, H2, W2, B, B, C2) needs to be modified. The modified matrix shape of the output tensor of the second convolution operator is [N3, H3, W3, C3]. [N3, H3, W3, C3] and [N1, H1, W1, C1] have the same matrix shape.

[0070] In addition to generating a second convolution operator, this embodiment of the application also generates a depth-to-space operator. The depth-to-space operator has more continuous data, better locality, and higher performance than the original transpose operator. In order to ensure that the output tensor of the target operator set is the same as the output tensor of the candidate operator set, the output tensor of the depth-to-space operator is configured to be the same as the output tensor of the transpose operator.

[0071] like Figure 3 As shown, it exemplarily illustrates a schematic diagram of converting a candidate operator into a target operator according to an embodiment of this application. Figure 3 The left side shows the structure of the candidate operator set, which includes the first convolution operator conv1, the reshape operator reshape, and the transpose operator transpose. The output tensor of conv1 is (N1, H1, W1, C1), and the matrix shape of the output tensor of conv1 is [N1, H1, W1, C1]. The output tensor of reshape is (N2, H2, W2, C2, B, B), and the matrix shape of the output tensor of reshape is [N2, H2, W2, C2, B, B]. The output tensor of transpose is (N2, H2, B, W2, B, C2), and the matrix shape of the output tensor of transpose is [N2, H2, B, W2, B, C2].

[0072] Should Figure 3The right side shows the structure of the target operator set, which includes the second convolution operator conv2 and the depth to space operator. The output tensor of conv2 is the same as the input tensor of conv1 (not shown in the figure). The output tensor of conv2 is (N2, H2, W2, B, B, C2), which is different from the output tensor of conv1. The matrix shape of the output tensor of conv2 is [N3, H3, W3, C3], which is the same as the matrix shape of the output tensor of conv1. The output tensor of depth to space is (N2, H2, B, W2, B, C2), which is the same as the output tensor of transpose. The matrix shape of the output tensor of depth to space is [N2, H2, B, W2, B, C2], which is the same as the matrix shape of the output tensor of transpose.

[0073] In this embodiment, the data transfer process corresponding to the candidate operator set is (N2, H2, W2, C2, B, B) -> (N2, H2, B, W2, B, C2). By rearranging the weights of the first convolution operator in advance to obtain the second convolution operator, the data transfer process becomes (N2, H2, W2, B, B, C2) -> (N2, H2, B, W2, B, C2). The transfer before the transformation uses B as the basic unit, while the transfer after the transformation uses C2 as the basic unit. When C2 is greater than B, the data transfer efficiency can be improved.

[0074] Step S208: Replace the candidate operator set in the neural network model with the target operator set, and deploy the replaced neural network model on the AI ​​accelerator.

[0075] In this embodiment of the application, after obtaining the target operator set, the candidate operator set in the neural network model is replaced with the target operator set, that is, the nodes corresponding to each operator in the candidate operator set in the overall computation graph are modified to the nodes corresponding to each operator in the target operator set.

[0076] In this embodiment of the application, after replacing the candidate operator set in the neural network model with the target operator set, the replaced neural network model is deployed on an AI accelerator, so that when the AI ​​accelerator performs data transfer, it uses the last dimension attribute of the output tensor of the second convolution operator in the target operator set as the basic unit for data transfer.

[0077] In this embodiment, the candidate operator set in the neural network model is replaced with a target operator set. The candidate operator set includes a first convolution operator, a reshaping operator, and a transpose operator, while the target operator set includes a second convolution operator and a depth-to-breadth operator. This allows the AI ​​accelerator to perform data transfer based on the target operator set, rather than the candidate operator set, after deploying the neural network model on the AI ​​accelerator. When the AI ​​accelerator performs data transfer based on the candidate operator set, it uses the last dimension attribute of the output tensor of the first convolution operator as the basic unit. The last dimension attribute of the output tensor of the first convolution operator represents the block size, which is typically 2. In this embodiment, the last dimension attribute of the output tensor of the second convolution operator represents the number of output channels, which is typically 16 or 32, much larger than the block size corresponding to 2. By using the number of output channels as the basic unit for data transfer, the AI ​​accelerator significantly improves the continuity and efficiency of data transfer.

[0078] This application provides a possible implementation method in which the second convolution operator in the target operator set is determined in the following way:

[0079] Obtain the first weight matrix, the first bias term, and the output tensor of the remodeling operator for the first convolution operator. The output tensor of the remodeling operator includes six dimensions, where the fourth dimension of the output tensor represents the number of output channels, and the fifth dimension represents the block size.

[0080] The input tensor of the first convolution operator is used as the input tensor of the second convolution operator. The attributes of each dimension in the output tensor of the reshaping operator are rearranged to obtain the rearranged output tensor. The rearranged output tensor is used as the output tensor of the second convolution operator.

[0081] A second weight matrix and a second bias term are generated based on the first weight matrix of the first convolution operator and the fourth and fifth dimension attributes of the output tensor of the reshaping operator.

[0082] The second weight matrix is ​​used as the weight matrix of the second convolution operator, and the second bias term is used as the weight matrix of the second convolution operator.

[0083] In addition to the input tensor and output tensor, the parameters of the convolution operator also include a weight matrix (also called the convolution kernel) and a bias term (also called a bias unit). In this embodiment, the input tensor, output tensor, weight matrix, and bias term of the second convolution operator need to be configured. Specifically, in order to ensure that the input tensor of the target operator set is the same as the input tensor of the candidate operator set, this embodiment directly uses the input tensor of the first convolution operator as the input tensor of the second convolution operator.

[0084] In addition, in order to improve the data continuity between transported data and thus improve data transport efficiency, this application embodiment does not directly use the output tensor of the reshaping operator as the output tensor of the second convolution operator. Instead, it rearranges the attributes of each dimension of the output tensor of the reshaping operator to obtain the rearranged output tensor. The last dimension attribute of the rearranged output tensor is the number of output channels C2. The rearranged output tensor is used as the output tensor of the second convolution operator.

[0085] Specifically, the output tensor of the reshaping operator is (N2, H2, W2, C2, B, B). The fourth and fifth dimensions of the output tensor can be swapped to achieve rearrangement; or the fourth and sixth dimensions can be swapped to achieve rearrangement. The rearranged output tensor is then used as the output tensor of the second convolution operator, which is (N, H, W, B, B, C2), with its last dimension being the number of output channels, C2.

[0086] In this embodiment, the first convolution operator is replaced by a second convolution operator. The output tensor of the second convolution operator is different from that of the first convolution operator. The weight matrix and bias terms of the second convolution operator are also different from those of the first convolution operator. In this embodiment, the weight matrix of the first convolution operator is a first weight matrix, the bias terms of the first convolution operator are first bias terms, the weight matrix of the second convolution operator is a second weight matrix, and the bias terms of the second convolution operator are second bias terms.

[0087] The output tensor of the reshaping operator in this embodiment includes six dimensions. The fourth dimension of the output tensor of the reshaping operator represents the number of output channels, and the fifth dimension of the output tensor of the reshaping operator represents the block size. The second weight matrix and the second bias term are generated based on the first weight matrix of the first convolution operator, the fourth dimension of the output tensor of the reshaping operator, and the second weight matrix and the second bias term are used as the weight matrix of the second convolution operator.

[0088] Specifically, the first weight matrix of the first convolution operator in this application embodiment includes four dimensions: the first dimension of the first weight matrix is ​​the number of input channels, the second dimension of the first weight matrix represents the image height, the third dimension of the first weight matrix represents the image width, and the fourth dimension of the first weight matrix represents the number of input channels.

[0089] A second weight matrix and a second bias term are generated based on the first weight matrix of the first convolution operator and the fourth and fifth dimension attributes of the output tensor of the reshaping operator, including:

[0090] Configure the attributes of each dimension of the third weight matrix, wherein the first dimension attribute of the third weight matrix is ​​the number of input channels represented by the fourth dimension attribute of the output tensor of the reshaping operator; the second and third dimensions attributes of the third weight matrix are both the block size represented by the fifth dimension attribute of the output tensor of the reshaping operator; the fourth dimension attribute of the third weight matrix is ​​the product of the second, third, and fourth dimensions attributes of the first weight matrix.

[0091] The attributes of each dimension of the third weight matrix are rearranged based on the first preset rearrangement parameters to obtain the rearranged third weight matrix; the matrix shape of the first weight matrix is ​​obtained, and the matrix shape of the rearranged third weight matrix is ​​modified based on the matrix shape of the first weight matrix to obtain the second weight matrix;

[0092] Configure the attributes of each dimension of the third bias term, wherein the first dimension attribute of the third bias term is the number of input channels represented by the fourth dimension attribute of the output tensor of the reshaping operator; the second and third dimension attributes of the third bias term are both the block size represented by the fifth dimension attribute of the output tensor of the reshaping operator.

[0093] The attributes of each dimension of the third bias term are rearranged based on the second preset rearrangement parameters to obtain the rearranged third bias term; the matrix shape of the first bias term is obtained, and the matrix shape of the rearranged third bias term is modified based on the matrix shape of the first bias term to obtain the second bias term.

[0094] In this embodiment, the first weight matrix weight1 and the first bias term bias2 of the first convolution operator need to be obtained. The first dimension attribute of weight1 is the number of output channels C1, the second dimension attribute is the image height H1, the third dimension attribute is the image height W1, and the fourth dimension attribute is the number of input channels C. in The shape of the first weight matrix, conv.weight.shape, is [C1,H1,W1,C]. inThe first bias term has only one dimension attribute, which is the number of output channels C1, and the matrix shape of the first bias term is conv.bias.shape [C1]. In the embodiments of this application, the fourth dimension attribute of the output tensor of the reshaping operator represents the number of output channels, that is, reshape.out.shape[3] = C2; the fifth dimension attribute of the output tensor of the reshaping operator represents the block size, that is, reshape.out.shape[4] = B.

[0095] The third weight matrix, weight3, is generated based on the first weight matrix and the fourth and fifth dimension attributes of the output tensor of the reshaping operator. This is achieved by: weight3 = conv.weight.reshape(C2, B, B, H1 * W1 * C). in That is, the first dimension attribute of the third weight matrix is ​​the number of input channels C2 represented by the fourth dimension attribute of the output tensor of the reshaping operator; the second and third dimension attributes of the third weight matrix are both the block size B represented by the fifth dimension attribute of the output tensor of the reshaping operator; the fourth dimension attribute of the third weight matrix is ​​the product of the second, third, and fourth dimension attributes of the first weight matrix (H1*W1*C). in ).

[0096] The third weight matrix is ​​generated to generate the second weight matrix of the second convolution operator. To ensure consistency between the candidate operator set and the target operator set, this embodiment further permutes the attributes of each dimension of the third weight matrix based on a first preset permute parameter (1,2,0,3) to obtain the permute third weight matrix. The first preset permute parameter is (1,2,0,3). The matrix shape of the first weight matrix is ​​obtained, and the matrix shape of the permute third weight matrix is ​​modified based on the matrix shape of the first weight matrix to obtain the second weight matrix weight2. This second weight matrix has the same matrix shape as the first weight matrix, i.e.:

[0097] weight2=conv.weight.reshape(C2,B,B,H1*W1*C in ).permute(1,2,0,3).res

[0098] hape([C1,H1,W1,C in The `permute` function rearranges the attributes of each dimension of the third weight matrix, while `reshape` modifies the shape of the rearranged third weight matrix.

[0099] The rearrangement of the weight matrix in this embodiment occurs before the neural network model is deployed to the AI ​​accelerator, which can reduce the time consumed during the operation of the AI ​​accelerator.

[0100] The third bias term, bias3 = conv.bias.reshape(C2,B,B), is generated based on the first bias term, the fourth dimension attribute of the output tensor of the reshaping operator, and the fifth dimension attribute. That is, the first dimension attribute of the third bias term is the number of input channels C2 represented by the fourth dimension attribute of the output tensor of the reshaping operator; the second and third dimension attributes of the third bias term are both the block size B represented by the fifth dimension attribute of the output tensor of the reshaping operator.

[0101] In this embodiment, after obtaining the third bias term bias3, the attributes of each dimension of the third bias term are rearranged based on a second preset rearrangement parameter (1,2,0) to obtain the rearranged third bias term. The matrix shape of the first bias term is then obtained, and the matrix shape of the rearranged third bias term is modified based on the matrix shape of the first bias term to obtain the second bias term bias2. The second bias term bias2 and the first bias term have the same matrix shape, i.e.:

[0102] bias 2 = conv.bias.reshape(C2,B,B).permute(1,2,0).reshape([C1]). Here, permute is used to rearrange the attributes of each dimension of the third bias term, and reshape is used to modify the matrix shape of the rearranged third bias term.

[0103] This application provides a possible implementation method in which the depth-to-breadth operators in the target operator set are generated in the following way:

[0104] Create a depth-to-breadth operator and configure its input tensor; the zeroth input tensor of the depth-to-breadth operator is the output tensor of the second convolution operator, and the first input tensor of the depth-to-breadth operator is the block size represented by the fourth dimension attribute of the output tensor of the reshaping operator;

[0105] Configure the output tensor of the depth-to-breadth operator; the output tensor of the depth-to-breadth operator is the output tensor of the transpose operator.

[0106] In this embodiment, a depth-to-breadth operator is also required to be created and the input tensor of the depth-to-breadth operator is configured. The zeroth input tensor of the depth-to-breadth operator is the output tensor of the second convolution operator, i.e., DepthToSpace.Input[0] = conv2.output = (N2, H2, W2, B, B, C2). The first input tensor of the depth-to-breadth operator is the block size represented by the fourth dimension attribute of the output tensor of the reshape operator, i.e., DepthToSpace.Input[0] = B, B = reshape.out.shape[4].

[0107] Configure the output tensor of the DepthToSpace operator. The output tensor of the depth-to-breadth operator is the output tensor of the transpose operator, i.e., DepthToSpace.output = transpose.output.

[0108] This application provides a possible implementation method, with the following preset conditions: the output tensor of the convolution operator and the output tensor of the reshaping operator are not the output tensor of the last operator of the neural network model;

[0109] The first dimension attribute of the input tensor of the reshaping operator is the same as the first dimension attribute of the output tensor of the reshaping operator; both the first dimension attribute of the input tensor of the reshaping operator and the first dimension attribute of the output tensor of the reshaping operator characterize the batch size;

[0110] The second-dimensional attribute of the input tensor of the reshaping operator is the same as the second-dimensional attribute of the output tensor of the reshaping operator; both the second-dimensional attribute of the input tensor of the reshaping operator and the second-dimensional attribute of the output tensor of the reshaping operator characterize the image height.

[0111] The third dimension attribute of the input tensor of the reshaping operator is the same as the third dimension attribute of the output tensor of the reshaping operator; both the third dimension attribute of the input tensor of the reshaping operator and the third dimension attribute of the output tensor of the reshaping operator characterize the image width;

[0112] The fifth and sixth dimensions of the output tensor of the reshaping operator are the same; both the fifth and sixth dimensions of the output tensor of the reshaping operator characterize the block size.

[0113] The fourth-dimensional attribute in the input tensor of the reshaping operator is equal to the product of the fourth, fifth, and sixth-dimensional attributes of the output tensor of the reshaping operator; the fourth-dimensional attribute in the input tensor of the reshaping operator represents the number of input channels; the fourth-dimensional attribute in the output tensor of the reshaping operator represents the number of output channels.

[0114] The rearrangement parameters of the transpose operator meet the preset conditions, and the rearrangement parameters are determined according to the order of the attributes of each dimension in the input tensor and output tensor of the transpose operator.

[0115] The foregoing embodiments have already described this, and the embodiments of this application will not be described in detail here.

[0116] This application provides a possible implementation method in which the input sample is at least one of image samples, text samples, audio-visual samples, and video samples.

[0117] The input samples in this application embodiment are at least one of image samples, text samples, and audio / video samples. Image samples can be pictures or video frames, text samples can be text files, audio samples can be voice or recordings, and video samples can be continuous video frames. This application embodiment does not limit these.

[0118] This application provides a data processing apparatus, such as... Figure 4 As shown, the data processing device 40 may include:

[0119] The first determining module 410 is used to determine the input samples and the neural network model to be deployed. The neural network model includes multiple operators, and the types of multiple operators include convolution operators, reshaping operators and transpose operators.

[0120] Tensor acquisition module 420 is used to input input samples into the neural network model and obtain the input tensors and output tensors of each operator in the neural network model;

[0121] The target operator set determination module 430 is used to determine the target operator set corresponding to the candidate operator set if the neural network model is determined to include at least one candidate operator set, the candidate operator set includes a first convolution operator, a reshaping operator and a transpose operator in sequence, and each operator in the candidate operator set satisfies a preset condition; the target operator set includes a second convolution operator and a depth-to-breadth operator in sequence.

[0122] Among them, the second convolution operator and the first convolution operator have the same input tensor but different output tensors. The last dimension of the output tensor of the first convolution operator represents the block size, and the last dimension of the output tensor of the second convolution operator represents the number of output channels. The depth-to-breadth operator and the transpose operator have the same output tensor.

[0123] Deployment module 440 is used to replace the candidate operator set in the neural network model with the target operator set and deploy the replaced neural network model on the AI ​​accelerator.

[0124] This application provides a possible implementation, wherein the target operator set determination module includes a first convolution operator determination submodule, which includes:

[0125] The acquisition unit is used to acquire the first weight matrix, the first bias term, and the output tensor of the reshaping operator of the first convolution operator. The output tensor of the reshaping operator includes six dimensions, where the fourth dimension represents the number of output channels, and the fifth dimension represents the block size.

[0126] The rearrangement unit is used to take the input tensor of the first convolution operator as the input tensor of the second convolution operator, rearrange the attributes of each dimension in the output tensor of the reshaping operator to obtain the rearranged output tensor, and use the rearranged output tensor as the output tensor of the second convolution operator.

[0127] The generation unit is used to generate a second weight matrix and a second bias term based on the first weight matrix of the first convolution operator and the fourth and fifth dimension attributes of the output tensor of the reshaping operator;

[0128] The obtaining unit is used to use the second weight matrix as the weight matrix of the second convolution operator and the second bias term as the weight matrix of the second convolution operator.

[0129] This application provides a possible implementation, in which the rearrangement unit is specifically used to: swap the positions of the fourth-dimensional attribute and the fifth-dimensional attribute in the output tensor of the reshaping operator; or swap the positions of the fourth-dimensional attribute and the sixth-dimensional attribute in the output tensor of the reshaping operator;

[0130] The fourth dimension of the output tensor of the reshaping operator represents the number of channels, while the fifth and sixth dimensions both represent the block size.

[0131] This application provides a possible implementation method in which the first weight matrix of the first convolution operator includes four-dimensional attributes, wherein the second-dimensional attribute of the first weight matrix represents the image height, the third-dimensional attribute of the first weight matrix represents the image width, and the fourth-dimensional attribute of the first weight matrix represents the number of input channels.

[0132] The generation unit is specifically used for: configuring the attributes of each dimension of the third weight matrix, wherein the first dimension attribute of the third weight matrix is ​​the number of input channels represented by the fourth dimension attribute of the output tensor of the reshaping operator; the second and third dimension attributes of the third weight matrix are both the block size represented by the fifth dimension attribute of the output tensor of the reshaping operator; the fourth dimension attribute of the third weight matrix is ​​the product of the second, third, and fourth dimension attributes of the first weight matrix; rearranging the attributes of each dimension of the third weight matrix based on the first preset rearrangement parameters to obtain the rearranged third weight matrix; obtaining the matrix shape of the first weight matrix, and based on the first weight matrix... The shape of the rearranged third weight matrix is ​​modified to obtain the second weight matrix; the attributes of each dimension of the third bias term are configured, wherein the first dimension of the third bias term is the number of input channels represented by the fourth dimension of the output tensor of the reshaping operator; the second and third dimension attributes of the third bias term are the block size represented by the fifth dimension of the output tensor of the reshaping operator; the attributes of each dimension of the third bias term are rearranged based on the second preset rearrangement parameters to obtain the rearranged third bias term; the matrix shape of the first bias term is obtained, and the matrix shape of the rearranged third bias term is modified based on the matrix shape of the first bias term to obtain the second bias term.

[0133] This application provides a possible implementation, in which the target operator set determination module further includes:

[0134] The depth-to-breadth operator creation submodule is used to create a depth-to-breadth operator and configure the input tensor of the depth-to-breadth operator; the zeroth input tensor of the depth-to-breadth operator is the output tensor of the second convolution operator, and the first input tensor of the depth-to-breadth operator is the block size represented by the fourth dimension attribute of the output tensor of the reshaping operator;

[0135] The configuration submodule is used to configure the output tensor of the depth-to-breadth operator; the output tensor of the depth-to-breadth operator is the output tensor of the transpose operator.

[0136] This application provides a possible implementation method, with the following preset conditions: the output tensor of the convolution operator and the output tensor of the reshaping operator are not the output tensor of the last operator of the neural network model;

[0137] The first dimension attribute of the input tensor of the reshaping operator is the same as the first dimension attribute of the output tensor of the reshaping operator; both the first dimension attribute of the input tensor of the reshaping operator and the first dimension attribute of the output tensor of the reshaping operator characterize the batch size;

[0138] The second-dimensional attribute of the input tensor of the reshaping operator is the same as the second-dimensional attribute of the output tensor of the reshaping operator; both the second-dimensional attribute of the input tensor of the reshaping operator and the second-dimensional attribute of the output tensor of the reshaping operator characterize the image height.

[0139] The third dimension attribute of the input tensor of the reshaping operator is the same as the third dimension attribute of the output tensor of the reshaping operator; both the third dimension attribute of the input tensor of the reshaping operator and the third dimension attribute of the output tensor of the reshaping operator characterize the image width;

[0140] The fifth and sixth dimensions of the output tensor of the reshaping operator are the same; both the fifth and sixth dimensions of the output tensor of the reshaping operator characterize the block size.

[0141] The fourth-dimensional attribute in the input tensor of the reshaping operator is equal to the product of the fourth, fifth, and sixth-dimensional attributes of the output tensor of the reshaping operator; the fourth-dimensional attribute in the input tensor of the reshaping operator represents the number of input channels; the fourth-dimensional attribute in the output tensor of the reshaping operator represents the number of output channels.

[0142] The rearrangement parameters of the transpose operator meet the preset conditions, and the rearrangement parameters are determined according to the order of the attributes of each dimension in the input tensor and output tensor of the transpose operator.

[0143] This application provides a possible implementation method in which the input sample is at least one of image samples, text samples, audio-visual samples, and video samples.

[0144] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0145] This application provides an electronic device including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of a data processing method. Compared with related technologies, this application replaces the candidate operator set in the neural network model with a target operator set. The candidate operator set sequentially includes a first convolution operator, a reshaping operator, and a transpose operator. The target operator set sequentially includes a second convolution operator and a depth-to-breadth operator. This allows the AI ​​accelerator to perform data transfer based on the target operator set, rather than the candidate operator set, after deploying the neural network model on an AI accelerator. When the AI ​​accelerator performs data transfer based on the candidate operator set, it uses the last dimension attribute of the output tensor of the first convolution operator as the basic unit for data transfer. The last dimension attribute of the output tensor of the first convolution operator represents the block size, which is usually 2. In this embodiment, the last dimension attribute of the output tensor of the second convolution operator is used as the basic unit for data transfer. The last dimension attribute of the output tensor of the second convolution operator represents the number of output channels, which is usually 16 or 32, much larger than the block size corresponding to 2. This allows the AI ​​accelerator to perform data transfer using the number of output channels as the basic unit, which greatly improves the continuity of data transfer and increases the efficiency of data transfer.

[0146] In one alternative embodiment, an electronic device is provided, such as Figure 5 As shown, Figure 5 The illustrated electronic device 5000 includes a processor 5001 and a memory 5003. The processor 5001 and the memory 5003 are connected, for example, via a bus 5002. Optionally, the electronic device 5000 may further include a transceiver 5004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 5004 is not limited to one type, and the structure of the electronic device 5000 does not constitute a limitation on the embodiments of this application.

[0147] Processor 5001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 5001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0148] Bus 5002 may include a path for transmitting information between the aforementioned components. Bus 5002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 5002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0149] The memory 5003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0150] The memory 5003 is used to store computer programs that execute the embodiments of this application, and its execution is controlled by the processor 5001. The processor 5001 is used to execute the computer programs stored in the memory 5003 to implement the steps shown in the foregoing method embodiments.

[0151] The electronic device package may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0152] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding content of the aforementioned method embodiments. Compared with existing technologies, this application's embodiments achieve the following: The candidate operator set in the neural network model is replaced with a target operator set. The candidate operator set sequentially includes a first convolution operator, a reshaping operator, and a transpose operator. The target operator set sequentially includes a second convolution operator and a depth-to-breadth operator. This allows the AI ​​accelerator to perform data transfer based on the target operator set, rather than the candidate operator set, after deploying the neural network model on the AI ​​accelerator. When the AI ​​accelerator performs data transfer based on the candidate operator set, it uses the last dimension attribute of the output tensor of the first convolution operator as the basic unit. This last dimension attribute of the output tensor of the first convolution operator represents the block size, which is typically 2. In this application's embodiments, the last dimension attribute of the output tensor of the second convolution operator represents the number of output channels, which is typically 16 or 32, much larger than the block size corresponding to 2. This allows the AI ​​accelerator to perform data transfer based on the number of output channels, greatly improving the continuity and efficiency of data transfer.

[0153] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium, a computer-readable medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0154] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments. Compared with existing technologies, this application's embodiments achieve the following: The candidate operator set in the neural network model is replaced with a target operator set. The candidate operator set sequentially includes a first convolution operator, a reshaping operator, and a transpose operator. The target operator set sequentially includes a second convolution operator and a depth-to-breadth operator. This allows the AI ​​accelerator to perform data transfer based on the target operator set, rather than the candidate operator set, after deploying the neural network model on the AI ​​accelerator. When the AI ​​accelerator performs data transfer based on the candidate operator set, it uses the last dimension attribute of the output tensor of the first convolution operator as the basic unit. This last dimension attribute of the output tensor of the first convolution operator represents the block size, which is typically 2. In this application's embodiments, the last dimension attribute of the output tensor of the second convolution operator represents the number of output channels, which is typically 16 or 32, much larger than the block size corresponding to 2. This allows the AI ​​accelerator to perform data transfer based on the number of output channels, greatly improving the continuity and efficiency of data transfer.

[0155] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0156] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0157] The above are only optional implementation methods for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application, without departing from the technical concept of this application, also fall within the protection scope of the embodiments of this application.

Claims

1. A data processing method, characterized by, The method comprises the following steps: determining an input sample and a neural network model to be deployed, wherein the neural network model comprises a plurality of operators, and the types of the plurality of operators comprise a convolution operator, a reshape operator and a transpose operator; the input sample is at least one of an image sample, a text sample, an audio-visual sample and a video sample; inputting the input sample into the neural network model to obtain input tensors and output tensors of each operator in the neural network model; if it is determined that the neural network model comprises at least one candidate operator set, the candidate operator set comprises a first convolution operator, a reshape operator and a transpose operator in sequence, and each operator in the candidate operator set satisfies a preset condition, then a target operator set corresponding to the candidate operator set is determined; the target operator set comprises a second convolution operator and a depth-to-breadth operator in sequence; wherein the input tensors of the second convolution operator and the first convolution operator are the same, and the output tensors are different; the last dimension attribute of the output tensor of the first convolution operator represents a block size, and the last dimension attribute of the output tensor of the second convolution operator represents an output channel number; the output tensors of the depth-to-breadth operator and the transpose operator are the same; replacing the candidate operator set in the neural network model with the target operator set, and deploying the replaced neural network model on an AI accelerator; the preset condition comprises that the output tensor of the convolution operator and the output tensor of the reshape operator are not the output tensor of the last operator of the neural network model; the first dimension attribute in the input tensor of the reshape operator and the first dimension attribute of the output tensor of the reshape operator are the same; the first dimension attribute in the input tensor of the reshape operator and the first dimension attribute of the output tensor of the reshape operator both represent a batch size; the second dimension attribute in the input tensor of the reshape operator and the second dimension attribute of the output tensor of the reshape operator are the same; the second dimension attribute in the input tensor of the reshape operator and the second dimension attribute of the output tensor of the reshape operator both represent an image height; the third dimension attribute in the input tensor of the reshape operator and the third dimension attribute of the output tensor of the reshape operator are the same; the third dimension attribute in the input tensor of the reshape operator and the third dimension attribute of the output tensor of the reshape operator both represent an image width; the fifth dimension attribute and the sixth dimension attribute in the output tensor of the reshape operator are the same; the fifth dimension attribute and the sixth dimension attribute in the output tensor of the reshape operator both represent a block size; the fourth dimension attribute in the input tensor of the reshape operator is equal to the product of the fourth dimension attribute, the fifth dimension attribute and the sixth dimension attribute of the output tensor of the reshape operator; the fourth dimension attribute in the input tensor of the reshape operator represents an input channel number; the fourth dimension attribute of the output tensor of the reshape operator represents an output channel number; the rearrangement parameters of the transpose operator meet a preset condition, and the rearrangement parameters are determined according to the order of the dimension attributes in the input tensor and the output tensor of the transpose operator.

2. The method of claim 1, wherein, the second convolution operator in the target operator set is determined by the following way: obtaining a first weight matrix, a first bias term of the first convolutional operator and an output tensor of the reshaping operator; the output tensor of the reshaping operator comprises six dimension attributes, wherein a fourth dimension attribute of the output tensor of the reshaping operator represents an output channel number; and a fifth dimension attribute of the output tensor of the reshaping operator represents a block size; rearranging each dimension attribute in the output tensor of the reshaping operator to obtain a rearranged output tensor, and taking the rearranged output tensor as an input tensor of the second convolutional operator; generating a second weight matrix and a second bias term based on the first weight matrix of the first convolutional operator, the fourth dimension attribute and the fifth dimension attribute of the output tensor of the reshaping operator; taking the second weight matrix as a weight matrix of the second convolutional operator, and taking the second bias term as a weight matrix of the second convolutional operator.

3. The method of claim 2, wherein, The rearranging each dimension attribute in the output tensor of the reshaping operator comprises: swapping the positions of the fourth dimension attribute and the fifth dimension attribute in the output tensor of the reshaping operator; or swapping the positions of the fourth dimension attribute and the sixth dimension attribute in the output tensor of the reshaping operator; The fourth dimension attribute of the output tensor of the reshaping operator represents a channel number, and the fifth dimension attribute and the sixth dimension attribute of the output tensor of the reshaping operator both represent a block size.

4. The method of claim 2, wherein, The first weight matrix of the first convolutional operator comprises four dimension attributes, wherein a second dimension attribute of the first weight matrix represents an image height, a third dimension attribute of the first weight matrix represents an image width, and a fourth dimension attribute of the first weight matrix represents an input channel number; The generating a second weight matrix and a second bias term based on the first weight matrix of the first convolutional operator, the fourth dimension attribute and the fifth dimension attribute of the output tensor of the reshaping operator comprises: configuring each dimension attribute of a third weight matrix, wherein a first dimension attribute of the third weight matrix is an input channel number represented by the fourth dimension attribute of the output tensor of the reshaping operator; a second dimension attribute and a third dimension attribute of the third weight matrix are both a block size represented by the fifth dimension attribute of the output tensor of the reshaping operator; and a fourth dimension attribute of the third weight matrix is a product of the second dimension attribute, the third dimension attribute and the fourth dimension attribute of the first weight matrix; rearranging each dimension attribute of the third weight matrix based on a first preset rearrangement parameter to obtain a rearranged third weight matrix, and modifying a matrix shape of the rearranged third weight matrix based on a matrix shape of the first weight matrix to obtain a second weight matrix; configuring each dimension attribute of a third bias term, wherein a first dimension attribute of the third bias term is an input channel number represented by the fourth dimension attribute of the output tensor of the reshaping operator; a second dimension attribute and a third dimension attribute of the third bias term are both a block size represented by the fifth dimension attribute of the output tensor of the reshaping operator. The third bias term is rearranged based on a second preset rearrangement parameter, and a rearranged third bias term is obtained; a matrix shape of the first bias term is obtained, and the matrix shape of the rearranged third bias term is modified based on the matrix shape of the first bias term, and a second bias term is obtained.

5. The method of claim 2, wherein, The depth-to-width operator in the target operator set is generated by the following method: A depth-to-width operator is created, and an input tensor of the depth-to-width operator is configured; the zeroth input tensor of the depth-to-width operator is the output tensor of the second convolutional operator, and the first input tensor of the depth-to-width operator is the block size represented by the fourth dimension attribute of the output tensor of the reshaping operator; The output tensor of the depth-to-width operator is configured; the output tensor of the depth-to-width operator is the output tensor of the transpose operator.

6. A data processing apparatus, characterized by, Comprise: The first determination module is used for determining an input sample and a neural network model to be deployed; the neural network model comprises a plurality of operators, and the types of the plurality of operators comprise a convolutional operator, a reshaping operator and a transpose operator; the input sample is at least one of an image sample, a text sample, an audio-visual sample and a video sample; The tensor obtaining module is used for inputting the input sample into the neural network model to obtain input tensors and output tensors of each operator in the neural network model; The target operator set determination module is used for determining a target operator set corresponding to a candidate operator set if it is determined that the neural network model comprises the candidate operator set, the candidate operator set comprises a first convolutional operator, a reshaping operator and a transpose operator in sequence, and each operator in the candidate operator set satisfies a preset condition; The target operator set comprises a second convolutional operator and a depth-to-width operator in sequence; The input tensors of the second convolutional operator and the first convolutional operator are the same, and the output tensors are different; the last dimension attribute of the output tensor of the first convolutional operator represents a block size, and the last dimension attribute of the output tensor of the second convolutional operator represents an output channel number; the output tensor of the depth-to-width operator is the same as that of the transpose operator. The deployment module is used for replacing the candidate operator set in the neural network model with the target operator set, and deploying the replaced neural network model on an AI accelerator; The preset condition comprises that the output tensor of the convolutional operator and the output tensor of the reshaping operator are not the output tensor of the last operator of the neural network model; The first dimension attribute in the input tensor of the reshaping operator is the same as the first dimension attribute of the output tensor of the reshaping operator; and the first dimension attribute in the input tensor of the reshaping operator and the first dimension attribute of the output tensor of the reshaping operator both represent a batch size; The second dimension attribute in the input tensor of the reshaping operator is the same as the second dimension attribute of the output tensor of the reshaping operator; and the second dimension attribute in the input tensor of the reshaping operator and the second dimension attribute of the output tensor of the reshaping operator both represent an image height; The third dimension attribute in the input tensor of the reshaping operator is the same as the third dimension attribute of the output tensor of the reshaping operator; and the third dimension attribute in the input tensor of the reshaping operator and the third dimension attribute of the output tensor of the reshaping operator both represent image width. The fifth dimension attribute and the sixth dimension attribute in the output tensor of the reshaping operator are the same; and the fifth dimension attribute and the sixth dimension attribute in the output tensor of the reshaping operator both represent block size. The fourth dimension attribute in the input tensor of the reshaping operator is equal to the product of the fourth dimension attribute, the fifth dimension attribute and the sixth dimension attribute of the output tensor of the reshaping operator; and the fourth dimension attribute in the input tensor of the reshaping operator represents input channel number; and the fourth dimension attribute of the output tensor of the reshaping operator represents output channel number. The rearrangement parameter of the transposition operator meets a preset condition, and the rearrangement parameter is determined according to the order of the dimension attributes in the input tensor and the output tensor of the transposition operator.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program comprises instructions that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1-6. The processor executes the computer program to implement the steps of the method in any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1-5.

Images