Data processing method and device, storage medium and electronic equipment
By constructing a convolution kernel and using cross-offset jump reading technology, the problem of low efficiency of transposed operators in deep learning is solved, and the hardware parallelism and overall network of transposed operators are improved without adding special circuits is achieved. efficiency.
Patent Information
- Application Number
- CN202311013212.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-11
- Publication Date
- 2025-02-18
AI Technical Summary
The frequent occurrence of transpose operators in deep learning leads to unfriendly locality and low parallelism of IPs such as CPU/GPU, and the switching efficiency between NPUs is large, affecting the overall network efficiency.
By constructing a convolution kernel, cross-offset jump reading is performed based on the arrangement of input tensors and output tensors, transposition operations are realized without adding special circuits, and the hardware parallelism of transposition operators is improved.
Without adding special circuits, the hardware parallelism of the transpose operator is improved, the NPU design area is reduced, the overall network efficiency is improved, and larger transpose operator cache and prefetching are supported.
Smart Images
Figure CN119473218A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure belongs to the field of data processing technology, and in particular, relates to a data processing method and device, a storage medium, and an electronic device. Background Art
[0002] Deep learning (DL) is a type of machine learning (ML), which is the only way to achieve artificial intelligence (AI). The concept of deep learning originated from the study of artificial neural networks. Multilayer perceptrons with multiple hidden layers are a type of deep learning structure. Deep learning combines low-level features to form more abstract high-level representations of attribute categories or features to discover distributed feature representations of data. The motivation for studying deep learning is to establish a neural network that simulates the human brain for analysis and learning. It imitates the mechanism of the human brain to interpret data, such as images, sounds, and text.
[0003] There is a relatively commonly used operator in deep learning: the transpose operator (Transpose). The transpose operator implements the transpose operation of the feature map (feature). For example, it is most commonly used in the pre-processing and post-processing of some networks, and also appears in the middle layer of models such as segmentation applications and natural language translation applications.
[0004] Since the transpose operator appears frequently and its transposed dimensions are diverse, if it runs on IPs such as CPU / GPU, it will bring about efficiency overheads such as local unfriendliness, low parallelism, and switching between the NPU and it, which will affect the overall efficiency of the entire network; however, if a transpose operator (Transpose) module of the NPU is specially designed to achieve sufficient parallelism, additional design area needs to be added. Summary of the invention
[0005] The present disclosure provides a data processing method and device, a storage medium and an electronic device, which can enable a target circuit or device to have the ability to process a transposition operator without adding a dedicated circuit.
[0006] In a first aspect, an embodiment of the present disclosure provides a data processing method. The data processing method comprises: obtaining an input tensor and a convolution kernel, wherein the convolution kernel is determined based on an input tensor arrangement of the input tensor and an output tensor arrangement of a target output tensor; performing a convolution operation on the input tensor based on the convolution kernel through a multiplication accumulator array; and performing a cross-offset jump read on the result of the convolution operation to output the target output tensor having the output tensor arrangement, wherein the output tensor arrangement and the input tensor arrangement have a transposed relationship.
[0007] In an implementation of the first aspect, the data processing method further includes: configuring a cross-offset jump parameter for output based on the input tensor arrangement method and the output tensor arrangement method; wherein performing a cross-offset jump reading on the result of the convolution operation includes: reading out the result of the convolution operation as the target output tensor based on the cross-offset jump parameter for output through an output circuit.
[0008] In an implementation of the first aspect, the data processing method further includes: configuring cross-offset jump parameters for input based on the input tensor arrangement method and the output tensor arrangement method, wherein obtaining the input tensor and the convolution kernel includes: reading in the input tensor from the outside based on the cross-offset jump parameters for input through a first input circuit; and reading in the convolution kernel from the outside through a second input circuit.
[0009] In an implementation of the first aspect, the data processing method further includes: configuring an initialization value for the convolution kernel; and configuring a convolution step, boundary padding, and size of the convolution kernel.
[0010] In an implementation of the first aspect, reading the input tensor from the outside based on the cross-offset jump parameter for input through the first input circuit includes: reading external data into the input tensor that is continuous in one dimension based on the cross-offset jump parameter for input.
[0011] In an implementation of the first aspect, the cross-offset jump parameter for input includes a row jump value and a surface jump value; wherein reading the external data into the input tensor as continuous in one dimension based on the cross-offset jump parameter for input includes: reading the external data of different rows into one dimension continuous among channels, heights and widths based on the row jump value; and reading the external data of different surfaces into the one dimension continuous based on the surface jump value.
[0012] In an implementation of the first aspect, performing cross-offset jump reading on the result of the convolution operation to output the target output tensor having the output tensor arrangement includes: based on the cross-offset jump parameter used for output, converting the result of the convolution operation from continuous in one dimension to continuous in another dimension.
[0013] In an implementation of the first aspect, the cross-offset jump parameter for output includes a row jump value and a surface jump value, wherein converting the result of the convolution operation from continuous in one dimension to continuous in another dimension based on the cross-offset jump parameter for output includes: converting the results of the convolution operation of different rows from continuous in one dimension of channel, height and width to continuous in another dimension based on the row jump value; and converting the results of the convolution operation of different surfaces from continuous in the one dimension to continuous in the other dimension based on the surface jump value.
[0014] In an implementation of the first aspect, the data processing method further includes: storing the input variable, the convolution kernel, and the result of the convolution operation of the multiplier accumulator array.
[0015] In an implementation of the first aspect, the input tensor is arranged in (N, C1, H, W, C2), (N, C, H, W) or (NHWC), and the output tensor is arranged in (N, C1, H, W, C2), (N, C, H, W) or (NHWC), where N represents batch, C represents channel, H represents height, W represents width, and C1 and C2 are decompositions of C.
[0016] In a second aspect, an embodiment of the present disclosure provides a data processing device. The data processing device includes: an input circuit configured to obtain an input tensor and a convolution kernel, wherein the convolution kernel is determined based on an input tensor arrangement of the input tensor and an output tensor arrangement of a target output tensor; a multiplier accumulator array configured to perform a convolution operation on the input tensor based on the convolution kernel; an output circuit configured to read the result of the convolution operation across offset jumps to output the target output tensor having the output tensor arrangement, wherein the output tensor arrangement and the input tensor arrangement have a transposed relationship; and a cache configured to store the input tensor, the convolution kernel, and the result of the convolution operation.
[0017] In an implementation of the second aspect, the data processing device also includes: a controller, configured to configure cross-offset jump parameters for output based on the input tensor arrangement and the output tensor arrangement, wherein the output circuit is configured to read out the result of the convolution operation as the target output tensor based on the cross-offset jump parameters for output.
[0018] In an implementation of the second aspect, the controller is further configured to configure cross-offset jump parameters for input based on the input tensor arrangement and the output tensor arrangement, wherein the input circuit includes: a first input circuit, configured to read in the input tensor from the outside based on the cross-offset jump parameters for input; and a second input circuit, configured to read in the convolution kernel from the outside.
[0019] In a third aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored. The computer program is executed to implement the data processing method according to the first aspect of the present disclosure.
[0020] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory configured to store executable instructions; and a processor configured to execute the instructions to implement the data processing method described in the first aspect of the present disclosure.
[0021] In the data processing method and device provided by the embodiments of the present disclosure, a convolution kernel is constructed according to an input tensor arrangement and an output tensor arrangement. The arrangement of the tensors can be changed without changing the original data. At the same time, based on the cross-offset jump characteristics of reading and writing, the arrangement of the input tensor can be transposed to the arrangement of the output tensor, thereby increasing the hardware parallelism of the transposition operator and enabling the target circuit or data processing device to have the ability to process the transposition operator without adding a special circuit.
[0022] In addition, compared with the prior art, the multiplication and accumulation device array can be reused in the transposition operator to reduce the NPU design area.
[0023] Furthermore, in the data processing method and device provided by the embodiments of the present disclosure, the transposition operator can reuse the multiplication and accumulation device array to reduce the NPU design area.
[0024] Furthermore, the data processing method and device provided by the embodiments of the present disclosure can reuse the cache of the multiplier accumulator array, can cache larger transposition operators, support prefetching, improve efficiency, and can implement various transposition operators through inductive combination. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 Shown is a structural schematic diagram of a data processing device in an embodiment of the present disclosure.
[0026] Figure 2 Shown is a flow chart of a data processing method in an embodiment of the present disclosure.
[0027] Figure 3 Shown is a schematic diagram of the tensor arrangement method NC1HWC2 in the data processing method in an embodiment of the present disclosure.
[0028] Figure 4 Shown is a flowchart of reading memory-contiguous external data into the input tensor having the input tensor arrangement based on the cross-offset jump parameter for input in an embodiment of the present disclosure.
[0029] Figure 5 Shown is a flowchart of converting the result of a convolution operation from memory continuous to channel continuous based on a cross-offset jump parameter for output in an embodiment of the present disclosure.
[0030] Figure 6 Shown is a schematic diagram of an input tensor arrangement method and an output tensor arrangement method in an embodiment of the present disclosure.
[0031] Figure 7 Display as transposed Figure 6 Schematic diagram of the convolution kernels arranged in .
[0032] Figure 8 Another schematic diagram showing an input tensor arrangement method and an output tensor arrangement method in an embodiment of the present disclosure is shown.
[0033] Fig. 9 Display as Figure 8 Schematic diagram of the grouping of the input tensors in .
[0034] Fig.10 Display as transposed Fig. 9 Schematic diagram of the convolution kernels arranged in .
[0035] Fig.11 Another schematic diagram showing an input tensor arrangement method and an output tensor arrangement method in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0036] The following is an explanation of the embodiments of the present disclosure by specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.
[0037] It should be noted that the illustrations provided in the following embodiments are only used to schematically illustrate the basic concept of the present disclosure. Therefore, the drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.
[0038] In practical applications, different data of a tensor are multiplied by different weights and then added to obtain the result. This is a widely used multiplication and addition operation, which includes but is not limited to matrix multiplication, convolution, deconvolution, etc. Taking convolution as an example, the traditional convolution operation includes input features, weights, biases, and output features. The input features and a set of weights are multiplied and added to the bias to obtain the output features. The specific formula is shown in Formula 1 below:
[0039]
[0040] in, Represents the complete output feature map of the convolution, N i represents a batch of feature maps, represents the output feature map channel, express The corresponding bias, C in Represents the input feature map channel, input(N i ,k) represents the complete input feature map.
[0041] In the field of artificial intelligence, convolution operations are usually implemented using multiplier-accumulators in neural-network processing units (NPUs). However, with the gradual popularization of artificial intelligence technology and the continuous updating and iteration of algorithms, the demand for NPU computing power is increasing, and the area and power consumption required by NPU are also increasing.
[0042] At least for the above-mentioned problems, the disclosed embodiments provide a data processing solution for NPU. In the data processing method and device, the input tensor can be various types of data, including but not limited to image tensors, video tensors, voice signal tensors and / or text tensors in natural language processing. The output tensor is a feature graph or feature map obtained after the input tensor is processed. The data processing device disclosed in this embodiment constructs a convolution kernel according to the arrangement of the input tensor and the arrangement of the output tensor. The arrangement of the tensor can be changed without changing the original data. At the same time, based on the cross-offset jump characteristics of reading and writing, the arrangement of the input tensor is transposed to the arrangement of the output tensor, so as to increase the hardware parallelism of the transposition operator, and enable the data processing device to have the ability to process the transposition operator without adding special circuits.
[0043] Figure 1 1 is a schematic diagram showing the structure of a data processing device 1 provided according to an embodiment of the present disclosure. The data processing device 1 may be any processing device capable of implementing multiplication and addition operations. For example, in some embodiments, the data processing device 1 may be an NPU, but the present disclosure is not limited thereto. Figure 1As shown, the data processing device 1 provided in the embodiment of the present disclosure includes an input circuit 12 (RDMA), an output circuit 14 (WDMA), a cache 15 (cache), a multiplier accumulator array 11, and a controller 13 (such as Figure 1 ). This embodiment utilizes the input circuit 12 (RDMA), the output circuit 14 (WDMA), the cache 15 (cache), the multiplier accumulator array 11 and the controller 13 to complete the transposition requirement through the convolution operation designed in this embodiment.
[0044] In an embodiment of the present disclosure, the data processing device 1 has the ability to process a transpose operator. The input circuit 12 is configured to obtain an input tensor and a convolution kernel, and the convolution kernel is determined based on the input tensor arrangement of the input tensor and the output tensor arrangement of the target output tensor. The multiplier accumulator array 11 is configured to perform a convolution operation on the input tensor based on the convolution kernel. The output circuit 14 is configured to read the result of the convolution operation across an offset jump to output the target output tensor having the output tensor arrangement, and there is a transposition relationship between the output tensor arrangement and the input tensor arrangement. The cache 15 is configured to store the input tensor, the convolution kernel, and the result of the convolution operation.
[0045] The controller 13 is configured to configure the cross-offset jump parameter for output based on the input tensor arrangement mode and the output tensor arrangement mode. At this time, the output circuit is configured to read out the result of the convolution operation as the target output tensor based on the cross-offset jump parameter for output.
[0046] The controller 13 is further configured to configure a cross-offset jump parameter for input based on the input tensor arrangement mode and the output tensor arrangement mode. At this time, the input circuit includes a first input circuit and a second input circuit. The first input circuit is configured to read the input tensor from the outside based on the cross-offset jump parameter for input. The second input circuit is configured to read the convolution kernel from the outside.
[0047] In the embodiment of the present disclosure, the input circuit 12, the controller 13, the multiplier accumulator array 11, the output circuit 14 and the cache 15 are configured to perform Figure 2 The data processing method shown in Figure 2 As shown, the data processing method includes the following steps S21 to S24.
[0048] In step S21, an input tensor and a convolution kernel are obtained, wherein the convolution kernel is determined based on an input tensor arrangement of the input tensor and an output tensor arrangement of a target output tensor.
[0049] In step S22, a convolution operation is performed on the input tensor based on the convolution kernel through a multiplication accumulator array.
[0050] In step S23, the result of the convolution operation is read across offset jumps to output the target output tensor having the output tensor arrangement, and there is a transposition relationship between the output tensor arrangement and the input tensor arrangement.
[0051] This embodiment constructs a convolution kernel according to the input tensor arrangement and the output tensor arrangement. The convolution kernel is equivalent to a mask, which does not change the original data, but is only used to assist in transforming the written arrangement. According to the cross-offset jump characteristics of reading and writing by the input circuit 12 (RDMA) and the output circuit 14 (WDMA), the hardware parallelism of the transposition operator is increased to improve the efficiency. Compared with ordinary convolution, the convolution kernel of this embodiment has a small convolution weight, and it only performs cumulative calculation on one channel, so the amount of calculation is small, so that the data processing device 1 has the ability to process the transposition operator.
[0052] In some possible implementations, the data processing method may further include: configuring an initialization value for the convolution kernel, configuring the convolution step, boundary padding and size of the convolution kernel.
[0053] The input and output feature maps of the NPU have a fixed layout (arrangement), generally (N, C, H, W) (can be abbreviated as NCHW), (N, H, W, C) (can be abbreviated as NHWC) or (N, C1, H, W, C2) (can be abbreviated as NC1HWC2), etc. In some possible implementations, the input tensor arrangement includes but is not limited to (N, C1, H, W, C2), (N, C, H, W) or (N, H, W, C), and the output tensor arrangement includes but is not limited to (N, C1, H, W, C2), (N, C, H, W) or (N, H, W, C), where N represents batch, C represents channel, H represents height, W represents width, and C1 and C2 are decompositions of C. The input tensor arrangement is different from the output tensor arrangement. For example, the input tensor is arranged in (N, C1, H, W, C2), the output tensor is arranged in (N, C, H, W) or (N, H, W, C), and the transposition operator is transposed by convolution and cross-offset jump.
[0054] Figure 3Shown is a schematic diagram of the tensor arrangement method NC1HWC2 in the data processing method in the embodiment of the present disclosure. The tensor arrangement NC1HWC2 is an extended arrangement commonly used by NPU. N generally represents how many batches there are in the highest dimension, that is, how many batches of three-dimensional data there are. CHW represents three-dimensional data, C is the channel, C1 and C2 are the decomposition of C, that is, the Z-axis direction, H is the height, and W is the width. NC1HWC2 is generally used for optimization of specific CPUs, GPUs, and NPUs. Splitting C out is more friendly to parallel reading of data. C1*C2 will form a complete C, and C2 is a certain alignment factor, such as 8Byte alignment. If C is not aligned according to the alignment factor, it will be treated as padding at the end. For NC1HWC2, C1 and C2 are the decomposition of C, such as Figure 2 As shown, assuming that C is 15 and C2 is aligned to 8, then C1=2, C2=8, C1*C2=16, which is greater than C=15, and the extra 1 is the complement number, for example NC1HWC2=(1, 2, 2, 8, 8).
[0055] In this embodiment, C2 in NC1HWC2 is exemplified as 8 (int8 type). Depending on the NPU structure, it may also be int4, fp16, and other types. The difference between different types lies in the bit width of their data elements, such as 8bit, 16bit, and 32bit. The difference in the method of this embodiment lies in the set span size, and its parameters are also determined by the size of C2. C2 is decomposed from the input channel, and its value is generally determined by the hardware design. The C2 value is determined according to the hardware DMA read and write and MAC data acquisition calculation method, which is generally conducive to DMA read and write and MAC read.
[0056] According to an embodiment of the present disclosure, the input circuit 12 may use a direct memory access (Read Direct Memory Access, RDMA) technology to read data, but the present disclosure is not limited thereto.
[0057] For the input or output feature graph, the feature graph of the convolution input and output is controlled to achieve the corresponding arrangement effect by adding a cross-offset jump in the input circuit 12 (RDMA) and the output circuit 14 (WDMA). For example, Figure 2 The line stride and surf stride are shown.
[0058] The row span indicates the trigger row jump, which is triggered every time the read or write reaches the end of the row (the row here is represented by: W*C2), such as Figure 3 As shown in the area marked as a, it is triggered at point 56.
[0059] The surface span indicates the trigger surface jump, which is triggered every time the read or write reaches the end of the surface (the surface here is represented by: H*W*C2), such as Figure 2 As shown in the area marked as b, it is triggered at the serial number point 120.
[0060] In some possible implementations, the input circuit 12 includes a first input circuit and a second input circuit. The input circuit 12 configures the cross-offset jump parameter for input based on the input tensor arrangement mode and the output tensor arrangement mode. The input circuit 12 obtains the input tensor and the convolution kernel including: reading the input tensor from the outside based on the cross-offset jump parameter for input through the first input circuit; and reading the convolution kernel from the outside through the second input circuit.
[0061] Specifically, reading the input tensor from the outside based on the cross-offset jump parameter for input through the first input circuit includes: reading the external data that is continuous in the memory into the input tensor that is continuous in one dimension and has the input tensor arrangement method based on the cross-offset jump parameter for input.
[0062] Figure 4 The flowchart of reading continuous external data from memory into the input tensor having the input tensor arrangement mode based on the cross-offset jump parameter for input in the embodiment of the present disclosure is shown. Specifically, in some possible implementations, such as Figure 4 As shown, the cross-offset jump parameter for input includes a row jump value and a plane jump value. Reading the external data in memory into the input tensor continuous in one dimension based on the cross-offset jump parameter for input includes the following steps S211 to S212.
[0063] In step S211, the external data of different rows are read in to be continuous in one dimension among channel, height and width based on the row skip value.
[0064] In step S212, the external data of different surfaces are read in to be continuous in the one dimension based on the surface jump value.
[0065] It should be noted that the cross-offset jump configured in the input circuit 12 is not limited to a two-stage jump of a row jump and a surface jump, but can also be extended to a jump mode of more than two stages, so as to realize the related transposition in one step.
[0066] In some possible implementations, a cross-offset jump parameter for output is configured based on the input tensor arrangement and the output tensor arrangement. Reading the result of the convolution operation across the offset jump includes: reading out the result of the convolution operation as the target output tensor based on the cross-offset jump parameter for output through an output circuit.
[0067] Specifically, performing cross-offset jump reading on the result of the convolution operation to output the target output tensor having the output tensor arrangement includes: based on the cross-offset jump parameter used for output, converting the result of the convolution operation from continuous in one dimension to continuous in another dimension.
[0068] Figure 5 The flowchart of converting the result of the convolution operation from memory continuous to channel continuous based on the cross-offset jump parameter for output in the embodiment of the present disclosure is shown. Specifically, Figure 5 As shown, in some possible implementations, the cross-offset jump parameter for output includes a row jump value and a plane jump value. The step of converting the result of the convolution operation from a continuous dimension to a continuous dimension of another dimension based on the cross-offset jump parameter for output includes the following steps S241 to S242.
[0069] In step S241, the results of the convolution operation of different rows are converted from continuity in one dimension of channel, height and width to continuity in another dimension based on the row skip value.
[0070] In step S242, the results of the convolution operation on different surfaces are converted from continuous in the one dimension to continuous in the other dimension based on the surface jump value.
[0071] In the disclosed embodiment, for NHWC, since C is in the last dimension, it is called channel continuous. For NCHW, since W is in the last dimension, it is called width continuous. In addition, for NC1HWC2, since C2 is in the last dimension, it is partially channel continuous.
[0072] In some possible implementations, the cache 15 is configured to store the input variable, the convolution kernel, and the result of the convolution operation of the multiplier accumulator array 11.
[0073] Next, the implementation process of this embodiment will be described in detail through a specific example. Figure 6 The diagram shows a schematic diagram of the input tensor arrangement method and the output tensor arrangement method in the embodiment of the present disclosure. Figure 6 As shown, this specific example takes the input tensor arrangement as NC1HWC2, the output tensor arrangement as NHWC, and the implementation of the transposition operator transposition of NC1HWC2->NHWC as an example to illustrate, where NC1HWC2=(1,2,2,8,8), NHWC=(1,2,8,16).
[0074] 1) First, construct a convolution kernel. Figure 7 Display as transposed Figure 6 Schematic diagram of the convolution kernel arranged in . Figure 7 As shown, the convolution kernel is CoCiKhKw=(1,8,1,1), where the convolution kernel Ci=NC1HWC2=C1*C2(1,2,2,8,8).
[0075] 2) Initialization values are configured for the convolution kernel, and all values of the convolution kernel are initialized to 1 (the NPU reads the convolution kernel data into the internal cache via RDMA).
[0076] 3) Configure the convolution stride of the NPU hardware convolution to (1, 1) and the border padding to (0, 0, 0, 0).
[0077] 4) Configure the cross-offset jump of the NPU hardware input to: line jump value (line) = 8, surface jump value (surf) = 16.
[0078] 5) The output is regarded as a feature map with a width of 1 (equivalent to straightening, the extra dimension in width will be distributed to the high dimension), and the two-level cross-offset jump of the NPU hardware output is configured as: line jump value (line) = 16, surface jump value (surf) = 1. These two jump configurations, line jump value and surface jump value, can be understood as trigger points. After scanning the trigger point, the corresponding jump action is performed. Line is to jump the line (the unit needs *C2, that is, *8), and surf is to jump the surface (the unit needs *C2, that is, *8). In general, the NPU display will jump the line instead of the surf first, because the surface jump value is larger than the line jump value, but here it is reversed, the surface jump value is smaller than the line jump value. At this time, as long as C1 points are scanned at the 0 position, the entire surface (8*16 points) will be jumped, and after the entire cube channel jump is completed, it will return to the 8 position.
[0079] 6) According to the jump output scenario of the NPU hardware, only one convolution can obtain the target output of NHWC = (1, 2, 8, 16).
[0080] Next, the implementation process of this embodiment will be described in detail through another specific example. Figure 8 Another schematic diagram of the arrangement of input tensors and output tensors in the embodiment of the present disclosure is shown. Figure 8 As shown, this specific example takes the input tensor arrangement as NC1HWC2, the output tensor arrangement as NCHW, and the implementation of the transposition operator transposition of NC1HWC2->NCHW as an example to illustrate, where NC1HWC2=(1,2,2,8,8), NHWC=(1,16,2,8).
[0081] Fig. 9 Display as Figure 8Schematic diagram of the grouping of the input tensor arrangement in . The feature map of this input tensor arrangement can be divided into four blocks, of which ①② is a group, ③④ is a group, and a total of two groups perform one round according to the following process, and the convolution write position of each group is different.
[0082] 1) Since the transposition needs to be continuous in the W direction, this embodiment constructs a common convolution kernel CoCiKhKw=(64,8,1,8), Fig.10 Display as transposed Fig. 9 Schematic diagram of the convolution kernel arranged in . Fig.10 As shown, in order to output 64 points at a time, 64 8x1x8 convolution kernels are constructed in this embodiment. In other possible embodiments, a (8, 1, 1, 8) convolution kernel can also be constructed, but the task will be more divided.
[0083] 2) Initialize the convolution kernel with a value where each 8x1x8 weight of the convolution kernel contains only one 1 (e.g. Fig.10 The points that are 1 are recursively calculated in order.
[0084] 3) Configure the convolution stride of the NPU hardware convolution to (1,1) and the border padding to (0,0,0,0).
[0085] 4) Configure the cross-offset jump of the NPU hardware input to: line jump value (line) = 8, surface jump value (surf) = 16.
[0086] 5) The output is regarded as a feature map with a width of 1 (equivalent to straightening, the extra dimension in width will be distributed to the high dimension. The width of the example here is just 8, but in most cases it is greater than 8). At the same time, the two-level cross-offset jump of the NPU hardware output is configured as: line jump value (line) = 1, surface jump value (surf) = H*W / C2.
[0087] 6) According to the jump output scenario of the NPU hardware, only one convolution can obtain the target output of NCHW = (1, 16, 2, 8).
[0088] Fig.11 Another schematic diagram of the arrangement of input tensors and output tensors in the embodiment of the present disclosure is shown. Assume that the transposition calculation of the transposition operator of NC1HWC2->NW1HCW2 needs to be implemented, where NC1HWC2=(1,2,2,8,8), NW1HCW2=(1,1,2,16,8). The implementation process principle of the method is the same as above, and no further examples are given here.
[0089] It can be seen that in this embodiment, the transposition operator can reuse the multiplication accumulator array 11 array, reduce the NPU design area, increase the hardware parallelism of the transposition operator, and improve efficiency. The cache of the multiplication accumulator array 11 array can cache larger transposition operator calculation blocks, and support prefetching to improve efficiency. In theory, it can enable inductive combination to realize various transposition operators.
[0090] The embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored. The computer program is executed to implement the data processing method provided in the embodiment of the present disclosure.
[0091] In some possible implementations, any combination of one or more storage media may be used. The storage medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.
[0092] The embodiment of the present disclosure also provides an electronic device, the electronic device comprising a memory and a processor. The memory is configured to store executable instructions. The processor is configured to execute the instructions to implement the data processing method described above in the embodiment of the present disclosure.
[0093] In summary, in the data processing method and device provided by the embodiment of the present disclosure, the convolution kernel is constructed according to the arrangement of the input tensor and the arrangement of the output tensor. The arrangement of the tensor can be changed without changing the original data. At the same time, the arrangement of the input tensor is transposed to the arrangement of the output tensor based on the cross-offset jump characteristics of reading and writing, so as to increase the hardware parallelism of the transposition operator, and enable the target circuit or data processing device to have the ability to process the transposition operator without adding a special circuit. In addition, compared with the prior art, the transposition operator can reuse the multiplication accumulator array to reduce the NPU design area. Furthermore, in the data processing method and device provided by the embodiment of the present disclosure, the transposition operator can reuse the multiplication accumulator array to reduce the NPU design area. Further, the data processing method and device provided by the embodiment of the present disclosure can reuse the cache of the multiplication accumulator array, can cache a larger transposition operator, support pre-fetching, improve efficiency, and can enable inductive combination to realize various transposition operators. Therefore, the present disclosure overcomes the various defects of the prior art and has a high application prospect.
[0094] The examples are only illustrative of the principles and effects of the present disclosure and are not intended to limit the present disclosure. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present disclosure. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed in the present disclosure should still be covered by the claims of the present disclosure.
Claims
1. A data processing method, characterized in that: include: Acquire an input tensor and a convolution kernel, wherein the convolution kernel is determined based on an input tensor arrangement of the input tensor and an output tensor arrangement of a target output tensor; Performing a convolution operation on the input tensor based on the convolution kernel through a multiplier accumulator array; and The result of the convolution operation is read across offset jumps to output the target output tensor having the output tensor arrangement, wherein the output tensor arrangement has a transposed relationship with the input tensor arrangement.
2. The data processing method according to claim 1, characterized in that: Also includes: configuring a cross-offset jump parameter for output based on the input tensor arrangement and the output tensor arrangement, The step of performing cross-offset jump reading on the result of the convolution operation comprises: The result of the convolution operation is read out as the target output tensor through an output circuit based on the cross-offset jump parameter for output.
3. The data processing method according to claim 1, characterized in that: Also includes: configuring a cross-offset jump parameter for an input based on the input tensor arrangement and the output tensor arrangement, The steps to obtain the input tensor and convolution kernel include: Reading the input tensor from the outside based on the cross-offset jump parameter for input through a first input circuit; as well as The convolution kernel is read in from the outside through a second input circuit.
4. The data processing method according to claim 1, characterized in that: Also includes: Configuring an initialization value for the convolution kernel; as well as Configure the convolution step, border padding, and size of the convolution kernel.
5. The data processing method according to claim 3, characterized in that: Reading the input tensor from the outside based on the cross-offset jump parameter for input through the first input circuit includes: reading external data into the input tensor that is continuous in one dimension based on the cross-offset jump parameter for input.
6. The data processing method according to claim 5, characterized in that: The cross-offset jump parameters for input include a row jump value and a surface jump value, Wherein reading the external data into the input tensor that is continuous in one dimension based on the cross-offset skip parameter for input comprises: Reading the external data of different rows as continuous in one dimension among channel, height and width based on the row skip value; and The external data of different surfaces are read in to be continuous in the one dimension based on the face jump value.
7. The data processing method according to claim 1, characterized in that: Performing cross-offset jump reading on the result of the convolution operation to output the target output tensor having the output tensor arrangement includes: converting the result of the convolution operation from continuous in one dimension to continuous in another dimension based on the cross-offset jump parameter used for output.
8. The data processing method according to claim 7, characterized in that: The cross-offset jump parameter for output includes a row jump value and a plane jump value, The step of converting the result of the convolution operation from a continuous dimension to a continuous dimension of another dimension based on the cross-offset jump parameter for output includes: Converting the results of the convolution operation of different rows from a continuous dimension of one of the channel, height and width to a continuous dimension of another dimension based on the row skip value; and Results of the convolution operations on different surfaces are converted from being continuous in the one dimension to being continuous in the other dimension based on the surface jump value.
9. The data processing method according to claim 1, characterized in that: Also includes: The result of the convolution operation of the input variable, the convolution kernel, and the multiplier accumulator array is stored.
10. The data processing method according to claim 1, characterized in that: The input tensor is arranged in (N, C1, H, W, C2), (N, C, H, W) or (N, H, W, C), and the output tensor is arranged in (N, C1, H, W, C2), (N, C, H, W) or (N, H, W, C). Where N represents batch, C represents channel, H represents height, W represents width, and C1 and C2 are the decomposition of C.
11. A data processing device, characterized in that: include: An input circuit is configured to obtain an input tensor and a convolution kernel, wherein the convolution kernel is determined based on an input tensor arrangement of the input tensor and an output tensor arrangement of a target output tensor; a multiplier accumulator array, configured to perform a convolution operation on the input tensor based on the convolution kernel; an output circuit configured to perform a cross-offset jump read on the result of the convolution operation to output the target output tensor having the output tensor arrangement, wherein the output tensor arrangement has a transposed relationship with the input tensor arrangement; as well as A cache is configured to store the input tensor, the convolution kernel, and the result of the convolution operation.
12. The data processing device according to claim 11, characterized in that: Also includes: a controller configured to configure a cross-offset jump parameter for an output based on the input tensor arrangement and the output tensor arrangement, The output circuit is configured to read out the result of the convolution operation as the target output tensor based on the cross-offset jump parameter for output.
13. The data processing method according to claim 12, characterized in that: The controller is further configured to configure a cross-offset jump parameter for an input based on the input tensor arrangement and the output tensor arrangement, wherein the input circuit comprises: A first input circuit configured to read the input tensor from the outside based on the cross-offset jump parameter for input; and The second input circuit is configured to read the convolution kernel from the outside.
14. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed to implement the data processing method according to any one of claims 1 to 10.
15. An electronic device, characterized in that: include: a memory configured to store executable instructions; as well as A processor is configured to execute the instructions to implement the data processing method according to any one of claims 1 to 10.