Tensor data processing method and device and storage medium

Abstract

According to the tensor data processing method and device and the storage medium provided by the embodiment of the invention, all the input tensors and the output tensors of the to-be-processed operator are preprocessed, so that the calculation efficiency during execution of the to-be-processed operator is effectively improved, and the maximum performance of operator calculation is brought into play.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular to a tensor data processing method, device and storage medium. Background Art

[0002] In the design of processor hardware (such as graphics processing unit GPU or artificial intelligence processor, etc.), various operators such as element-wise operators are usually used to perform tensor data operations.

[0003] When an operator operates on tensor data, the input tensor and / or output tensor data of the operator are placed differently in the memory, which is not conducive to the operation of the operator and seriously affects the operation efficiency of the operator. Summary of the invention

[0004] The embodiments of the present application provide a tensor data processing method, device and storage medium to improve operator computing efficiency.

[0005] In a first aspect, an embodiment of the present application provides a tensor data processing method, including:

[0006] Filter the target input tensor from all input tensors of the operator to be processed;

[0007] Determine the memory mode category of the operator to be processed according to the memory modes of the target input tensor and the output tensor;

[0008] Determining a target stride according to the memory mode category and a common shape of all the input tensors;

[0009] According to the common shape and the target step size, the shapes and step sizes of all input tensors and output tensors of the operator to be processed are adjusted.

[0010] In a second aspect, an embodiment of the present application provides a tensor data processing device, including:

[0011] A filtering module is used to filter out the target input tensor from all input tensors of the operator to be processed;

[0012] A classification module, used to determine the memory mode category of the operator to be processed according to the memory mode of the target input tensor and the output tensor;

[0013] A decision module, configured to determine a target step size according to the memory mode category and a common shape of all the input tensors;

[0014] An adjustment module is used to adjust the shapes and step sizes of all input tensors and output tensors of the operator to be processed according to the common shape and the target step size.

[0015] In a third aspect, an embodiment of the present application provides a tensor data processing device, including: at least one processor and a memory;

[0016] The memory stores computer-executable instructions;

[0017] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the method described in the first aspect is implemented.

[0019] The tensor data processing method, device and storage medium provided in the embodiments of the present application screen out the target input tensor from all the input tensors of the operator to be processed; determine the memory mode category of the operator to be processed according to the memory mode of the target input tensor and the output tensor; determine the target step size according to the memory mode category and the common shape of all the input tensors; and adjust the shape and step size of all the input tensors and output tensors of the operator to be processed according to the common shape and the target step size. By determining the memory mode category and the common shape, the target step size is determined as the common step size, and then the shape and step size of all the input tensors and output tensors of the operator to be processed are preprocessed based on the common shape and the target step size, so that the memory of all the input tensors and output tensors is continuous and the data placement is unified, thereby effectively improving the computational efficiency of the operator to be processed when it is executed, so as to maximize the performance of the operator calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0021] Figure 1 A schematic diagram of a processing process of an operator to be processed provided in one embodiment of the present application;

[0022] Figure 2 A flowchart of a tensor data processing method provided for one embodiment of the present application;

[0023] Figure 3 A flowchart of a tensor data processing method provided by another embodiment of the present application;

[0024] Figure 4 A schematic diagram of the structure of a tensor data processing device provided in one embodiment of the present application;

[0025] Figure 5A schematic diagram of the structure of a tensor data processing device provided in another embodiment of the present application;

[0026] Figure 6 is a structural diagram showing a board according to an embodiment of the present application;

[0027] Figure 7 is a structural diagram showing a combined processing device according to an embodiment of the present application;

[0028] Figure 8 is a schematic diagram showing the internal structure of a single-core computing device according to an embodiment of the present application;

[0029] Fig. 9 is a schematic diagram showing the internal structure of a multi-core computing device according to an embodiment of the present application;

[0030] Fig.10 Schematic diagram showing the internal structure of a processor core according to an embodiment of the present application.

[0031] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0032] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0033] In order to clearly understand the technical solution of the present application, the solution of the prior art is first introduced in detail.

[0034] In the design of processor hardware (such as graphics processors GPU or artificial intelligence processors, etc.), various operators are usually used to perform tensor data operations. When the operator operates on tensor data, the input tensor is usually read from the memory as the input of the operator, and the output tensor is written to the memory after the operator operation. However, the placement of the input tensor and / or output tensor data in the memory is not uniform, which is not conducive to the operation of the operator and seriously affects the operation efficiency of the operator.

[0035] For example, the Element-wise operator operates on the corresponding elements between each input tensor. The premise for achieving efficient computing is that the memory of the input and output tensor data of the Element-wise operator is continuous (overflowing), and the data placement of each input tensor and output tensor is uniform. When the input and output tensors of the Element-wise operator do not meet the above premise, efficient computing cannot be achieved, which will seriously affect the computing efficiency of the Element-wise operator. For other operators similar to the Element-wise operator, the premise for achieving efficient computing is the same as that of the Element-wise operator. If the memory of the input and output tensor data is not continuous (overflowing), and the data placement of each input and output tensor is uniform, efficient computing cannot be achieved, which seriously affects the computing efficiency of the operator.

[0036] Therefore, before the operator starts the operation (launch), it becomes an important task to preprocess all input tensors and output tensors of the operator to be processed, so that all input tensors and output tensors meet the prerequisites for the above-mentioned efficient calculation. The main purpose of the preprocessing work is to derive the optimal tensor attribute information in the calculation based on the shape, data type and data placement of the input tensor, so as to improve the calculation performance of the operator. The present application provides a set of extremely flexible and efficient tensor processing tools (such as TensorIterator), which can implement preprocessing work such as placement strategy of tensor data involved in the operator according to the calculation characteristics of element wise operators, and give full play to the maximum calculation performance of the operator. The tensor processing tool and the processing method used by it can be applied to more hardware scenarios such as GPU or artificial intelligence processors to improve the calculation performance of the operator.

[0037] The present application provides a tensor data processing method, which selects a target input tensor from all input tensors of an operator to be processed; determines the memory mode category of the operator to be processed according to the memory mode of the target input tensor and the output tensor; determines the target step size according to the memory mode category and the common shape of all input tensors; and adjusts the shape and step size of all input tensors and output tensors of the operator to be processed according to the common shape and the target step size. By determining the memory mode category and the common shape, the target step size is determined as the common step size, and then the shape and step size of all input tensors and output tensors of the operator to be processed are preprocessed based on the common shape and the target step size, so that the memory of all input tensors and output tensors is continuous and the data placement is unified, thereby effectively improving the computational efficiency of the operator to be processed when it is executed, so as to maximize the performance of the operator calculation.

[0038] like Figure 1As shown, the tensor data processing method provided in the present application is suitable for the preprocessing stage before the operator starts operation (launch), and can be executed on hardware such as CPU to obtain input tensors and output tensors with continuous memory and uniform data placement, and then perform the operator operation stage to be processed based on the input tensors and output tensors on the device end such as GPU or artificial intelligence processor.

[0039] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0040] Figure 2 This is a flowchart of a tensor data processing method provided for one embodiment of the present application. The executor of this embodiment is hardware such as a CPU.

[0041] like Figure 2 As shown, the tensor data processing method provided in this embodiment includes the following steps:

[0042] S201. Filter out a target input tensor from all input tensors of the operator to be processed.

[0043] In this embodiment, the operator to be processed can be any operator that meets the following conditions in order to achieve efficient computing: the input tensor and output tensor data memory is continuous (overflowing), and the data placement of each input tensor and output tensor is uniform. The tensor data memory is continuous, that is, the tensor is stored continuously in a continuous memory block without gaps; the uniform placement of data of different tensors means that the shapes and strides of different tensors are the same.

[0044] For example, the operator to be processed is an Element-wise operator, which operates on the corresponding elements between each input tensor. The premise for the Element-wise operator to achieve efficient computing is that the memory of the input tensor and output tensor data of the Element-wise operator is continuous (overflowing), and the data placement of each input tensor and output tensor is unified. In addition, the memory mode of the tensor is also a factor that affects efficient computing. The memory modes of the tensor include: channels_last mode (CL), three-dimensional channels last mode (CL3D), channels first mode (CF), non-overlapping and dense mode (INOAD). Therefore, the input tensor also needs to choose the optimal memory mode.

[0045] However, the input tensor of the operator to be processed may not meet the conditions for efficient calculation, for example, the input tensor may be discontinuous (overflowing) in memory, and the data placement of each input tensor is not uniform (for example, the shape and / or stride of the tensor are inconsistent), and it is also possible that the input tensor of the operator to be processed needs to be adjusted, and the adjustment process can be based on the input tensors of all the input tensors of the operator to be processed that have continuous memory and no overlap. Decisions are made. In this embodiment, scalars in all inputs of the operator to be processed, as well as tensors with gaps or overlaps in memory, can be excluded, thereby screening out target input tensors with continuous memory and no overlap in all input tensors of the operator to be processed. The meaning of overlapping tensors is that if the same address is generated multiple times by traversing the entire dimensional range of the tensor, the tensor is defined as overlapping. Memory continuity means that the tensor is stored continuously in a continuous memory block without gaps, and the memory is discontinuous if there are gaps.

[0046] S202. Determine the memory mode category of the operator to be processed according to the memory modes of the target input tensor and the output tensor.

[0047] The input tensor and output tensor can be two-dimensional matrix, three-dimensional matrix or tensor data of higher dimension, but the input tensor and output tensor are usually stored in the memory in one-dimensional form. This requires that the high-dimensional input tensor and output tensor are expanded into one-dimensional data for storage according to certain rules, and the certain storage rules are called memory mode. In this embodiment, since the memory mode of the tensor is also a factor affecting efficient calculation, the data placement in certain memory modes has a better effect on improving the calculation speed. Therefore, when the memory mode of the input tensor and the output tensor are both adopted in this memory mode, more efficient calculation can be achieved. Therefore, in this embodiment, the memory mode can be classified according to the memory mode of the target input tensor and the output tensor. When the memory mode of the target input tensor and the output tensor belongs to a certain memory mode, it can be determined that the memory mode category corresponding to the memory mode (of course, specific analysis is also required in some cases), different memory mode categories correspond to different tensor data placement adjustment strategies, and finally the tensor data placement method can be adjusted according to the memory mode category.

[0048] Among them, optionally, the memory modes in this embodiment include: channel last mode (channels_last, CL), three-dimensional channel last mode (channels_last_3d, CL3D), channel priority mode (channels_first, CF), non-overlapping and dense mode (is_non_overlapping_and_dense, INOAD).

[0049] Among them, the dimensions of the tensor can be N, H, W, C, D, etc., where N is the batch size (Batch), indicating the number of images in a batch; H is the height (Height), indicating the number of pixels in the vertical dimension of the image; W is the width (Width), indicating the number of pixels in the horizontal dimension of the image; C is the number of channels (Channel), indicating the number of channels in an image; D is the depth (Deep), indicating the number of feature maps in the Z dimension of the image.

[0050] The tensor of the channel-last mode is 4-dimensional, that is, the NHWC format, which saves the image pixel by pixel and is the densest format in memory. The tensor of the three-dimensional channel-last mode is 5-dimensional, that is, the NDHWC format. The channel-first mode can use the channel as the first dimension, and the other dimensions can be any dimension, including but not limited to CHW, CHWN, CDHWN and other formats. The non-overlapping and dense mode is a memory mode that satisfies both the non-overlapping tensor and dense tensor formats. The channel-last mode, the three-dimensional channel-last mode, and the channel-first mode are also included in the non-overlapping and dense mode.

[0051] S203. Determine a target step size according to the memory mode category and a common shape of all the input tensors.

[0052] In this embodiment, in order to ensure that the final data placement of the input tensors is uniform, the shapes of all input tensors need to be unified in the end, so the common shape of the input tensors needs to be determined.

[0053] Among them, the shapes of all input tensors may be the same at the beginning, so the shape of any input tensor can be the common shape of the input tensors; the shapes of all input tensors may be different at the beginning, and the tensor shapes of all input tensors of the operator to be processed can be obtained after broadcasting and determined as the common shape. Among them, broadcasting is the characteristic that when calculating tensors of different shapes, one or more tensors are converted into two tensors of the same shape through implicit conversion, thereby completing the calculation. In this embodiment, the common shape of the input tensor can be obtained from the TensorIterator tool in the Pytorch framework.

[0054] In this embodiment, the target step size can be determined by combining the memory mode category and the common shape, that is, the common step size of all input tensors and output tensors. The step size means the number of elements that need to be skipped in order to obtain the next element along each dimension in the memory. Among them, for the memory mode categories corresponding to the channel last mode, the three-dimensional channel last mode, and the channel priority mode, since the formats of all input tensors and output tensors are the same, the step size calculation function can be called to determine the target step size according to the memory mode category and the common shape of all input tensors; and for the memory mode category corresponding to the non-overlapping and dense mode, the specific analysis can be carried out according to the specific situation to finally determine the target step size.

[0055] S204. Adjust the shapes and step sizes of all input tensors and output tensors of the operator to be processed according to the common shape and the target step size.

[0056] In this embodiment, after determining the common shape and target step size, the shapes and step sizes of all input tensors (including excluded tensors) and output tensors of the operator to be processed can be adjusted according to the common shape and target step size, so that the shapes of all input tensors and output tensors are common shapes, and the step sizes of all input tensors and output tensors are target step sizes. In this way, the data memory of the input tensors and output tensors can be guaranteed to be continuous (overflowed), and the data placement of each input tensor and output tensor is unified.

[0057] The tensor data processing method provided in this embodiment selects the target input tensor from all the input tensors of the operator to be processed; determines the memory mode category of the operator to be processed according to the memory mode of the target input tensor and the output tensor; determines the target step size according to the memory mode category and the common shape of all the input tensors; and adjusts the shape and step size of all the input tensors and output tensors of the operator to be processed according to the common shape and the target step size. By determining the memory mode category and the common shape, the target step size is determined as the common step size, and then the shape and step size of all the input tensors and output tensors of the operator to be processed are preprocessed based on the common shape and the target step size, so that the memory of all the input tensors and output tensors is continuous and the data placement is unified, thereby effectively improving the computational efficiency of the operator to be processed when it is executed, so as to maximize the performance of the operator calculation.

[0058] Based on any of the above embodiments, in this embodiment, the memory mode category of the operator to be processed can be determined according to the memory mode of the target input tensor and the output tensor, and the specific classification is as follows:

[0059] If the memory modes of the target input tensor and the output tensor are both channel-last mode, the memory mode category is determined to be the first category; or

[0060] If the memory modes of the target input tensor and the output tensor are both three-dimensional channel last mode, the memory mode category is determined to be the second category; or

[0061] If the memory modes of the target input tensor and the output tensor are both channel priority mode, the memory mode category is determined to be the third category; or

[0062] If the memory modes of the target input tensor and the output tensor are both non-overlapping and dense modes, and the step sizes of the target input tensor and the output tensor are consistent, then the memory mode category is determined to be the fourth category; or

[0063] If the memory modes of the target input tensor and the output tensor are both non-overlapping and dense modes, and the step sizes of the target input tensor and the output tensor are inconsistent, the memory mode category is determined according to the memory mode of any one of the target input tensor and the output tensor.

[0064] In this embodiment, when the memory modes of the target input tensor and the output tensor are the same, and the memory mode is one of the channel last mode, the three-dimensional channel last mode, and the channel priority mode, the memory mode classification is directly determined.

[0065] When the target input tensor and the output tensor are both non-overlapping and dense mode, if the memory mode step sizes of the target input tensor and the output tensor are consistent, it means that the memory modes of the target input tensor and the output tensor are the same, and the memory mode category is determined to be the fourth category; when the target input tensor and the output tensor are both non-overlapping and dense mode, if the memory mode step sizes of the target input tensor and the output tensor are inconsistent, it means that the memory modes of the target input tensor and the output tensor are different. At this time, one of the target input tensors and the output tensor can be selected as a reference, and the memory mode of other tensors in the target input tensor and the output tensor can be aligned to the memory mode of the reference tensor through memory mode conversion. Specifically, if the memory mode of any tensor is the channel last mode, the memory mode category is determined to be the first category; or, if the memory mode of any tensor is the three-dimensional channel last mode, the memory mode category is determined to be the second category; or, if the memory mode of any tensor is the channel priority mode, the memory mode category is determined to be the third category; or, if the memory mode of any tensor is other modes, the memory mode category is determined to be unclassified.

[0066] For example, taking the target input tensor and the first tensor in the output tensor as a reference (as an example only, not limited to the first tensor), if the memory mode of the first tensor is the channel-last mode, the memory mode category can be determined as the first category; if the memory mode of the first tensor is the three-dimensional channel-last mode, the memory mode category can be determined as the second category; if the memory mode of the first tensor is the channel-first mode, the memory mode category can be determined as the third category; if the memory mode of the first tensor is other mode, the memory mode category is determined to be unclassified.

[0067] On the basis of the above embodiments, due to the characteristics of the operation unit, in some special cases, the memory modes of the target input tensor and the output tensor may belong to different memory modes at the same time. Therefore, there is a situation where the memory mode categories of the operator to be processed belong to different categories at the same time. If it is determined that the memory mode categories of the operator to be processed belong to different categories at the same time, a memory mode category is determined from the different categories according to the preset priority, so that the optimal memory mode category can be adopted to better improve the computing performance of the operator. Among them, optionally, the preset priorities are from high to low: first category = second category > third category > fourth category. This priority order takes into account that the conv (convolution) and matrix multiplication operators of the MLU operation unit have better computing performance when the last mode of the channel is continuous. Of course, the preset priority can also be set according to actual needs.

[0068] Based on the above embodiment, if the target input tensor and the output tensor do not satisfy any of the above conditions, the memory mode category is determined to be unclassified, that is, the memory mode category is not determined and does not belong to the above first to fourth categories.

[0069] Based on the above embodiments, there are some special scenarios when determining the memory mode category of the operator to be processed. It is necessary to modify and update the memory mode category for the special scenarios. Figure 3 As shown, it may include:

[0070] S301, determining a common dimension of all input tensors and output tensors of the operator to be processed, wherein the common dimension is a maximum dimension of all input tensors and output tensors of the operator to be processed;

[0071] S302: Modify the memory mode category according to the common dimension.

[0072] The common dimension is the maximum dimension of all input tensors and output tensors of the operator to be processed, which can be obtained through the API interface provided by the tensor processing tool. Since the excluded tensors will also affect the common dimension, especially the differences between the excluded tensors and the target input tensors and output tensors, it may lead to the scenario where the common dimension and the memory mode category do not match, so the memory mode category can be corrected.

[0073] For example, if the memory mode category is the first category, but the common dimension matches the second category, that is, the common dimension is 5 dimensions, then the memory mode category is corrected to the second category. For example, if the excluded tensor is 5 dimensions, and the target input tensor and the output tensor satisfy the final mode of the channel (that is, 4 dimensions), the tensors need to be aligned to 5 dimensions. Since the final mode of the three-dimensional channel is 5 dimensions, the memory mode category can be corrected from the first category to the second category.

[0074] For another example, if the memory mode category is the first category or the second category, but the common dimension does not match the first category and the second category, that is, the common dimension is not 4-dimensional or 5-dimensional, then the memory mode category is corrected to the third category. For example, if the excluded tensor is not 4-dimensional or 5-dimensional, and the memory mode category is the first category (4-dimensional) or the second category (5-dimensional), since the channel priority mode can be of any dimension, the memory mode category can be corrected to the third category.

[0075] In addition, if there are discontinuous tensors among all the input tensors of the operator to be processed, and the memory mode category is the fourth category, then the memory mode category can be updated to unclassified, that is, it does not belong to the above first to fourth categories.

[0076] Based on any of the above embodiments, determining the target step size according to the memory mode category and the common shape of all input tensors in S203 may specifically include:

[0077] If the memory mode category is the first category, the second category, or the third category, calling a step length calculation function, and determining the target step length according to the memory mode category and the common shape through the step length calculation function; or

[0078] If the memory mode category is the fourth category, the stride of any one of the target input tensor and the output tensor is determined as the target stride.

[0079] In this embodiment, for memory mode categories of the first category, the second category, or the third category, the step length calculation function in the deep learning framework (such as Pytorch) can be directly called through the API, and the memory mode category and the common shape are input into the step length calculation function. The target step length is calculated according to the memory mode category and the common shape. Different memory mode categories and different common shapes have different calculated target step lengths, and the specific calculation process will not be repeated here.

[0080] For the memory mode category of the fourth category, since the stride of the target input tensor and the output tensor of the fourth category are the same, the target stride can be the stride of any of the target input tensor and the output tensor.

[0081] In addition, if the memory mode category is not determined, you can first set the memory mode category to adjust the memory modes of all input tensors and output tensors to the memory modes corresponding to the set memory mode category to achieve the unification of the memory mode, and then determine the target step size based on the set memory mode category and the common shape.

[0082] When setting the memory mode category, the memory mode category can be set according to the preset priority, wherein optionally, the preset priority can be from high to low as follows: first category = second category > third category > fourth category, which is the same as the preset priority in the above embodiment, thereby unifying all input tensors and output tensors into the optimal memory mode, thereby better improving the computing performance of the operator.

[0083] Figure 4 A schematic diagram of the structure of a tensor data processing device provided in one embodiment of the present application is shown in FIG. Figure 4 As shown, the tensor data processing device provided in this embodiment is located in the device end, and the tensor data processing device 40 provided in this embodiment includes: a screening module 41, a classification module 42, a decision module 43, and an adjustment module 44.

[0084] The screening module 41 is used to screen out the target input tensor from all input tensors of the operator to be processed;

[0085] A classification module 42, configured to determine the memory mode category of the operator to be processed according to the memory modes of the target input tensor and the output tensor;

[0086] A decision module 43, configured to determine a target step size according to the memory mode category and a common shape of all input tensors;

[0087] The adjustment module 44 is used to adjust the shapes and step sizes of all input tensors and output tensors of the operator to be processed according to the common shape and the target step size.

[0088] In one or more embodiments of the present disclosure, the memory mode category includes: a first category, a second category, a third category, and a fourth category; when the classification module 42 determines the memory mode category of the operator to be processed according to the memory mode of the target input tensor and the output tensor, it is used to:

[0089] If the memory modes of the target input tensor and the output tensor are both channel-last modes, determining that the memory mode category is the first category; or

[0090] If the memory modes of the target input tensor and the output tensor are both three-dimensional channel final modes, determining that the memory mode category is the second category; or

[0091] If the memory modes of the target input tensor and the output tensor are both channel priority modes, determining that the memory mode category is the third category; or

[0092] If the memory modes of the target input tensor and the output tensor are both non-overlapping and dense modes, and the step sizes of the target input tensor and the output tensor are consistent, then the memory mode category is determined to be the fourth category; or

[0093] If the memory modes of the target input tensor and the output tensor are both non-overlapping and dense modes, and the step sizes of the target input tensor and the output tensor are inconsistent, the memory mode category is determined according to the memory mode of any one of the target input tensor and the output tensor.

[0094] In one or more embodiments of the present disclosure, when determining the memory mode category according to the memory mode of any one of the target input tensor and the output tensor, the classification module 42 is configured to:

[0095] If the memory mode of any tensor is the channel-last mode, determining that the memory mode category is the first category; or

[0096] If the memory mode of any tensor is the last mode of the three-dimensional channel, determining that the memory mode category is the second category; or

[0097] If the memory mode of any tensor is the channel priority mode, determining that the memory mode category is the third category; or

[0098] If the memory mode of any of the tensors is other modes, the memory mode category is determined to be unclassified.

[0099] In one or more embodiments of the present disclosure, the classification module 42 is further configured to:

[0100] If the memory mode categories of the operator to be processed include at least two different categories, a memory mode category is determined from the different categories according to a preset priority.

[0101] In one or more embodiments of the present disclosure, the preset priorities are in the following order from high to low: first category=second category>third category>fourth category.

[0102] In one or more embodiments of the present disclosure, the classification module 42 is further configured to:

[0103] Determine a common dimension of all input tensors and output tensors of the operator to be processed, wherein the common dimension is a maximum dimension of all input tensors and output tensors of the operator to be processed;

[0104] The memory mode category is modified according to the common dimension.

[0105] In one or more embodiments of the present disclosure, when the classification module 42 modifies the memory mode category according to the common dimension, it is configured to:

[0106] If the memory mode category is the first category, but the common dimension matches the second category, then correcting the memory mode category to the second category; or

[0107] If the memory mode category is the first category or the second category, but the common dimension does not match the first category and the second category, the memory mode category is corrected to the third category.

[0108] In one or more embodiments of the present disclosure, the classification module 42 is further configured to:

[0109] If there are discontinuous tensors among all the input tensors of the operator to be processed, and the memory mode category is the fourth category, the memory mode category is corrected to unclassified.

[0110] In one or more embodiments of the present disclosure, when determining the target step size according to the memory mode category and the common shape of all input tensors, the decision module 43 is used to:

[0111] If the memory mode category is the first category, the second category, or the third category, calling a step length calculation function, and determining the target step length according to the memory mode category and the common shape through the step length calculation function; or

[0112] If the memory mode category is the fourth category, the stride of any one of the target input tensor and the output tensor is determined as the target stride.

[0113] In one or more embodiments of the present disclosure, the decision module 43 is further configured to:

[0114] If the memory mode category is not determined, the memory mode category is set, and a target stride is determined according to the set memory mode category and the common shapes of all the input tensors.

[0115] In one or more embodiments of the present disclosure, when setting the memory mode category, the decision module 43 is used to:

[0116] The memory mode category is set according to a preset priority.

[0117] In one or more embodiments of the present disclosure, when determining the common shape of the input tensor according to all the input tensors of the operator to be processed, the shape determination module 43 is used to:

[0118] The tensor shapes of all input tensors of the operator to be processed after broadcasting are obtained and determined as the common shape.

[0119] In one or more embodiments of the present disclosure, the operator to be processed is an Element-wise operator.

[0120] The tensor data processing device provided in this embodiment can execute Figure 2 and Figure 3 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0121] Figure 5 A schematic diagram of the structure of a tensor data processing device provided in another embodiment of the present application is shown in FIG. Figure 5 As shown, the tensor data processing device 50 provided in the embodiment of the present application includes: at least one processor 51 and a memory 52;

[0122] The memory 52 stores computer executable instructions;

[0123] At least one processor 51 executes the computer-executable instructions stored in the memory 52, so that at least one processor performs Figure 2 and Figure 3 A tensor data processing method provided by any one of the embodiments.

[0124] In a possible implementation, a computer-readable storage medium is also disclosed. The computer-readable storage medium stores a computer program. When the computer program is executed by at least one processor, Figure 2 and Figure 3 A tensor data processing method provided by any one of the embodiments.

[0125] In a possible implementation manner, a board card is also disclosed, and the board card may be a board card at the device end. Figure 6 FIG. 6 is a schematic diagram showing the structure of a board 60 according to an embodiment of the present application. Figure 6 As shown, the board 60 includes a chip 601, which is a system-on-chip (SoC), or system on chip, integrated with one or more combined processing devices, which are artificial intelligence computing units that support various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in the fields of computer vision, speech, natural language processing, data mining, etc. In particular, deep learning technology is widely used in the field of cloud intelligence. A notable feature of cloud intelligence applications is the large amount of input data, which places high demands on the storage capacity and computing power of the platform. The board 60 of this embodiment is suitable for cloud intelligence applications and has huge off-chip storage, on-chip storage, and powerful computing power.

[0126] The chip 601 is connected to an external device 603 via an external interface device 602. The external device 603 is, for example, a server, a computer, a camera, a display, a mouse, a keyboard, a network card, or a wifi interface. The data to be processed can be transmitted from the external device 603 to the chip 601 via the external interface device 602. The calculation result of the chip 601 can be transmitted back to the external device 603 via the external interface device 602. According to different application scenarios, the external interface device 602 can have different interface forms, such as a PCIe interface.

[0127] The board 60 also includes a storage device 604 for storing data, which includes one or more storage units 605. The storage device 604 is connected to the control device 606 and the chip 601 through a bus and transmits data. The control device 606 in the board 60 is configured to regulate the state of the chip 601. To this end, in an application scenario, the control device 606 may include a single-chip microcomputer (Micro Controller Unit, MCU).

[0128] In a possible implementation, a combined processing device is also provided. Figure 7 6 is a block diagram showing the combined processing device in the chip 601 of this embodiment. Figure 7 As shown in , the combined processing device 70 includes a computing device 701 , an interface device 702 , a processing device 703 and a storage device 704 .

[0129] The computing device 701 is configured to perform user-specified operations, and is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 703 through the interface device 702 to jointly complete the user-specified operations.

[0130] The interface device 702 is used to transmit data and control instructions between the computing device 701 and the processing device 703. For example, the computing device 701 can obtain input data from the processing device 703 via the interface device 702 and write it into the storage device on the computing device 701 chip. Further, the computing device 701 can obtain control instructions from the processing device 703 via the interface device 702 and write them into the control cache on the computing device 701 chip. Alternatively or optionally, the interface device 702 can also read data in the storage device of the computing device 701 and transmit it to the processing device 703.

[0131] The processing device 703, as a general processing device, performs basic controls including but not limited to data handling, starting and / or stopping the computing device 701. Depending on the implementation, the processing device 703 can be a central processing unit (CPU), a graphics processing unit (GPU) or one or more types of processors in other general and / or special processors, these processors include but are not limited to digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As before, only with respect to the computing device 701 of the present application, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 701 and the processing device 703 are integrated and considered together, the two are regarded as forming a heterogeneous multi-core structure.

[0132] The storage device 704 is used to store data to be processed, which may be DRAM 704 , which is a DDR memory, and is usually 16G or larger in size, and is used to store data of the computing device 701 and / or the processing device 703 .

[0133] Figure 8 The single-core computing device 801 is used to process input data such as computer vision, speech, natural language, and data mining. The single-core computing device 801 includes three modules: a control module 81 , a computing module 82 , and a storage module 83 .

[0134] The control module 81 is used to coordinate and control the operation of the operation module 82 and the storage module 83 to complete the deep learning task, and includes an instruction fetch unit (IFU) 811 and an instruction decode unit (IDU) 812. The instruction fetch unit 811 is used to obtain instructions from the processing device 1203, and the instruction decode unit 812 decodes the obtained instructions and sends the decoding results to the operation module 82 and the storage module 83 as control information.

[0135] The operation module 82 includes a vector operation unit 821 and a matrix operation unit 822. The vector operation unit 821 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformation; the matrix operation unit 822 is responsible for the core calculation of the deep learning algorithm, namely matrix multiplication and convolution.

[0136] The storage module 83 is used to store or transfer relevant data, including a neuron RAM (NRAM) 831, a parameter RAM (WRAM) 832, and a direct memory access module (DMA) 833. NRAM 831 is used to store input neurons, output neurons, and intermediate results after calculation; WRAM 832 is used to store the convolution kernel of the deep learning network, that is, the weight; DMA 833 is connected to DRAM 704 through bus 84, and is responsible for data transfer between the single-core computing device 801 and DRAM 704.

[0137] Fig. 9 The schematic diagram of the internal structure of the computing device 701 is a multi-core computing device. The multi-core computing device 901 adopts a hierarchical structure design. The multi-core computing device 901 is a system on a chip, which includes at least one cluster, and each cluster includes multiple processor cores. In other words, the multi-core computing device 901 is composed of a hierarchy of system on a chip-cluster-processor core.

[0138] At the system-on-chip level, Fig. 9 As shown, the multi-core computing device 901 includes an external storage controller 901 , a peripheral communication module 902 , an on-chip interconnect module 903 , a synchronization module 904 and multiple clusters 905 .

[0139] There may be multiple external storage controllers 901, and two are shown in the figure as an example, which are used to respond to access requests issued by the processor core and access external storage devices, such as Figure 7DRAM 704 in the chip, thereby reading data from outside the chip or writing data. The peripheral communication module 902 is used to receive control signals from the processing device 703 through the interface device 702 to start the computing device 701 to perform tasks. The on-chip interconnect module 903 connects the external storage controller 901, the peripheral communication module 902 and multiple clusters 905 to transmit data and control signals between various modules. The synchronization module 904 is a global synchronization barrier controller (GBC) used to coordinate the work progress of each cluster and ensure information synchronization. Multiple clusters 905 are the computing cores of the multi-core computing device 901. Four are shown as an example in the figure. The four clusters 905 form the following Figure 1 With the development of hardware, the multi-core computing device 901 of the present application may also include 8, 16, 64, or even more clusters 905. The cluster 905 is used to efficiently execute the deep learning algorithm.

[0140] At the cluster level, Fig. 9 As shown, each cluster 905 includes multiple processor cores (IPU cores) 906 and a memory core (MEM core) 907. For example, each cluster 905 includes 4 processor cores and 1 memory, and the memory can be DRAM 704. Each processor core is equivalent to Figure 1 An operation unit in Figure 1 A storage unit in.

[0141] The figure shows four processor cores 906 as an example, and the present application does not limit the number of processor cores 906. Its internal architecture is as follows Fig.10 Each processor core 906 is similar to Figure 8The single-core computing device 801 also includes three modules: a control module 1001, an operation module 1002 and a storage module 1003. The functions and structures of the control module 1001, the operation module 1002 and the storage module 1003 are roughly the same as those of the control module 81, the operation module 82 and the storage module 83. The control module 1001 includes an instruction fetch unit 10011 and an instruction decoding unit 10012. The operation module 1002 includes a vector operation unit 10021 and a matrix operation unit 10022. No further details are given. It should be noted that the storage module 1003 includes an input / output direct memory access module (input / output direct memory access, IODMA) 10033 and a movedirect memory access module (movedirect memory access, MVDMA) 10034. IODMA 10033 controls memory access between NRAM 10031 / WRAM 10032 and DRAM 704 via broadcast bus 909 ; MVDMA 10034 controls memory access between NRAM 10031 / WRAM 10032 and storage unit (SRAM) 908 .

[0142] Back to Figure 7 The storage core 907 is mainly used for storage and communication, that is, to store shared data or intermediate results between the processor cores 906, and to perform communication between the cluster 905 and the DRAM 704, communication between the clusters 905, and communication between the processor cores 906. In other embodiments, the storage core 907 has the ability of scalar operation and is used to perform scalar operation.

[0143] The storage core 907 includes an SRAM 908, a broadcast bus 909, a cluster direct memory access module (CDMA) 910, and a global direct memory access module (GDMA) 911. The SRAM 908 plays the role of a high-performance data transfer station. The data reused between different processor cores 906 in the same cluster 905 does not need to be obtained from the DRAM 704 by each processor core 906, but is transferred between the processor cores 906 via the SRAM 908. The storage core 907 only needs to quickly distribute the reused data from the SRAM 908 to multiple processor cores 906, so as to improve the efficiency of inter-core communication and greatly reduce on-chip and off-chip input / output access.

[0144] The broadcast bus 909, CDMA 910 and GDMA 911 are respectively used to perform communication between processor cores 906, communication between clusters 905 and data transmission between clusters 905 and DRAM 704. They will be described below.

[0145] The broadcast bus 909 is used to complete high-speed communication between the processor cores 906 in the cluster 905. The broadcast bus 909 of this embodiment supports inter-core communication modes including unicast, multicast and broadcast. Unicast refers to point-to-point (e.g., single processor core to single processor core) data transmission, multicast is a communication mode of transmitting a copy of data from the SRAM 908 to several specific processor cores 906, and broadcast is a communication mode of transmitting a copy of data from the SRAM 908 to all processor cores 906, which is a special case of multicast.

[0146] CDMA 910 is used to control memory access to SRAM 908 between different clusters 905 within the same computing device 701 .

[0147] GDMA 911 cooperates with external memory controller 901 to control the access of SRAM 908 of cluster 905 to DRAM 704, or reads data from DRAM 704 to SRAM 908. As can be seen from the above, the communication between DRAM 704 and NRAM 10031 or WRAM 10032 can be realized through two channels. The first channel is to directly connect DRAM 704 and NRAM 10031 or WRAM 10032 through IODAM 10033; the second channel is to first transmit data between DRAM 704 and SRAM 908 through GDMA 911, and then transmit data between SRAM 908 and NRAM 10031 or WRAM 10032 through MVDMA 10034. Although it seems that the second channel requires more components to participate and the data flow is longer, in fact, in some embodiments, the bandwidth of the second channel is much greater than the first channel, so the communication between DRAM 704 and NRAM 10031 or WRAM 10032 may be more efficient through the second channel. The embodiments of the present application can select the data transmission channel according to their own hardware conditions.

[0148] In other embodiments, the functions of GDMA 911 and IODMA 10033 can be integrated into the same component. For the convenience of description, this application regards GDMA 911 and IODMA 10033 as different components. For those skilled in the art, as long as the functions implemented and the technical effects achieved are similar to those of this application, they belong to the protection scope of this application. Furthermore, the functions of GDMA 911, IODMA 10033, CDMA 910, and MVDMA 10034 can also be implemented by the same component.

[0149] The foregoing may be better understood in view of the following clauses:

[0150] Clause 1. A method for processing tensor data, comprising:

[0151] Filter the target input tensor from all input tensors of the operator to be processed;

[0152] Determine the memory mode category of the operator to be processed according to the memory modes of the target input tensor and the output tensor;

[0153] Determining a target stride according to the memory mode category and a common shape of all the input tensors;

[0154] According to the common shape and the target step size, the shapes and step sizes of all input tensors and output tensors of the operator to be processed are adjusted.

[0155] Clause 2. The method according to clause 1, wherein the memory mode categories include: a first category, a second category, a third category, and a fourth category;

[0156] The determining the memory mode category of the operator to be processed according to the memory mode of the target input tensor and the output tensor includes:

[0157] If the memory modes of the target input tensor and the output tensor are both channel-last modes, determining that the memory mode category is the first category; or

[0158] If the memory modes of the target input tensor and the output tensor are both three-dimensional channel final modes, determining that the memory mode category is the second category; or

[0159] If the memory modes of the target input tensor and the output tensor are both channel priority modes, determining that the memory mode category is the third category; or

[0160] If the memory modes of the target input tensor and the output tensor are both non-overlapping and dense modes, and the step sizes of the target input tensor and the output tensor are consistent, then the memory mode category is determined to be the fourth category; or

[0161] If the memory modes of the target input tensor and the output tensor are both non-overlapping and dense modes, and the step sizes of the target input tensor and the output tensor are inconsistent, the memory mode category is determined according to the memory mode of any one of the target input tensor and the output tensor.

[0162] Clause 3. The method according to clause 2, wherein determining the memory mode category according to the memory mode category of any one of the target input tensor and the output tensor comprises:

[0163] If the memory mode of any tensor is the channel-last mode, determining that the memory mode category is the first category; or

[0164] If the memory mode of any tensor is the last mode of the three-dimensional channel, determining that the memory mode category is the second category; or

[0165] If the memory mode of any tensor is a channel priority mode, determining that the memory mode category is the third category; or

[0166] If the memory mode of any of the tensors is other modes, the memory mode category is determined to be unclassified.

[0167] Clause 4. The method according to any one of clauses 1 to 3, further comprising:

[0168] If the memory mode categories of the operator to be processed include at least two different categories, a memory mode category is determined from the different categories according to a preset priority.

[0169] Clause 5. According to the method described in Clause 4, the preset priorities are from high to low: first category = second category > third category > fourth category.

[0170] Clause 6. The method according to any one of clauses 1 to 3, further comprising:

[0171] Determine a common dimension of all input tensors and output tensors of the operator to be processed, wherein the common dimension is a maximum dimension of all input tensors and output tensors of the operator to be processed;

[0172] The memory mode category is modified according to the common dimension.

[0173] Clause 7. The method of clause 6, wherein modifying the memory mode category according to the common dimension comprises:

[0174] If the memory mode category is the first category, but the common dimension matches the second category, then correcting the memory mode category to the second category; or

[0175] If the memory mode category is the first category or the second category, but the common dimension does not match the first category and the second category, the memory mode category is corrected to the third category.

[0176] Clause 8. The method according to clause 2, further comprising:

[0177] If there are discontinuous tensors among all the input tensors of the operator to be processed, and the memory mode category is the fourth category, the memory mode category is corrected to unclassified.

[0178] Clause 9. The method according to any one of clauses 1 to 8, wherein determining the target stride according to the memory mode category and the common shape of all input tensors comprises:

[0179] If the memory mode category is the first category, the second category, or the third category, calling a step length calculation function, and determining the target step length according to the memory mode category and the common shape through the step length calculation function; or

[0180] If the memory mode category is the fourth category, the stride of any one of the target input tensor and the output tensor is determined as the target stride.

[0181] Clause 10. The method according to clause 9, further comprising:

[0182] If the memory mode category is not determined, the memory mode category is set, and a target stride is determined according to the set memory mode category and the common shapes of all the input tensors.

[0183] Clause 11. The method according to clause 10, wherein setting the memory mode category comprises:

[0184] The memory mode category is set according to a preset priority.

[0185] Clause 12. A method according to any one of clauses 1 to 8, wherein determining the common shape of the input tensors according to all input tensors of the operator to be processed comprises:

[0186] The tensor shapes of all input tensors of the operator to be processed after broadcasting are obtained and determined as the common shape.

[0187] Clause 13. According to the method described in any one of clauses 1 to 8, the operator to be processed is an element-wise operator.

[0188] Clause 14. A tensor data processing apparatus comprising:

[0189] A filtering module is used to filter out the target input tensor from all input tensors of the operator to be processed;

[0190] A classification module, used to determine the memory mode category of the operator to be processed according to the memory mode of the target input tensor and the output tensor;

[0191] A decision module, configured to determine a target step size according to the memory mode category and a common shape of all the input tensors;

[0192] An adjustment module is used to adjust the shapes and step sizes of all input tensors and output tensors of the operator to be processed according to the common shape and the target step size.

[0193] Clause 15. A tensor data processing apparatus comprising: at least one processor and a memory;

[0194] The memory stores computer-executable instructions;

[0195] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method as described in any one of clauses 1-13.

[0196] Clause 16. A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by at least one processor, implements the method as described in any one of Clauses 1-13.

[0197] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0198] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0199] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0200] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0201] If the integrated unit / module is implemented in the form of hardware, the hardware can be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Unless otherwise specified, the artificial intelligence processor can be any appropriate hardware processor, such as CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any appropriate magnetic storage medium or magneto-optical storage medium, such as resistive random access memory RRAM (Resistive Random Access Memory), dynamic random access memory DRAM (Dynamic Random Access Memory), static random access memory SRAM (Static Random-Access Memory), enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), high-bandwidth memory HBM (High-Bandwidth Memory), hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0202] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0203] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A tensor data processing method, characterized in that: include: Filter the target input tensor from all input tensors of the operator to be processed; Determine the memory mode category of the operator to be processed according to the memory modes of the target input tensor and the output tensor; Determining a target stride according to the memory mode category and a common shape of all the input tensors; According to the common shape and the target step size, the shapes and step sizes of all input tensors and output tensors of the operator to be processed are adjusted.

2. The method according to claim 1, characterized in that: The memory mode categories include: a first category, a second category, a third category, and a fourth category; The determining the memory mode category of the operator to be processed according to the memory mode of the target input tensor and the output tensor includes: If the memory modes of the target input tensor and the output tensor are both channel-last modes, determining that the memory mode category is the first category; or If the memory modes of the target input tensor and the output tensor are both three-dimensional channel final modes, determining that the memory mode category is the second category; or If the memory modes of the target input tensor and the output tensor are both channel priority modes, determining that the memory mode category is the third category; or If the memory modes of the target input tensor and the output tensor are both non-overlapping and dense modes, and the step sizes of the target input tensor and the output tensor are consistent, then the memory mode category is determined to be the fourth category; or If the memory modes of the target input tensor and the output tensor are both non-overlapping and dense modes, and the step sizes of the target input tensor and the output tensor are inconsistent, the memory mode category is determined according to the memory mode category of any one of the target input tensor and the output tensor.

3. The method according to claim 2, characterized in that The determining the memory mode category according to the memory mode category of any one of the target input tensor and the output tensor comprises: If the memory mode of any tensor is the channel-last mode, determining that the memory mode category is the first category; or If the memory mode of any tensor is the last mode of the three-dimensional channel, determining that the memory mode category is the second category; or If the memory mode of any tensor is a channel priority mode, determining that the memory mode category is the third category; or If the memory mode of any of the tensors is other modes, the memory mode category is determined to be unclassified.

4. The method according to any one of claims 1 to 3, characterized in that: Also includes: If the memory mode categories of the operator to be processed include at least two different categories, a memory mode category is determined from the different categories according to a preset priority.

5. The method according to claim 4, characterized in that The preset priorities are as follows from high to low: first category=second category>third category>fourth category.

6. The method according to any one of claims 1 to 3, characterized in that: Also includes: Determine a common dimension of all input tensors and output tensors of the operator to be processed, wherein the common dimension is a maximum dimension of all input tensors and output tensors of the operator to be processed; The memory mode category is modified according to the common dimension.

7. The method according to claim 6, characterized in that The modifying of the memory mode category according to the common dimension includes: If the memory mode category is the first category, but the common dimension matches the second category, then correcting the memory mode category to the second category; or If the memory mode category is the first category or the second category, but the common dimension does not match the first category and the second category, the memory mode category is corrected to the third category.

8. The method according to claim 2, characterized in that: Also includes: If there are discontinuous tensors among all the input tensors of the operator to be processed, and the memory mode category is the fourth category, the memory mode category is corrected to unclassified.

9. The method according to any one of claims 1 to 8, characterized in that: The determining the target step size according to the memory mode category and the common shapes of all the input tensors comprises: If the memory mode category is the first category, the second category, or the third category, calling a step length calculation function, and determining the target step length according to the memory mode category and the common shape through the step length calculation function; or If the memory mode category is the fourth category, the stride of any one of the target input tensor and the output tensor is determined as the target stride.

10. The method according to claim 9, characterized in that Also includes: If the memory mode category is not determined, the memory mode category is set, and a target stride is determined according to the set memory mode category and the common shapes of all the input tensors.

11. The method according to claim 10, characterized in that The step of setting the memory mode category includes: The memory mode category is set according to a preset priority.

12. The method according to any one of claims 1 to 8, characterized in that: The determining the common shape of the input tensor according to all the input tensors of the operator to be processed comprises: The tensor shapes of all input tensors of the operator to be processed after broadcasting are obtained and determined as the common shape.

13. The method according to any one of claims 1 to 8, characterized in that: The operator to be processed is an Element-wise operator.

14. A tensor data processing device, characterized in that: include: A filtering module is used to filter out the target input tensor from all input tensors of the operator to be processed; A classification module, used to determine the memory mode category of the operator to be processed according to the memory mode of the target input tensor and the output tensor; A decision module, configured to determine a target step size according to the memory mode category and a common shape of all the input tensors; An adjustment module is used to adjust the shapes and step sizes of all input tensors and output tensors of the operator to be processed according to the common shape and the target step size.

15. A tensor data processing device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the method according to any one of claims 1 to 13 is implemented.

Images