Tensor data processing method and device and storage medium

Abstract

The embodiment of the invention provides a tensor data processing method and device and a storage medium, and aims to ensure that a to-be-processed operator can support a target public data type and avoid performance loss caused by introduction of unnecessary data type conversion so as to improve the operation efficiency of the operator.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular to a tensor data processing method, device and storage medium. Background Art

[0002] When using processor hardware (such as a graphics processor GPU or an artificial intelligence processor, etc.) to perform operations, various operators are usually used to perform tensor data operations. Specifically, each operator can be encapsulated into a computing kernel. During the operation process, the computing kernel can be loaded onto the processor hardware through a driver, and the processor hardware can execute the computing kernel to implement the corresponding operation.

[0003] In some scenarios, operators require input tensors to use a unified data type to perform operations, which makes data type conversion a necessary processing operation. Introducing too many data type conversion processes will bring performance losses and seriously affect the computing efficiency of the operator. Summary of the invention

[0004] The embodiments of the present application provide a tensor data processing method, device and storage medium for reasonably converting tensor data types and improving the computing efficiency of operators.

[0005] In a first aspect, an embodiment of the present application provides a tensor data processing method, including:

[0006] Get the initial and target common data types of all input tensors and output tensors of the operator to be processed;

[0007] For any target tensor among all input tensors and output tensors, determine whether the target tensor needs to be converted in data type based on the initial data type of the target tensor and the target common data type, and call the data type conversion operator to convert the data type of the target tensor when it is determined that data type conversion is required.

[0008] In a second aspect, an embodiment of the present application provides a tensor data processing device, including:

[0009] A common data type determination module, used to obtain the initial common data type and target common data type of all input tensors and output tensors of the operator to be processed;

[0010] A data processing module is used to determine whether a target tensor among all input tensors and output tensors needs to be converted into a data type according to the initial data type of the target tensor and the target common data type, and to call a data type conversion operator to convert the data type of the target tensor when it is determined that data type conversion is required.

[0011] In a third aspect, an embodiment of the present application provides a tensor data processing device, including: at least one processor and a memory;

[0012] The memory stores computer-executable instructions;

[0013] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to the first aspect.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the method described in the first aspect is implemented.

[0015] The tensor data processing method, device and storage medium provided in the embodiments of the present application obtain the initial common data type and target common data type of all input tensors and output tensors of the operator to be processed; for any target tensor in the input tensor and output tensor, determine whether the target tensor needs to be converted according to the initial data type and target common data type of the target tensor, and call the data type conversion operator to convert the data type of the target tensor when it is determined that the data type conversion is required. By determining the most suitable target common data type, it is ensured that the operator to be processed can support the target common data type, and it is determined whether each input tensor and output tensor needs to be converted, thereby avoiding the performance loss caused by the introduction of unnecessary data type conversion, so as to improve the operation efficiency of the operator. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0017] Figure 1 A schematic diagram of a processing process of an operator to be processed provided in one embodiment of the present application;

[0018] Figure 2 A flowchart of a tensor data processing method provided for one embodiment of the present application;

[0019] Figure 3 A flowchart of a tensor data processing method provided by another embodiment of the present application;

[0020] Figure 4 A schematic diagram of the structure of a tensor data processing device provided in one embodiment of the present application;

[0021] Figure 5 A schematic diagram of the structure of a tensor data processing device provided in another embodiment of the present application;

[0022] Figure 6 is a structural diagram showing a board according to an embodiment of the present application;

[0023] Figure 7 is a structural diagram showing a combined processing device according to an embodiment of the present application;

[0024] Figure 8 is a schematic diagram showing the internal structure of a single-core computing device according to an embodiment of the present application;

[0025] Fig. 9 is a schematic diagram showing the internal structure of a multi-core computing device according to an embodiment of the present application;

[0026] Fig.10 Schematic diagram showing the internal structure of a processor core according to an embodiment of the present application.

[0027] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0028] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0029] In order to clearly understand the technical solution of the present application, the solution of the prior art is first introduced in detail.

[0030] In some scenarios, operators need to input tensors of a uniform data type to perform operations. However, the data type of the input tensors is not necessarily uniform. In the prior art, each tensor is converted to a specified data type.

[0031] For example, when a hardware device does not support a certain data type and a user needs to use that data type, data type conversion is required.

[0032] In addition, for out or replace interfaces, the data type of the user input tensor is often uncontrollable, which inevitably introduces data type conversion operations. In this case, the data type conversion operator (cast operator) is usually called uniformly to perform data type conversion on each tensor to unify the data type of all tensors to the specified data type. However, the specified data type may not be appropriate, and the data type conversion process for all tensors will result in performance loss, seriously affecting the operation efficiency of the operator.

[0033] In order to solve the above technical problems, the present application provides a tensor data processing method, which obtains the initial common data type and target common data type of all input tensors and output tensors of the operator to be processed; for any target tensor in the input tensor and output tensor, according to the initial data type and target common data type of the target tensor, it is judged whether the target tensor needs to be converted into a data type, and when it is determined that the data type conversion is required, the data type conversion operator is called to convert the data type of the target tensor. By determining the most suitable target common data type, it is ensured that the operator to be processed can support the target common data type, and it is judged whether each input tensor and output tensor needs to be converted into a data type, thereby avoiding the performance loss caused by the introduction of unnecessary data type conversion, so as to improve the operation efficiency of the operator.

[0034] like Figure 1 As shown, the tensor data processing method provided in the present application is suitable for the preprocessing stage before the operator starts operation (launch), which can be executed on hardware such as CPU, convert the data type of the output tensor, and then perform the operator operation stage to be processed on the device end such as GPU or artificial intelligence processor.

[0035] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0036] Figure 2 This is a flowchart of a tensor data processing method provided for one embodiment of the present application. The executor of this embodiment is hardware such as a CPU.

[0037] like Figure 2 As shown, the tensor data processing method provided in this embodiment includes the following steps:

[0038] S201. Obtain the initial common data type and target common data type of all input tensors and output tensors of the operator to be processed.

[0039] In this embodiment, all input tensors (input tensor) and output tensors (outputtensor) of the operator to be processed may include different data types (dtype). In order to avoid errors in the device-side computing unit due to different data types, they usually need to be converted to the same data type, that is, all input tensors and output tensors are aligned to the common data type (common_dtype). The operator to be processed can be an Element-wise operator, which operates on the corresponding elements between each input tensor. Of course, the operator to be processed may also be other operators, which are not given one by one here.

[0040] Optionally, since each data type can represent a different number of data bits, the more data bits, the higher the precision. Therefore, this application can use the data type with the highest precision among the data types of all input tensors and output tensors as a common data type, recorded as the initial common data type. For example, the data types of all input tensors and output tensors include float (single-precision floating point number) and half (half-precision floating point number), then the initial common data type is float; for another example, the data types of all input tensors and output tensors include float and int (integer), then the initial common data type is float; for another example, the data types of all input tensors and output tensors include half and int, then the initial common data type is float. The data types and initial common data types of all input tensors and output tensors of the operator to be processed can be obtained through the API of the tensor processing tool (such as TensorIterator).

[0041] In this embodiment, although the initial common data type has been determined based on the data types of all input tensors and output tensors of the operator to be processed, the initial common data type may not be the data type supported by the operator to be processed. If the tensor of the initial common data type is input into the operator to be processed, an error will result. Therefore, it is also necessary to determine the target common data type supported by the operator to be processed based on the characteristics of the operator to be processed.

[0042] Optionally, the target common data type may be determined based on the initial common data type and a set of preset data types related to the operator to be processed.

[0043] In this embodiment, a preset data type set is pre-configured for the operator to be processed, and the preset data type set includes data types supported by the operator to be processed.

[0044] On this basis, after the initial public data type is determined, the target public data type supported by the operator to be processed can be finally determined based on the initial public data type and the preset type set.

[0045] Optionally, determine whether the preset data type set includes the initial public data type; if it is determined that the preset data type set includes the initial public data type, it means that the operator to be processed supports the initial public data type, and the initial public data type can be directly determined as the target public data type; if it is determined that the preset data type set does not include the initial public data type, it means that the operator to be processed does not support the initial public data type, and according to the characteristics of the operator to be processed, the target public data type is selected from the preset data type set, or a data type error prompt message is output.

[0046] Specifically, for example, some operators to be processed only support half and float and have high requirements for calculation accuracy, but the initial public data type is int. If the target public data type is determined to be half or float, there will be precision loss when converting from int to half or float, and the operator to be processed does not allow precision loss, then a data type error prompt message can be output at this time; if the operator to be processed does not have high requirements for calculation accuracy and allows precision loss, one of the supported half and float can be selected as the target public data type, and the specific selection of which to use as the target public data type can be comprehensively selected based on factors such as the calculation efficiency, precision loss, etc. of the operator to be processed for each data type, so as to maximize the performance of the operator calculation.

[0047] It should be noted that, considering that the device-side operation unit may also have certain requirements for data types, for example, the device-side operation unit only supports float and half types. Therefore, before determining the target public data type based on the initial public data type and the preset data type set related to the operator to be processed, it can be determined whether the initial public data type is a data type supported by the operation unit; if it is determined that the initial public data type is not a data type supported by the operation unit, the initial public data type can be converted into a data type supported by the operation unit according to the preset mapping relationship to achieve the compatibility of the operation unit. For example, the device-side operation unit only supports float and int types. If the initial public data type is double (double-precision floating point number), it is mapped to float according to the preset mapping relationship. If the initial public data type is long, it is mapped to int according to the preset mapping relationship; if it is determined that the initial public data type is a data type supported by the operation unit, the target public data type is determined based on the initial public data type and the preset data type set related to the operator to be processed, and it is also necessary to ensure that the determined target public data type is also a data type supported by the operation unit (for example, the above-mentioned preset mapping relationship can be used for mapping, or the data type in the preset data type set is ensured to be a data type supported by the operation unit).

[0048] S202. For any target tensor among all input tensors and output tensors, determine whether the target tensor needs to be converted into a data type according to the initial data type of the target tensor and the target common data type.

[0049] In this embodiment, after determining the target common data type, the input tensors and output tensors can be uniformly aligned to the target common data type to ensure that the operator to be processed can be operated in the device-side computing unit. Considering that the initial data type of any tensor among all input tensors and output tensors (recorded as target tensors) may be the same as or different from the target common data type, the target tensor may or may not need to undergo data type conversion. Whether the target tensor needs to undergo data type conversion can be determined based on the initial data type of the target tensor and the target common data type in combination with the specific situation.

[0050] S203. When it is determined that data type conversion is required, a data type conversion operator is called to perform data type conversion on the target tensor.

[0051] In this embodiment, if it is determined that the target tensor needs to perform data type conversion, a data type conversion operator (such as a cast operator) can be called to perform data type conversion on the target tensor in combination with the initial data type of the target tensor and the target common data type to convert the data type of the target tensor from the initial data type to the target common data type.

[0052] The tensor data processing method provided in this embodiment obtains the initial common data type and target common data type of all input tensors and output tensors of the operator to be processed; for any target tensor in the input tensor and output tensor, it is determined whether the target tensor needs to be converted according to the initial data type and target common data type of the target tensor, and when it is determined that the data type conversion is required, the data type conversion operator is called to convert the data type of the target tensor. By determining the most suitable target common data type, it is ensured that the operator to be processed can support the target common data type, and it is determined whether each input tensor and output tensor needs to be converted, thereby avoiding the performance loss caused by the introduction of unnecessary data type conversion, so as to improve the operation efficiency of the operator.

[0053] Based on any of the above embodiments, the following describes in detail different situations regarding when data type conversion is required.

[0054] The first case is that the target tensor is an input tensor, and the target tensor needs to be aligned with the target common data type, such as Figure 3 As shown, the details are as follows:

[0055] S301. If the target tensor is an input tensor, determine whether the initial data type of the target tensor is the same as the target common data type;

[0056] S302: If it is determined that the initial data type is different from the target common data type, it is determined that data type conversion is required; or

[0057] S303: If it is determined that the initial data type is the same as the target common data type, it is determined that the target tensor does not need to be converted into a data type.

[0058] In this embodiment, if the target tensor is the input tensor, it is determined whether the initial data type of the target tensor is the same as the target common data type. If they are the same, no data type conversion is required. If they are not the same, the data type conversion operator (cast operator) is called to convert the data type of the target tensor from the initial data type to the target common data type.

[0059] The second case is that the target tensor is an output tensor, which is undefined, that is, the user has not input a tensor to define the target tensor. At this time, the data type of the target tensor is not defined. It can be determined that the target tensor does not need to be converted into a data type. The target tensor is created in memory according to the target public data type, and the data type of the created target tensor is the target public data type.

[0060] The third case is that the target tensor is an output tensor, which has been defined. That is, the user has defined a tensor to represent the target tensor. The user-defined tensor is the initialization tensor (output origin tensor), indicating that the user hopes that the shape (size), stride (stride), and data type (dtype) of the target tensor obtained after the operator to be processed is the same as the defined initialization tensor (output origin tensor).

[0061] When writing to the memory, the embodiment of the present application will determine whether the shape of the initialized tensor is the same as the common shape (commonshape, the shape of the input tensor after broadcasting). If not, the initialized tensor may be resized so that the initialized tensor is transformed into the same shape as the common shape after changing the shape of the tensor, including the same size and stride, and the memory is continuous. At this time, the bool value of will_resize is true; if the shape of the initialized tensor is the same as the common shape, there is no need to resize the initialized tensor, and the initialized tensor has not changed its shape. At this time, the bool value of will_resize is false. Of course, some operators to be processed may not support the resize operation.

[0062] Therefore, for any target tensor among all output tensors (output tensors), if the target tensor has been defined, it can be determined whether the target tensor (output current tensor) obtained after the operation of the operator to be processed needs to be converted in data type based on whether the defined initialization tensor (output origin tensor) has changed the tensor shape, the initial data type of the initialization tensor, and the target common data type. When it is determined that data type conversion is required, the data type conversion operator is called to convert the data type of the target tensor.

[0063] Specifically, if the initialization tensor (output origin tensor) changes the tensor shape (the bool value of will_resize is true), and the initial data type of the initialization tensor is different from the target public data type, it means that the data type of the target tensor (output current tensor) obtained after the operation of the operator to be processed is the target public data type, which is different from the data type of the initialization tensor, but the size and stride are the same. Therefore, it is determined that data type conversion is required. Further, the embodiment of the present application can call a data type conversion operator (cast operator), reuse the memory of the initialization tensor, and convert the data type of the target tensor obtained after the operation to the initial data type, so that the result of the target tensor obtained after the operation of the operator to be processed meets the data type defined by the user; of course, if the initialization tensor (output origin tensor) changes the tensor shape (the bool value of will_resize is true), and the initial data type of the initialization tensor is the same as the target public data type, it is determined that no data conversion is required.

[0064] If the initialization tensor (output origin tensor) has not changed the tensor shape (the bool value of will_resize is false), and the current shape and / or current step size of the target tensor (output current tensor) obtained after the operation are the same as the initialization tensor, and the initial data type of the initialization tensor is different from the target public data type, it means that the initialization tensor is originally the same as the public shape (no resize operation is required), and the situation is similar to the situation where the initialization tensor has changed its tensor shape, and it is determined that data type conversion is required. Further, the embodiment of the present application can call a data type conversion operator (cast operator), reuse the memory of the initialization tensor, and convert the data type of the target tensor to the initial data type, so that the result of the target tensor obtained after the operator operation to be processed meets the data type defined by the user.

[0065] If the initialization tensor (output origin tensor) has not changed its tensor shape (the bool value of will_resize is false), and the current shape and / or current step size of the target tensor (output current tensor) obtained after the operation are different from the initialization tensor, and the initial data type of the initialization tensor is the same as the target common data type, then it is determined that no data type conversion is required. At this time, you only need to call the stride copy operation to align the shape of the target tensor to the initial shape of the initialization tensor, and / or align the step size of the target tensor to the initial step size of the initialization tensor, so that the shape and step size of the target tensor meet the shape and step size defined by the user.

[0066] If the initialization tensor (output origin tensor) has not changed its tensor shape (the bool value of will_resize is false), and the current shape and / or current step size of the target tensor (output current tensor) obtained after the operation are different from the initialization tensor, and the initial data type of the initialization tensor is different from the target common data type, then it means that the size, stride, and dtype of the initialization tensor and the target tensor are different at this time, and it is determined that data type conversion is required, and shape and step size alignment are also required. Further, the data type conversion operator (cast operator) can be called to reuse the memory of the initialization tensor, convert the data type of the target tensor to the initial data type, so that the target tensor meets the data type defined by the user, and call the stride copy operation to align the shape of the target tensor to the initial shape of the initialization tensor, and / or align the step size of the target tensor to the initial step size of the initialization tensor, so that the shape and step size of the target tensor meet the shape and step size defined by the user.

[0067] In the above embodiment, the processing of the output tensor is achieved by subdividing the scene and reusing the memory, which effectively avoids the performance loss caused by introducing unnecessary data type conversion, so as to improve the computing efficiency of the operator.

[0068] The present application also provides another embodiment. In order to solve the problem that the operation unit does not support 64-bit data types, such as not supporting double or long (long integer) data types, a virtual 64-bit data type tensor is provided. The real memory of the virtual 64-bit data type tensor is still a 32-bit data type, but the 64-bit data type tensor is displayed to the user, so that the user perceives that the operation unit can process 64-bit data types. The user inputs a 64-bit data type, but its real memory is still a 32-bit data type. The framework and the underlying operation perceive it as a 32-bit data type, and there is no need to call a data type conversion operator (cast operator) for data type conversion. Of course, this is all based on the premise that the accuracy of the operator operation will not be affected.

[0069] Figure 4 A schematic diagram of the structure of a tensor data processing device provided in one embodiment of the present application is shown in FIG. Figure 4 As shown, the tensor data processing device provided in this embodiment is located in the device end, and the tensor data processing device 40 provided in this embodiment includes: a common data type determination module 41 and a data processing module 42.

[0070] The common data type determination module 41 is used to obtain the initial common data type and target common data type of all input tensors and output tensors of the operator to be processed;

[0071] The data processing module 42 is used to determine whether the target tensor needs to be converted into a data type according to the initial data type of the target tensor and the target common data type for any target tensor among all input tensors and output tensors, and to call the data type conversion operator to convert the data type of the target tensor when it is determined that the data type conversion is required.

[0072] In one or more embodiments of the present application, the common data type determination module 41 is further used to:

[0073] A target public data type is determined according to the initial public data type and a set of preset data types related to the operator to be processed.

[0074] In one or more embodiments of the present application, when determining the target public data type according to the initial public data type and the preset data type set related to the operator to be processed, the public data type determination module 41 is used to:

[0075] Determining whether the preset data type set includes the initial public data type;

[0076] If it is determined that the preset data type set includes the initial public data type, the initial public data type is determined as the target public data type; or

[0077] If it is determined that the preset data type set does not include the initial common data type, the target common data type is selected from the preset data type set according to the characteristics of the operator to be processed, or a data type error prompt message is output.

[0078] In one or more embodiments of the present application, before determining the target public data type according to the initial public data type and the preset data type set related to the operator to be processed, the public data type determination module 41 is further configured to:

[0079] Determining whether the initial public data type is a data type supported by the operation unit;

[0080] If it is determined that the initial public data type is not a data type supported by the operation unit, the initial public data type is converted into a data type supported by the operation unit according to a preset mapping relationship.

[0081] In one or more embodiments of the present application, the data processing module 42, for any target tensor among all input tensors and output tensors, determines whether the target tensor needs to be converted into a data type according to the initial data type of the target tensor and the target common data type, is used to:

[0082] If the target tensor is an input tensor, determining whether the initial data type of the target tensor is the same as the target common data type;

[0083] If it is determined that the initial data type is different from the target common data type, it is determined that data type conversion is required; or

[0084] If it is determined that the initial data type is the same as the target common data type, it is determined that no data type conversion is required for the target tensor.

[0085] In one or more embodiments of the present application, when the data processing module 42 calls the data type conversion operator to perform data type conversion on the target tensor, it is used to:

[0086] The data type conversion operator is called to convert the data type of the target tensor from the initial data type to the target common data type.

[0087] In one or more embodiments of the present application, for any target tensor among all input tensors and output tensors, the data processing module 42, when determining whether the target tensor needs to be converted into a data type according to the initial data type of the target tensor and the target common data type, is used to:

[0088] If the target tensor is an output tensor and the target tensor is undefined, determining that the target tensor does not need to be converted into a data type, and creating the target tensor in memory according to the target common data type; or

[0089] If the target tensor is an output tensor and the target tensor has been defined, it is determined whether the target tensor obtained after the operation of the operator to be processed needs to be converted in data type according to whether the initialization tensor corresponding to the target tensor has changed its tensor shape, the initial data type of the initialization tensor, and the target common data type, where the initialization tensor is used to define the target tensor.

[0090] In one or more embodiments of the present application, the data processing module 42 determines whether the target tensor obtained after the operation of the to-be-processed operator needs to be converted into a data type according to whether the initialization tensor corresponding to the target tensor has changed the tensor shape, the initial data type of the initialization tensor, and the target common data type, and calls the data type conversion operator to convert the data type of the current tensor when it is determined that the data type conversion is required, and is used to:

[0091] If the initialized tensor has undergone a tensor shape change and the initial data type of the initialized tensor is different from the target common data type, it is determined that data type conversion is required.

[0092] In one or more embodiments of the present application, when calling the data type conversion operator to perform data type conversion on the target tensor, the data processing module 42 is used to:

[0093] The data type conversion operator is called to reuse the memory of the initial tensor and convert the data type of the target tensor to the initial data type.

[0094] In one or more embodiments of the present application, the data processing module 42 determines whether the target tensor obtained after the to-be-processed operator operation needs to be converted in data type according to whether the initialization tensor corresponding to the target tensor has changed the tensor shape, the initial data type of the initialization tensor, and the target common data type:

[0095] If the initialization tensor has not undergone a tensor shape change, and the current shape and / or current step size of the target tensor are the same as the initial tensor, and the initial data type of the initialization tensor is different from the target common data type, it is determined that data type conversion is required.

[0096] In one or more embodiments of the present application, when the data processing module 42 calls the data type conversion operator to perform data type conversion on the target tensor, it is used to:

[0097] Convert the data type of the target tensor to the initial data type.

[0098] In one or more embodiments of the present application, the data processing module 42 determines whether the target tensor obtained after the to-be-processed operator operation needs to be converted in data type according to whether the initialization tensor corresponding to the target tensor has changed the tensor shape, the initial data type of the initialization tensor, and the target common data type:

[0099] If the initialization tensor has not undergone a tensor shape change, and the current shape and / or current step size of the target tensor are different from those of the initialization tensor, and the initial data type of the initialization tensor is the same as the target common data type, it is determined that no data type conversion is required.

[0100] In one or more embodiments of the present application, the data processing module 42 is further used for:

[0101] The shape of the target tensor is aligned to the initial shape of the initialization tensor, and / or the stride of the target tensor is aligned to the initial stride of the initialization tensor.

[0102] In one or more embodiments of the present application, the data processing module 42 determines whether the target tensor obtained after the to-be-processed operator operation needs to be converted in data type according to whether the initialization tensor corresponding to the target tensor has changed the tensor shape, the initial data type of the initialization tensor, and the target common data type:

[0103] If the initialization tensor has not undergone a tensor shape change, and the current shape and / or current step size of the target tensor are different from those of the initialization tensor, and the initial data type of the initialization tensor is different from the target common data type, it is determined that data type conversion is required.

[0104] In one or more embodiments of the present application, when the data processing module 42 calls the data type conversion operator to perform data type conversion on the target tensor, it is used to:

[0105] Call a data type conversion operator to reuse the memory of the initialization tensor, convert the data type of the target tensor to the initial data type, align the shape of the target tensor to the initial shape of the initialization tensor, and / or align the step size of the target tensor to the initial step size of the initialization tensor.

[0106] The tensor data processing device provided in this embodiment can execute Figure 2 and Figure 3 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0107] Figure 5 A schematic diagram of the structure of a tensor data processing device provided in another embodiment of the present application is shown in FIG. Figure 5 As shown, the tensor data processing device 50 provided in the embodiment of the present application includes: at least one processor 51 and a memory 52;

[0108] The memory 52 stores computer executable instructions;

[0109] At least one processor 51 executes the computer-executable instructions stored in the memory 52, so that at least one processor performs Figure 2 and Figure 3 A tensor data processing method provided by any one of the embodiments.

[0110] In a possible implementation, a computer-readable storage medium is also disclosed. The computer-readable storage medium stores a computer program. When the computer program is executed by at least one processor, Figure 2 and Figure 3 A tensor data processing method provided by any one of the embodiments.

[0111] In a possible implementation manner, a board card is also disclosed, and the board card may be a board card at the device end. Figure 6 FIG. 6 is a schematic diagram showing the structure of a board 60 according to an embodiment of the present application. Figure 6 As shown, the board 60 includes a chip 601, which is a system-on-chip (SoC), or system on chip, integrated with one or more combined processing devices, which are artificial intelligence computing units that support various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in the fields of computer vision, speech, natural language processing, data mining, etc. In particular, deep learning technology is widely used in the field of cloud intelligence. A notable feature of cloud intelligence applications is the large amount of input data, which places high demands on the storage capacity and computing power of the platform. The board 60 of this embodiment is suitable for cloud intelligence applications and has huge off-chip storage, on-chip storage, and powerful computing power.

[0112] The chip 601 is connected to an external device 603 via an external interface device 602. The external device 603 is, for example, a server, a computer, a camera, a display, a mouse, a keyboard, a network card, or a wifi interface. The data to be processed can be transmitted from the external device 603 to the chip 601 via the external interface device 602. The calculation result of the chip 601 can be transmitted back to the external device 603 via the external interface device 602. According to different application scenarios, the external interface device 602 can have different interface forms, such as a PCIe interface.

[0113] The board 60 also includes a storage device 604 for storing data, which includes one or more storage units 605. The storage device 604 is connected to the control device 606 and the chip 601 through a bus and transmits data. The control device 606 in the board 60 is configured to regulate the state of the chip 601. To this end, in an application scenario, the control device 606 may include a single-chip microcomputer (Micro Controller Unit, MCU).

[0114] In a possible implementation, a combined processing device is also provided. Figure 7 6 is a block diagram showing the combined processing device in the chip 601 of this embodiment. Figure 7 As shown in , the combined processing device 70 includes a computing device 701 , an interface device 702 , a processing device 703 and a storage device 704 .

[0115] The computing device 701 is configured to perform user-specified operations, and is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 703 through the interface device 702 to jointly complete the user-specified operations.

[0116] The interface device 702 is used to transmit data and control instructions between the computing device 701 and the processing device 703. For example, the computing device 701 can obtain input data from the processing device 703 via the interface device 702 and write it into the storage device on the computing device 701 chip. Further, the computing device 701 can obtain control instructions from the processing device 703 via the interface device 702 and write them into the control cache on the computing device 701 chip. Alternatively or optionally, the interface device 702 can also read data in the storage device of the computing device 701 and transmit it to the processing device 703.

[0117] The processing device 703, as a general processing device, performs basic controls including but not limited to data handling, starting and / or stopping the computing device 701. Depending on the implementation, the processing device 703 can be a central processing unit (CPU), a graphics processing unit (GPU) or one or more types of processors in other general and / or special processors, these processors include but are not limited to digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As before, only with respect to the computing device 701 of the present application, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 701 and the processing device 703 are integrated and considered together, the two are regarded as forming a heterogeneous multi-core structure.

[0118] The storage device 704 is used to store data to be processed, which may be DRAM 704 , which is a DDR memory, and is usually 16G or larger in size, and is used to store data of the computing device 701 and / or the processing device 703 .

[0119] Figure 8 The single-core computing device 801 is used to process input data such as computer vision, speech, natural language, and data mining. The single-core computing device 801 includes three modules: a control module 81 , a computing module 82 , and a storage module 83 .

[0120] The control module 81 is used to coordinate and control the operation of the operation module 82 and the storage module 83 to complete the deep learning task, and includes an instruction fetch unit (IFU) 811 and an instruction decode unit (IDU) 812. The instruction fetch unit 811 is used to obtain instructions from the processing device 1203, and the instruction decode unit 812 decodes the obtained instructions and sends the decoding results to the operation module 82 and the storage module 83 as control information.

[0121] The operation module 82 includes a vector operation unit 821 and a matrix operation unit 822. The vector operation unit 821 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformation; the matrix operation unit 822 is responsible for the core calculation of the deep learning algorithm, namely matrix multiplication and convolution.

[0122] The storage module 83 is used to store or transfer relevant data, including a neuron RAM (NRAM) 831, a parameter RAM (WRAM) 832, and a direct memory access module (DMA) 833. NRAM 831 is used to store input neurons, output neurons, and intermediate results after calculation; WRAM 832 is used to store the convolution kernel of the deep learning network, that is, the weight; DMA 833 is connected to DRAM 704 through bus 84, and is responsible for data transfer between the single-core computing device 801 and DRAM 704.

[0123] Fig. 9 The schematic diagram of the internal structure of the computing device 701 is a multi-core computing device. The multi-core computing device 901 adopts a hierarchical structure design. The multi-core computing device 901 is a system on a chip, which includes at least one cluster, and each cluster includes multiple processor cores. In other words, the multi-core computing device 901 is composed of a hierarchy of system on a chip-cluster-processor core.

[0124] At the system-on-chip level, Fig. 9 As shown, the multi-core computing device 901 includes an external storage controller 901 , a peripheral communication module 902 , an on-chip interconnect module 903 , a synchronization module 904 and multiple clusters 905 .

[0125] There may be multiple external storage controllers 901, and two are shown in the figure as an example, which are used to respond to access requests issued by the processor core and access external storage devices, such as Figure 7 DRAM 704 in the chip, thereby reading data from outside the chip or writing data. The peripheral communication module 902 is used to receive control signals from the processing device 703 through the interface device 702 to start the computing device 701 to perform tasks. The on-chip interconnect module 903 connects the external storage controller 901, the peripheral communication module 902 and multiple clusters 905 to transmit data and control signals between various modules. The synchronization module 904 is a global synchronization barrier controller (GBC) used to coordinate the work progress of each cluster and ensure information synchronization. Multiple clusters 905 are the computing cores of the multi-core computing device 901. Four are shown as an example in the figure. The four clusters 905 form the following Figure 1With the development of hardware, the multi-core computing device 901 of the present application may also include 8, 16, 64, or even more clusters 905. The cluster 905 is used to efficiently execute the deep learning algorithm.

[0126] At the cluster level, Fig. 9 As shown, each cluster 905 includes multiple processor cores (IPU cores) 906 and a memory core (MEM core) 907. For example, each cluster 905 includes 4 processor cores and 1 memory, and the memory can be DRAM 704. Each processor core is equivalent to Figure 1 An operation unit in Figure 1 A storage unit in.

[0127] The figure shows four processor cores 906 as an example, and the present application does not limit the number of processor cores 906. Its internal architecture is as follows Fig.10 Each processor core 906 is similar to Figure 8 The single-core computing device 801 also includes three modules: a control module 1001, an operation module 1002 and a storage module 1003. The functions and structures of the control module 1001, the operation module 1002 and the storage module 1003 are roughly the same as those of the control module 81, the operation module 82 and the storage module 83. The control module 1001 includes an instruction fetch unit 10011 and an instruction decoding unit 10012. The operation module 1002 includes a vector operation unit 10021 and a matrix operation unit 10022. No further details are given. It should be noted that the storage module 1003 includes an input / output direct memory access module (input / output direct memory access, IODMA) 10033 and a movedirect memory access module (movedirect memory access, MVDMA) 10034. IODMA 10033 controls memory access between NRAM 10031 / WRAM 10032 and DRAM 704 via broadcast bus 909 ; MVDMA 10034 controls memory access between NRAM 10031 / WRAM 10032 and storage unit (SRAM) 908 .

[0128] Back to Figure 7 The storage core 907 is mainly used for storage and communication, that is, to store shared data or intermediate results between the processor cores 906, and to perform communication between the cluster 905 and the DRAM 704, communication between the clusters 905, and communication between the processor cores 906. In other embodiments, the storage core 907 has the ability of scalar operation and is used to perform scalar operation.

[0129] The storage core 907 includes an SRAM 908, a broadcast bus 909, a cluster direct memory access module (CDMA) 910, and a global direct memory access module (GDMA) 911. The SRAM 908 plays the role of a high-performance data transfer station. The data reused between different processor cores 906 in the same cluster 905 does not need to be obtained from the DRAM 704 by each processor core 906, but is transferred between the processor cores 906 via the SRAM 908. The storage core 907 only needs to quickly distribute the reused data from the SRAM 908 to multiple processor cores 906, so as to improve the efficiency of inter-core communication and greatly reduce on-chip and off-chip input / output access.

[0130] The broadcast bus 909, CDMA 910 and GDMA 911 are respectively used to perform communication between processor cores 906, communication between clusters 905 and data transmission between clusters 905 and DRAM 704. They will be described below.

[0131] The broadcast bus 909 is used to complete high-speed communication between the processor cores 906 in the cluster 905. The broadcast bus 909 of this embodiment supports inter-core communication modes including unicast, multicast and broadcast. Unicast refers to point-to-point (e.g., single processor core to single processor core) data transmission, multicast is a communication mode of transmitting a copy of data from the SRAM 908 to several specific processor cores 906, and broadcast is a communication mode of transmitting a copy of data from the SRAM 908 to all processor cores 906, which is a special case of multicast.

[0132] CDMA 910 is used to control memory access to SRAM 908 between different clusters 905 within the same computing device 701 .

[0133] GDMA 911 cooperates with external memory controller 901 to control the access of SRAM 908 of cluster 905 to DRAM 704, or reads data from DRAM 704 to SRAM 908. As can be seen from the above, the communication between DRAM 704 and NRAM 10031 or WRAM 10032 can be realized through two channels. The first channel is to directly connect DRAM 704 and NRAM 10031 or WRAM 10032 through IODAM 10033; the second channel is to first transmit data between DRAM 704 and SRAM 908 through GDMA 911, and then transmit data between SRAM 908 and NRAM 10031 or WRAM 10032 through MVDMA 10034. Although it seems that the second channel requires more components to participate and the data flow is longer, in fact, in some embodiments, the bandwidth of the second channel is much greater than the first channel, so the communication between DRAM 704 and NRAM 10031 or WRAM 10032 may be more efficient through the second channel. The embodiments of the present application can select the data transmission channel according to their own hardware conditions.

[0134] In other embodiments, the functions of GDMA 911 and IODMA 10033 can be integrated into the same component. For the convenience of description, this application regards GDMA 911 and IODMA 10033 as different components. For those skilled in the art, as long as the functions implemented and the technical effects achieved are similar to those of this application, they belong to the protection scope of this application. Furthermore, the functions of GDMA 911, IODMA 10033, CDMA 910, and MVDMA 10034 can also be implemented by the same component.

[0135] The foregoing may be better understood in view of the following clauses:

[0136] Clause 1. A method for processing tensor data, comprising:

[0137] Get the initial and target common data types of all input tensors and output tensors of the operator to be processed;

[0138] For any target tensor among all input tensors and output tensors, judging whether the target tensor needs to be converted into a data type according to the initial data type of the target tensor and the target common data type;

[0139] When it is determined that data type conversion is required, the data type conversion operator is called to perform data type conversion on the target tensor.

[0140] Clause 2. The method according to clause 1, further comprising:

[0141] A target public data type is determined according to the initial public data type and a set of preset data types related to the operator to be processed.

[0142] Clause 3. The method according to clause 2, wherein determining the target public data type according to the initial public data type and the preset data type set related to the operator to be processed comprises:

[0143] Determining whether the preset data type set includes the initial public data type;

[0144] If it is determined that the preset data type set includes the initial public data type, the initial public data type is determined as the target public data type; or

[0145] If it is determined that the preset data type set does not include the initial common data type, the target common data type is selected from the preset data type set according to the characteristics of the operator to be processed, or a data type error prompt message is output.

[0146] Clause 4. The method according to any one of clauses 1 to 3, before determining the target public data type according to the initial public data type and the preset data type set related to the operator to be processed, further comprising:

[0147] Determining whether the initial public data type is a data type supported by the operation unit;

[0148] If it is determined that the initial public data type is not a data type supported by the operation unit, the initial public data type is converted into a data type supported by the operation unit according to a preset mapping relationship.

[0149] Clause 5. The method according to any one of clauses 1 to 3, wherein for any target tensor among all input tensors and output tensors, judging whether the target tensor needs to be converted in data type according to the initial data type of the target tensor and the target common data type, comprises:

[0150] If the target tensor is an input tensor, determining whether the initial data type of the target tensor is the same as the target common data type;

[0151] If it is determined that the initial data type is different from the target common data type, it is determined that data type conversion is required; or

[0152] If it is determined that the initial data type is the same as the target common data type, it is determined that no data type conversion is required for the target tensor.

[0153] Clause 6. The method according to clause 5, wherein the calling of the data type conversion operator to perform data type conversion on the target tensor comprises:

[0154] The data type conversion operator is called to convert the data type of the target tensor from the initial data type to the target common data type.

[0155] Clause 7. A method according to any one of clauses 1 to 3, wherein for any target tensor among all input tensors and output tensors, judging whether the target tensor needs to be converted in data type according to the initial data type of the target tensor and the target common data type, comprises:

[0156] If the target tensor is an output tensor and the target tensor is undefined, determining that the target tensor does not need to be converted into a data type, and creating the target tensor in memory according to the target common data type; or

[0157] If the target tensor is an output tensor and the target tensor has been defined, it is determined whether the target tensor obtained after the operation of the operator to be processed needs to be converted in data type according to whether the initialization tensor corresponding to the target tensor has changed its tensor shape, the initial data type of the initialization tensor, and the target common data type, where the initialization tensor is used to define the target tensor.

[0158] Clause 8. The method according to Clause 7, wherein judging whether the target tensor obtained after the operation of the operator to be processed needs to be converted in data type according to whether the initialization tensor corresponding to the target tensor has changed the tensor shape, the initial data type of the initialization tensor, and the target common data type, comprises:

[0159] If the initialized tensor has undergone a tensor shape change and the initial data type of the initialized tensor is different from the target common data type, it is determined that data type conversion is required.

[0160] Clause 9. The method according to clause 8, wherein the calling of the data type conversion operator to perform data type conversion on the target tensor comprises:

[0161] The data type conversion operator is called to reuse the memory of the initial tensor and convert the data type of the target tensor to the initial data type.

[0162] Clause 10. The method according to Clause 7, wherein judging whether the target tensor obtained after the operation of the operator to be processed needs to be converted in data type according to whether the initialization tensor corresponding to the target tensor has changed the tensor shape, the initial data type of the initialization tensor, and the target common data type, comprises:

[0163] If the initialization tensor has not undergone a tensor shape change, and the current shape and / or current step size of the target tensor are the same as the initial tensor, and the initial data type of the initialization tensor is different from the target common data type, it is determined that data type conversion is required.

[0164] Clause 11. The method according to clause 10, wherein the calling of the data type conversion operator to perform data type conversion on the target tensor comprises:

[0165] Convert the data type of the target tensor to the initial data type.

[0166] Clause 12. The method according to Clause 7, wherein the step of determining whether the target tensor obtained after the operation of the operator to be processed needs to be converted in data type according to whether the initialization tensor corresponding to the target tensor has changed the tensor shape, the initial data type of the initialization tensor, and the target common data type, comprises:

[0167] If the initialization tensor has not undergone a tensor shape change, and the current shape and / or current step size of the target tensor are different from those of the initialization tensor, and the initial data type of the initialization tensor is the same as the target common data type, it is determined that no data type conversion is required.

[0168] Clause 13. The method according to clause 12, further comprising:

[0169] The shape of the target tensor is aligned to the initial shape of the initialization tensor, and / or the stride of the target tensor is aligned to the initial stride of the initialization tensor.

[0170] Clause 14. The method according to Clause 7, wherein the step of determining whether the target tensor obtained after the operation of the operator to be processed needs to be converted in data type according to whether the initialization tensor corresponding to the target tensor has changed the tensor shape, the initial data type of the initialization tensor, and the target common data type, comprises:

[0171] If the initialization tensor has not undergone a tensor shape change, and the current shape and / or current step size of the target tensor are different from those of the initialization tensor, and the initial data type of the initialization tensor is different from the target common data type, it is determined that data type conversion is required.

[0172] Clause 15. The method according to clause 14, wherein the calling of the data type conversion operator to perform data type conversion on the target tensor comprises:

[0173] Call a data type conversion operator to reuse the memory of the initialization tensor, convert the data type of the target tensor to the initial data type, align the shape of the target tensor to the initial shape of the initialization tensor, and / or align the step size of the target tensor to the initial step size of the initialization tensor.

[0174] Clause 16. A tensor data processing apparatus comprising:

[0175] A common data type determination module, used to obtain the initial common data type and target common data type of all input tensors and output tensors of the operator to be processed;

[0176] A data processing module is used to determine whether a target tensor among all input tensors and output tensors needs to be converted into a data type according to the initial data type of the target tensor and the target common data type, and to call a data type conversion operator to convert the data type of the target tensor when it is determined that data type conversion is required.

[0177] Clause 17. A tensor data processing apparatus comprising: at least one processor and a memory;

[0178] The memory stores computer-executable instructions;

[0179] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method as described in any one of clauses 1-15.

[0180] Clause 18. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by at least one processor, implements the method as described in any one of Clauses 1-15.

[0181] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0182] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0183] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0184] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0185] If the integrated unit / module is implemented in the form of hardware, the hardware can be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Unless otherwise specified, the artificial intelligence processor can be any appropriate hardware processor, such as CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any appropriate magnetic storage medium or magneto-optical storage medium, such as resistive random access memory RRAM (Resistive Random Access Memory), dynamic random access memory DRAM (Dynamic Random Access Memory), static random access memory SRAM (Static Random-Access Memory), enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), high-bandwidth memory HBM (High-Bandwidth Memory), hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0186] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0187] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A tensor data processing method, characterized in that: include: Get the initial and target common data types of all input tensors and output tensors of the operator to be processed; For any target tensor among all input tensors and output tensors, judging whether the target tensor needs to be converted into a data type according to the initial data type of the target tensor and the target common data type; When it is determined that data type conversion is required, the data type conversion operator is called to perform data type conversion on the target tensor.

2. The method according to claim 1, characterized in that: The method further comprises: A target public data type is determined according to the initial public data type and a set of preset data types related to the operator to be processed.

3. The method according to claim 2, characterized in that The determining the target common data type according to the initial common data type and a set of preset data types related to the operator to be processed includes: Determining whether the preset data type set includes the initial public data type; If it is determined that the preset data type set includes the initial public data type, the initial public data type is determined as the target public data type; or If it is determined that the preset data type set does not include the initial common data type, the target common data type is selected from the preset data type set according to the characteristics of the operator to be processed, or a data type error prompt message is output.

4. The method according to any one of claims 1 to 3, characterized in that: Before determining the target public data type according to the initial public data type and the preset data type set related to the operator to be processed, the method further includes: Determining whether the initial public data type is a data type supported by the operation unit; If it is determined that the initial public data type is not a data type supported by the operation unit, the initial public data type is converted into a data type supported by the operation unit according to a preset mapping relationship.

5. The method according to any one of claims 1 to 3, characterized in that: For any target tensor among all input tensors and output tensors, judging whether the target tensor needs to be converted into a data type according to the initial data type of the target tensor and the target common data type includes: If the target tensor is an input tensor, determining whether the initial data type of the target tensor is the same as the target common data type; If it is determined that the initial data type is different from the target common data type, it is determined that data type conversion is required; or If it is determined that the initial data type is the same as the target common data type, it is determined that no data type conversion is required for the target tensor.

6. The method according to claim 5, characterized in that The calling of the data type conversion operator to perform data type conversion on the target tensor includes: The data type conversion operator is called to convert the data type of the target tensor from the initial data type to the target common data type.

7. The method according to any one of claims 1 to 3, characterized in that: For any target tensor among all input tensors and output tensors, judging whether the target tensor needs to be converted in data type according to the initial data type of the target tensor and the target common data type includes: If the target tensor is an output tensor and the target tensor is undefined, determining that the target tensor does not need to be converted into a data type, and creating the target tensor in memory according to the target common data type; or If the target tensor is an output tensor and the target tensor has been defined, then judging whether the target tensor obtained after the operation of the operator to be processed needs to be converted into a data type according to whether the defined initialization tensor has been changed in shape, the initial data type of the initialization tensor, and the target common data type, wherein the initialization tensor is the defined target tensor; If the initialization tensor has been changed in shape, and the initial data type of the initialization tensor is different from the target common data type, determining that data type conversion is required; If the initialization tensor has not undergone a tensor shape change, the current shape and / or current step size of the target tensor are the same as the initial tensor, and the initial data type of the initialization tensor is different from the target common data type, it is determined that data type conversion is required; If the initialization tensor has not undergone a tensor shape change, and the current shape and / or current step size of the target tensor are different from those of the initialization tensor, and the initial data type of the initialization tensor is different from the target common data type, it is determined that data type conversion is required.

8. The method according to claim 7, characterized in that The calling of the data type conversion operator to perform data type conversion on the target tensor includes: The data type conversion operator is called to reuse the memory of the initial tensor and convert the data type of the target tensor to the initial data type.

9. The method according to claim 7, characterized in that: The method further comprises: If the initialization tensor has not undergone a tensor shape change, and the current shape and / or current step size of the target tensor are different from those of the initialization tensor, and the initial data type of the initialization tensor is the same as the target common data type, it is determined that no data type conversion is required.

10. The method according to claim 9, characterized in that The method further comprises: The shape of the target tensor is aligned to the initial shape of the initialization tensor, and / or the stride of the target tensor is aligned to the initial stride of the initialization tensor.

11. A tensor data processing device, characterized in that: include: A common data type determination module, used to obtain the initial common data type and target common data type of all input tensors and output tensors of the operator to be processed; A data processing module is used to determine whether a target tensor among all input tensors and output tensors needs to be converted into a data type according to the initial data type of the target tensor and the target common data type, and to call a data type conversion operator to convert the data type of the target tensor when it is determined that data type conversion is required.

12. A tensor data processing device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the method according to any one of claims 1 to 10 is implemented.

Images