Data operation method and device, electronic equipment and storage medium

Abstract

This application discloses a data processing method, apparatus, electronic device, and storage medium, relating to the field of computer science and technology. The method includes: transferring a TensorInfor variable to a unified computing device architecture, where the architecture automatically performs implicit content copying, copying the TensorInfor value from the central processing unit to the device, avoiding explicit memory creation, copying, and deallocation. It also involves transferring a CastHelper instance to the unified computing device architecture, where the architecture performs type conversion based on the type of the tensor parameters and the target data type in CastHelper, avoiding memory allocation and extensive memory copying. Therefore, avoiding explicit memory creation, copying, and deallocation, as well as avoiding extensive memory copying, can improve the computational performance of operators.

Description

Technical Field

[0001] This application relates to the field of computer science and technology, and in particular to data processing methods, apparatus, electronic devices and storage media. Background Technology

[0002] Intelligent question answering models are typically built using deep learning frameworks, which usually incorporate graphics processing unit (GPU) chips and employ various Compute Unified Device Architecture (CUDA) operators for training and computation.

[0003] During the training and computation of the intelligent question answering model, frequent calls to operators inevitably result in a large number of memory operations, leading to poor operator performance. Summary of the Invention

[0004] This application provides data processing methods, apparatus, electronic devices, and storage media to improve the computational performance of operators.

[0005] This application provides a data processing method applied to a computing device, including:

[0006] Obtain the target operator;

[0007] Determine the tensor parameters corresponding to the target operator;

[0008] Create TensorInfor variables corresponding to the tensor parameters;

[0009] Obtain the target data type corresponding to the tensor parameters;

[0010] Based on the target data type, create CastHelper instances corresponding to the tensor parameters;

[0011] The TensorInfor variable and CastHelper instance are transferred to the unified computing device architecture to call the unified computing device architecture's code to obtain the data to be processed, perform type conversion and operations on the data to be processed, and obtain the result data.

[0012] Output the result data.

[0013] This application also provides a data processing device, applied to a computing device, comprising:

[0014] The acquisition module is used to acquire the target operator;

[0015] The determination module is used to determine the tensor parameters corresponding to the target operator;

[0016] The processing module is used to create TensorInfor variables corresponding to the tensor parameters;

[0017] The acquisition module is also used to acquire the target data type corresponding to the tensor parameters;

[0018] The processing module is also used to create CastHelper instances corresponding to tensor parameters based on the target data type;

[0019] The processing module is also used to transfer TensorInfor variables and CastHelper instances to the unified computing device architecture, so as to call the code of the unified computing device architecture to obtain the data to be processed, perform type conversion and operation on the data to be processed, and obtain the result data.

[0020] The output module is used to output the result data.

[0021] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement any of the above-described data processing methods.

[0022] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described data processing methods.

[0023] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described data processing methods.

[0024] The data processing method, apparatus, electronic device, and storage medium provided in this application transfer TensorInfor variables to a unified computing device architecture. The unified computing device architecture automatically performs implicit content copying, copying the TensorInfor value from the Central Processing Unit (CPU) to the device, avoiding explicit memory creation, copying, and deallocation. When a CastHelper instance is transferred to the unified computing device architecture, the architecture can perform type conversion based on the type of the tensor parameters in CastHelper and the target data type, avoiding memory allocation and extensive memory copying. Therefore, avoiding explicit memory creation, copying, and deallocation, as well as avoiding extensive memory copying, can improve the computational performance of operators. Attached Figure Description

[0025] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of a scenario provided for an embodiment of this application;

[0027] Figure 2 Flowchart of the data processing method provided in the embodiments of this application Figure 1 ;

[0028] Figure 3 A schematic diagram illustrating the optimization of the target operator as an example;

[0029] Figure 4 Flowchart of the data processing method provided in the embodiments of this application Figure 2 ;

[0030] Figure 5 Flowchart of the data processing method provided in the embodiments of this application Figure 3 ;

[0031] Figure 6 Flowchart of the data processing method provided in the embodiments of this application Figure 4 ;

[0032] Figure 7 Flowchart of the data processing method provided in the embodiments of this application Figure 5 ;

[0033] Figure 8 Flowchart of the data processing method provided in the embodiments of this application Figure 6 ;

[0034] Figure 9 Flowchart of the data processing method provided in the embodiments of this application Figure 7 ;

[0035] Figure 10 Flowchart of the data processing method provided in the embodiments of this application Figure 8 ;

[0036] Figure 11 Flowchart of the data processing method provided in the embodiments of this application Figure 9 ;

[0037] Figure 12 Flowchart of the data processing method provided in the embodiments of this application Figure 10 ;

[0038] Figure 13This is a schematic diagram of the structure of the data processing device provided in the embodiments of this application;

[0039] Figure 14 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0041] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0042] The data processing method provided in this application first obtains the target operator, then determines the tensor parameters and target data type corresponding to the target operator, creates TensorInfor variables and CastHelper instances corresponding to the tensor parameters, and then transmits the TensorInfor variables and CastHelper instances to a unified computing device architecture. The method then retrieves the data to be processed by calling the code of the unified computing device architecture, performs type conversion and operations on the data, and finally outputs the result data. Based on the data processing method provided in this application, the computational performance of the operator can be improved by avoiding explicit memory creation, copying, and releasing, as well as avoiding memory allocation and large amounts of memory copying.

[0043] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] This section describes the specific application environment architecture or hardware architecture upon which the data processing methods depend. (References) Figure 1 , Figure 1 This is a schematic diagram of a scenario provided for an embodiment of this application, such as... Figure 1As shown, the computing device ① is the executing entity of this application, and can be a server, a chip or a board. The chip can be an artificial intelligence (AI) chip, which includes at least one hardware processor.

[0045] Figure 2 Flowchart of the data processing method provided in the embodiments of this application Figure 1 ,like Figure 2 As shown, it includes:

[0046] S201. Obtain the target operator.

[0047] In the context of specific scenarios, operators typically refer to mathematical or logical operators used to perform calculations on data. Operator operations consist of two parts: one is C++ code, used for preparatory work such as parameter parsing, which runs on the CPU; the other is CUDA code, used for the operator's computation, with the data stored on the device. The target operator is a predefined operator, such as the Silu operator.

[0048] S202. Determine the tensor parameters corresponding to the target operator.

[0049] With the example scenario, a tensor is a mathematical concept that is a generalization of the concepts of vector and matrix. The target operator includes multiple tensor parameters, and the number of tensor parameters is determined by the operation type of the target operator.

[0050] S203. Create the TensorInfor variable corresponding to the tensor parameter.

[0051] Combined with scenario examples, Figure 3 An example of optimizing the target operator, such as... Figure 3 As shown, the TensorInfor variable corresponding to the tensor parameter of the target operator can be determined through C++ code, and then the TensorInfor variable is transferred to the CUDA core by value. The CUDA core will automatically perform an implicit memory copy, copying the TensorInfo value from the CPU to the device, avoiding explicit memory creation, copying, and deallocation.

[0052] S204. Obtain the target data type corresponding to the tensor parameters.

[0053] With a scenario example, an internal data type can be defined, denoted as TGDataType-t, to abstract the front-end type. The target data type is the optimal data type corresponding to the tensor parameter. Additionally, basic type conversion rules between data types can be defined, denoted as CommonTypeMap, which can be encapsulated using std::map. std::map is an associative container in the C++ standard library, implemented based on a red-black tree, used to store key-value pairs. std::map's functionality includes fast retrieval, insertion, and deletion of elements, and its elements are automatically sorted according to the order of the keys, typically in ascending order.

[0054] S205. Based on the target data type, create a CastHelper instance corresponding to the tensor parameters.

[0055] In computer science, CastHelper typically refers to an auxiliary tool or library that provides type conversion functionality during programming. CastHelper instances are used to encapsulate basic type conversion rules, the data types of various tensor parameters, and the target data type.

[0056] S206. Transfer the TensorInfor variable and CastHelper instance to the unified computing device architecture to call the unified computing device architecture's code to obtain the data to be processed, perform type conversion and operations on the data to be processed, and obtain the result data.

[0057] Combined with scenario examples, Figure 3 The obtained CastHelper instance can be transferred to the CUDA core. The transfer method can be pass-by-value. The CUDA core performs type conversion and operations on the data to be operated on based on the basic type conversion rules encapsulated in the CastHelper instance, the data types of each tensor parameter, and the target data type, so as to avoid memory allocation and memory copying.

[0058] It is worth mentioning that the target data type does not need to be encapsulated in the CastHelper instance. It can be transferred to CUDA independently. In this case, CUDA can select the corresponding CUDA core to perform data operations based on the target data type, so as to realize data operations based on the target data type.

[0059] S207, Output the result data.

[0060] Based on the scenario examples, the final result data will be output.

[0061] This example demonstrates how transferring TensorInfor variables and CastHelper instances to the CUDA core can improve operator performance by avoiding explicit memory creation, copying, and deallocation, as well as avoiding memory allocation and extensive memory copying.

[0062] Optional, Figure 4 Flowchart of the data processing method provided in the embodiments of this application Figure 2 ,like Figure 4 As shown, tensor parameters include input tensor parameters and output tensor parameters;

[0063] Accordingly, S202 includes:

[0064] S401. Determine the parameters and result parameters involved in the operation based on the operation type of the target operator.

[0065] In a scenario example, if the target operator is an add operator, such as c=a+b, then "a" and "b" are the parameters involved in the operation, and "c" is the result parameter.

[0066] S402. Determine the parameters involved in the calculation as input tensor parameters, and determine the result parameters as output tensor parameters.

[0067] Based on the scenario example, the parameters “a” and “b” involved in the operation can be determined as the input tensor parameters corresponding to the add operator, and the result parameter “c” can be determined as the output tensor parameter corresponding to the add operator.

[0068] Based on the method provided in this example, all input tensor parameters and output tensor parameters corresponding to the target operator can be determined.

[0069] Optional, Figure 5 Flowchart of the data processing method provided in the embodiments of this application Figure 3 ,like Figure 5 As shown, S203 includes:

[0070] S501. Define the TensorInfor structure.

[0071] In the example scenario, TensorInfor represents tensor information. The TensorInfor structure is used to encapsulate the basic information of the tensor to obtain the TensorInfor variable. For each tensor parameter, a corresponding TensorInfor variable can be created.

[0072] S502. Based on the TensorInfor structure, the basic information corresponding to the tensor parameters is encapsulated to obtain the TensorInfor variables corresponding to the tensor parameters. The basic information includes shape, stride and ndim.

[0073] With a scenario example, the shape of a tensor is an integer tuple representing the size of the tensor in each dimension. The stride of a tensor is an integer tuple of the same length as shape, representing the number of elements to skip when traversing a single dimension in memory; that is, it is used to calculate the position of the next element from its memory address when accessing a specific dimension of the tensor. ndim represents the dimension of the tensor.

[0074] For example, taking the silu operator as an example, the silu operator includes an input tensor parameter and an output tensor parameter. Therefore, by creating TensorInfor variables corresponding to the input tensor parameter and the output tensor parameter respectively, they can be denoted as info-x and info-y respectively. info-x is used to encapsulate the shape, stride and ndim corresponding to the input tensor parameter, and info-y is used to encapsulate the shape, stride and ndim corresponding to the output tensor parameter.

[0075] Based on the method provided in this example, information such as hape, stride, and ndim of each tensor can be encapsulated through the TensorInfor structure to obtain the TensorInfor variable corresponding to each tensor.

[0076] Optional, Figure 6 Flowchart of the data processing method provided in the embodiments of this application Figure 4 ,like Figure 6 As shown, S204 includes:

[0077] S601. Define basic type conversion rules, which include multiple preset conversion rules.

[0078] With a scenario example, different rules can be configured for different operators by maintaining a configuration file. Specifically, a fixed value can be defined for different data types using an enum, as follows:

[0079] TG-Dtype-inwalid = 0, TG-Dtype-half = 1, TG-Dtype-FLOAT = 2, TG-Dtype-DOUBLE = 3, TG-Dtype-INT8 = 4, TG-Dtype-INT16 = 5, TG-Dtype-INT31 = 6, TG-Dtype-INT32 = 7, TG-Dtype-INT64 = 8, TG-Dtype-UINT8 = 9, TG-Dtype-UINT16 = 10, TG-Dtype-UINT32 = 11, TG-Dtype-UINT64 = 12, TG-Dtype-BOOL = 13, TG-Dtype-BHALF = 14。

[0080] The following are multiple preset conversion rules included in the basic type conversion rule CommonTypeMap:

[0081] CommonTypeMap =

[0082] {{{TG-Dtype-INT32, TG-Dtype-INT64}, TG-Dtype-INT32},

[0083] {{TG-Dtype-INT32, TG-Dtype-DOUBLE}, TG-Dtype-DOUBLE},

[0084] {{TG-Dtype-INT64, TG-Dtype-DOUBLE}, TG-Dtype-DOUBLE},

[0085] {{TG-Dtype-FLOAT, TG-Dtype-INT32}, TG-Dtype-FLOAT},

[0086] {{TG-Dtype-FLOAT, TG-Dtype-INT64}, TG-Dtype-FLOAT},

[0087] {{TG-Dtype-FLOAT, TG-Dtype-DOUBLE}, TG-Dtype-FLOAT},

[0088] {{TG-Dtype-UINT8, TG-Dtype-FLOAT}, TG-Dtype-FLOAT},

[0089] {{TG-Dtype-BOOL, TG-Dtype-INT8}, TG-Dtype-INT8},

[0090] {{TG-Dtype-BOOL,TG-Dtype-UINT8},TG-Dtype-UINT8},

[0091] {{TG-Dtype-BOOL,TG-Dtype-FLOAT},TG-Dtype-FLOAT},

[0092] {{TG-Dtype-BOOL,TG-Dtype-HALF},TG-Dtype-HALF},

[0093] {{TG-Dtype-BOOL,TG-Dtype-BHALF},TG-Dtype-BHALF},

[0094] {{TG-Dtype-BOOL,TG-Dtype-INT64},TG-Dtype-INT64},

[0095] {{TG-Dtype-BOOL,TG-Dtype-INT32},TG-Dtype-INT32},

[0096] {{TG-Dtype-BOOL,TG-Dtype-DOUBLE},TG-Dtype-DOUBLE},

[0097] {{TG-Dtype-HALF,TG-Dtype-DOUBLE},TG-Dtype-HALF},

[0098] {{TG-Dtype-HALF,TG-Dtype-FLOAT},TG-Dtype-HALF},

[0099] {{TG-Dtype-HALF,TG-Dtype-INT32},TG-Dtype-HALF},

[0100] {{TG-Dtype-HALF,TG-Dtype-INT64},TG-Dtype-HALF},

[0101] {{TG-Dtype-HALF,TG-Dtype-UINT8},TG-Dtype-HALF},

[0102] {{TG-Dtype-HALF,TG-Dtype-BOOL},TG-Dtype-HALF},

[0103] {{TG-Dtype-BHALF, TG-Dtype-DOUBLE}, TG-Dtype-BHALF},

[0104] {{TG-Dtype-BHALF, TG-Dtype-FLOAT}, TG-Dtype-BHALF},

[0105] {{TG-Dtype-BHALF, TG-Dtype-INT32}, TG-Dtype-BHALF},

[0106] {{TG-Dtype-BHALF, TG-Dtype-INT64}, TG-Dtype-BHALF},

[0107] {{TG-Dtype-BHALF, TG-Dtype-UINT8}, TG-Dtype-BHALF},

[0108] {{TG-Dtype-INT8, TG-Dtype-FLOAT}, TG-Dtype-FLOAT},

[0109] {{TG-Dtype-FLOAT, TG-Dtype-INT8}, TG-Dtype-FLOAT}}

[0110] Each line represents a conversion rule. For example, taking “{TG-Dtype-INT32,TG-Dtype-INT64},TG-Dtype-INT32}” as an example, when the data type of the first tensor parameter being compared is INT32 and the data type of the second tensor parameter is INT64, an upward compatibility approach is adopted, which determines that the data types of the first and second tensor parameters are both INT32.

[0111] All the conversion rules represented by the rows together constitute a rule set, which can be used to determine the basic type conversion rule CommonTypeMap.

[0112] S602. Determine the data type of the input tensor parameters and the data type of the output tensor parameters.

[0113] Using the `add` operator as an example, "a" and "b" are the input tensor parameters of the `add` operator, and "c" is the output tensor parameter. We need to determine the data types of the two input tensor parameters "a" and "b", and also determine the data type of the output tensor parameter "c".

[0114] S603. Using the principle of pairwise comparison, tensor parameters are traversed based on multiple preset conversion rules to obtain the target data type.

[0115] In a scenario example, for the three tensor parameters of the `add` operator, based on the aforementioned basic type conversion rules, a pairwise comparison method is used to determine the optimal target data type corresponding to the target operator. This optimal target data type can be denoted as `common-type`. Specifically, this is used to compare the data types corresponding to "a" and "b", and then to compare the data types corresponding to "b" and "c" to obtain the `common-type` corresponding to the three tensor parameters of the `add` operator.

[0116] Based on the method provided in this example, the optimal target data type corresponding to the target operator can be determined.

[0117] Optional, Figure 7 Flowchart of the data processing method provided in the embodiments of this application Figure 5 ,like Figure 7 As shown, S205 includes:

[0118] S701, Obtain the data type conversion method.

[0119] With the example scenario, the data type conversion method is used to convert the data type of tensor parameters into the optimal target data type common-type.

[0120] S702, Define the CastHelper module.

[0121] In the context of specific scenarios, CastHelper typically refers to an auxiliary tool or library that provides type conversion functionality during the programming process.

[0122] S703. Encapsulate the data type conversion method, the data type corresponding to the tensor parameter, the size of the data type, and the target data type into the CastHelper module to obtain a CastHelper instance.

[0123] Based on the scenario example, iterate through all tensor parameters in the target operator to obtain the data type corresponding to each tensor parameter. Then, call the CastHelper method PushTypeAndSize to encapsulate the data type of all tensor parameters, the size of each data type, the data type conversion method, and the optimal target data type common-type into the CastHelper module.

[0124] Based on the method provided in this example, the CastHelper instance corresponding to the target operator can be obtained.

[0125] Optional, Figure 8 Flowchart of the data processing method provided in the embodiments of this application Figure 6 ,like Figure 8 As shown, S206 includes:

[0126] S801, calls the code of the unified computing device architecture to obtain the data to be processed based on the TensorInfor variable and the CastHelper instance.

[0127] Using a scenario example, the CUDA code is invoked to start the CUDA core. The CUDA core determines the corresponding data to be processed based on the received TensorInfor variable and CastHelper instance.

[0128] S802. Convert the data to be processed into the target data type.

[0129] Based on scenario examples, the CUDA core can determine the optimal target data type (common-type) corresponding to the target operator based on the CastHelper instance, and at the same time determine the data type of the data to be operated on. The CUDA core can then convert the data format of the data to be operated on into the common-type based on the data type conversion method encapsulated in the CastHelper instance.

[0130] S803. Perform operations on the target data type to be operated on to obtain the result data.

[0131] Based on the scenario example, after converting the data format of the data to be processed into common-type, the data to be processed in common-type format is subjected to the corresponding operation based on the operation type of the target operator, and then the corresponding result data is obtained. At this time, the data type of the result data is also the optimal target data type common-type.

[0132] Figure 9 Flowchart of the data processing method provided in the embodiments of this application Figure 7 ,like Figure 9 As shown, S207 includes:

[0133] S901. Obtain the data type corresponding to the output tensor parameters.

[0134] Based on scenario examples, determine the data type corresponding to the output tensor parameters in the target operator.

[0135] S902. Convert the result data into the data type corresponding to the output tensor parameters and then output it.

[0136] In the example scenario, when outputting result data, it is necessary to convert the result data to the data type corresponding to the output tensor parameter in the target operator. For example, if the data type corresponding to the output tensor parameter in the target operator is int8, then the data type of the result data needs to be converted to int8 before output.

[0137] Based on the method provided in this example, operations can be performed on the data to be operated on based on the optimal target data type common-type, which can improve the efficiency of operator operations. Converting the data type of the result data to the data type corresponding to the output tensor parameters can ensure the consistency of data types.

[0138] Optional, Figure 10 Flowchart of the data processing method provided in the embodiments of this application Figure 8 The unified computing device architecture includes multiple kernel functions, and the CastHelper instance includes data loading instructions;

[0139] like Figure 10 As shown, in S801, the data to be processed is obtained based on the TensorInfor variable and the CastHelper instance, including:

[0140] S1001. Determine the target data type based on the CastHelper instance.

[0141] Based on the scenario example, since the CastHelper instance encapsulates the optimal target data type common-type, the corresponding optimal target data type common-type can be determined based on the CastHelper instance.

[0142] S1002. Determine the target kernel function corresponding to the target data type.

[0143] Based on the scenario example, since CUDA kernels are multi-type reused, the kernel function corresponding to the optimal target data type common-type can be determined, and this kernel function can be determined as the target kernel function.

[0144] S1003. Execute the target kernel function to determine the shape, stride, and ndim information of the corresponding tensor parameters based on the TensorInfor variable.

[0145] Based on the scenario example, since the TensorInfor variable corresponding to the tensor parameter encapsulates the shape, stride, and ndim information of the tensor parameter, the shape, stride, and ndim information of the corresponding tensor parameter can be determined based on the TensorInfor variable.

[0146] S1004. Based on the shape, stride, and ndim information in the tensor parameters, obtain the corresponding offset.

[0147] Based on the scenario examples, the corresponding offset can be calculated according to the shape, stride, and ndim information corresponding to the tensor parameters.

[0148] S1005. Obtain the data type corresponding to the input tensor parameters.

[0149] In the example scenario, since the CastHelper instance encapsulates data types, the data type of the input tensor parameter can be determined based on the CastHelper instance. The CastHelper instance contains the tg-types variable, which represents the data type of the tensor parameter. Therefore, CastHelper can determine the data type of the input tensor parameter from tg-types.

[0150] S1006. Determine the first memory size occupied by the data type corresponding to the input tensor parameter.

[0151] In the scenario example, the CastHelper instance also contains an element-sizes variable, which represents the memory size occupied by the data type. For example, an int is 4 bytes, and an int64 is 8 bytes. The offset calculated above is passed to the CastHelper's LoadData instruction. The purpose of LoadData is to retrieve the data in src-ptr. The element-sizes corresponding to the data type of the input tensor parameter can be obtained through arg-index to get the first memory size corresponding to the input tensor parameter.

[0152] S1007. Determine the first memory location based on the offset and the first memory size.

[0153] Based on the scenario example, LoadData calculates the first memory location corresponding to the input tensor parameter according to the first memory size and offset obtained above.

[0154] S1008. Obtain the data to be processed based on the first memory location.

[0155] Based on the scenario example, and using the first memory location obtained above, the corresponding data to be processed is extracted from the first memory location.

[0156] Figure 11 Flowchart of the data processing method provided in the embodiments of this application Figure 9 ,like Figure 11 As shown, S902 includes:

[0157] S1101. Determine the second memory size occupied by the data type corresponding to the output tensor parameter.

[0158] Based on the scenario example, the LoadData instruction is called, and the element-sizes corresponding to the data type of the output tensor parameter are obtained through arg-index, so as to obtain the second memory size corresponding to the input tensor parameter.

[0159] S1102. Determine the second memory location based on the offset and the second memory size.

[0160] Based on the scenario example, LoadData calculates the second memory location corresponding to the output tensor parameters according to the second memory size and offset obtained above.

[0161] S1103. After converting the result data into the data type corresponding to the output tensor parameters, write it to the second memory location to output the result data.

[0162] Using a scenario example, CastHelper can determine the data type corresponding to the output tensor parameters from tg-types. When outputting the result data, it first converts the result data from the optimal target data type common-type to the data type corresponding to the output tensor parameters, and then writes the converted result data to the second memory location determined above. It's worth noting that if the optimal target data type common-type is the same as the data type corresponding to the output tensor parameters, no conversion is needed.

[0163] Based on the method provided in this example, the purpose of data type conversion, operation, and output of the data to be processed can be accurately achieved.

[0164] Optional, Figure 12 Flowchart of the data processing method provided in the embodiments of this application Figure 10 ,like Figure 12 As shown, it also includes:

[0165] S1201. Determine whether the target operator is a special operator.

[0166] Based on scenario examples, special operators are preset operators, such as the Add and Silu operators.

[0167] S1202. If the target operator is a special operator, the target data type corresponding to the tensor parameters in the target operator is determined based on the special type conversion rules.

[0168] Based on scenario examples, there are corresponding special type conversion rules for special operators. For example, for the Add operator, the special type conversion rules include the following two:

[0169] {{TG-Dtype-BHALF, TG-Dtype-INT64}, TG-Dtype-BHALF};

[0170] {{TG-Dtype-HALF, TG-Dtype-FLOAT}, TG-Dtype-HALF}.

[0171] For the Silu operator, the special type conversion rule includes the following special type conversion rule:

[0172] {{TG-Dtype-HALF, TG-Dtype-FLOAT}, TG-Dtype-HALF}.

[0173] Therefore, for the Add and Silu operators, the optimal target data type corresponding to the tensor parameters in the operator can be determined based on the corresponding special type conversion rules.

[0174] S1203. If the target operator is not a special operator, the target data type corresponding to the tensor parameters in the target operator is determined based on the basic type conversion rules.

[0175] Based on the scenario example, if the target operator is not a special operator such as Add or Silu, the optimal target data type corresponding to the tensor parameter in the target operator is determined based on the basic type conversion rules defined above.

[0176] Based on the method provided in this example, ordinary operators and special operators can be distinguished, and the optimal target data type corresponding to tensor parameters can be determined more accurately.

[0177] This embodiment can improve the computational performance of the operator by avoiding explicit memory creation, copying, and release, as well as avoiding memory allocation and large amounts of memory copying.

[0178] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0179] Embodiments of this application also provide a data processing device. Figure 13 This is a schematic diagram of the structure of the data processing device provided in the embodiments of this application, such as... Figure 13 As shown, the data processing device includes:

[0180] Module 131 is used to acquire the target operator;

[0181] Module 132 is used to determine the tensor parameters corresponding to the target operator;

[0182] Processing module 133 is used to create TensorInfor variables corresponding to tensor parameters;

[0183] The acquisition module 131 is also used to acquire the target data type corresponding to the tensor parameters;

[0184] Processing module 133 is also used to create CastHelper instances corresponding to tensor parameters based on the target data type;

[0185] The processing module 133 is also used to transfer the TensorInfor variable and CastHelper instance to the unified computing device architecture, so as to call the code of the unified computing device architecture to obtain the data to be processed, perform type conversion and operation on the data to be processed, and obtain the result data.

[0186] Output module 134 is used to output result data.

[0187] Optionally, tensor parameters include input tensor parameters and output tensor parameters;

[0188] The determination module 132 is specifically used to determine the parameters involved in the operation and the result parameters based on the operation type of the target operator;

[0189] The determination module 132 is further used to determine the parameters involved in the calculation as input tensor parameters and the result parameters as output tensor parameters.

[0190] Optionally, processing module 133 is specifically used to define the TensorInfor structure;

[0191] The processing module 133 is further used to encapsulate the basic information corresponding to the tensor parameters based on the TensorInfor structure to obtain the TensorInfor variables corresponding to the tensor parameters. The basic information includes shape, stride and ndim.

[0192] Optionally, module 131 is used to define basic type conversion rules, which include multiple preset conversion rules;

[0193] The acquisition module 131 is also specifically used to determine the data type corresponding to the input tensor parameters and the data type corresponding to the output tensor parameters;

[0194] The acquisition module 131 is specifically used to traverse tensor parameters based on multiple preset conversion rules using a pairwise comparison principle to obtain the target data type.

[0195] Optionally, processing module 133 is also used to obtain data type conversion methods;

[0196] Processing module 133 is also used to define the CastHelper module;

[0197] The processing module 133 is further used to encapsulate the data type conversion method, the data type corresponding to the tensor parameter, the size of the data type and the target data type into the CastHelper module to obtain a CastHelper instance.

[0198] Optionally, the processing module 133 is further used to call the code of the unified computing device architecture to obtain the data to be processed based on the TensorInfor variable and the CastHelper instance;

[0199] The processing module 133 is also specifically used to convert the data to be processed into the target data type;

[0200] The processing module 133 is further used to perform operations on the data to be processed of the target data type in order to obtain the result data;

[0201] The processing module 133 is also specifically used to obtain the data type corresponding to the output tensor parameters;

[0202] The processing module 133 is also used to convert the result data into the data type corresponding to the output tensor parameters and then output it.

[0203] Optionally, the unified computing device architecture includes multiple kernel functions, and the CastHelper instance includes data loading instructions;

[0204] Accordingly, the processing module 133 is also specifically used to determine the target data type based on the CastHelper instance;

[0205] Processing module 133 is also specifically used to determine the target kernel function corresponding to the target data type;

[0206] The processing module 133 is further used to execute the target kernel function to determine the shape, stride and ndim information in the corresponding tensor parameters based on the TensorInfor variable;

[0207] The processing module 133 is further used to obtain the corresponding offset based on the shape, stride and ndim information in the tensor parameters;

[0208] The processing module 133 is also specifically used to obtain the data type corresponding to the input tensor parameters;

[0209] The processing module 133 is further used to determine the first memory size occupied by the data type corresponding to the input tensor parameter;

[0210] The processing module 133 is further used to determine the first memory location based on the offset and the first memory size;

[0211] The processing module 133 is further used to obtain the data to be processed based on the first memory location;

[0212] The processing module 133 is also specifically used to determine the second memory size occupied by the data type corresponding to the output tensor parameter;

[0213] The processing module 133 is further used to determine the second memory location based on the offset and the second memory size;

[0214] The processing module 133 is further used to convert the result data into the data type corresponding to the output tensor parameters and write it to the second memory location to output the result data.

[0215] For a description of the features in the embodiment corresponding to the data processing device, please refer to the relevant description in the embodiment corresponding to the data processing method, which will not be repeated here.

[0216] Figure 14 A schematic diagram of the structure of the electronic device provided in this application. Figure 14 As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus.

[0217] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to execute the above-described data processing method embodiment.

[0218] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0219] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0220] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0221] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0222] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above data processing method embodiments when it is run.

[0223] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0224] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above data processing method embodiments.

[0225] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above data processing method embodiments.

[0226] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0227] The data processing method provided in this application has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data processing method, characterized in that, Applied to computing devices, including: Obtain the target operator; Determine the tensor parameters corresponding to the target operator; Define a TensorInfor structure; Based on the TensorInfor structure, the basic information corresponding to the tensor parameters is encapsulated to obtain the TensorInfor variables corresponding to the tensor parameters. Obtain the target data type corresponding to the tensor parameters; Based on the target data type, create a CastHelper instance corresponding to the tensor parameter; The TensorInfor variable and CastHelper instance are transferred to the unified computing device architecture to call the code of the unified computing device architecture to obtain the data to be processed, perform type conversion and operation on the data to be processed, and obtain the result data. Output the resulting data.

2. The method according to claim 1, characterized in that, The tensor parameters include input tensor parameters and output tensor parameters; Accordingly, determining the tensor parameters corresponding to the target operator includes: Based on the operation type of the target operator, determine the parameters involved in the operation and the result parameters; The parameters involved in the calculation are defined as the input tensor parameters, and the result parameters are defined as the output tensor parameters.

3. The method according to claim 1, characterized in that, The basic information includes shape, stride, and ndim.

4. The method according to claim 2, characterized in that, The step of obtaining the target data type corresponding to the tensor parameters includes: Define basic type conversion rules, wherein the basic type conversion rules include multiple preset conversion rules; Determine the data type corresponding to the input tensor parameters and the data type corresponding to the output tensor parameters; The tensor parameters are traversed based on the multiple preset transformation rules using a pairwise comparison principle to obtain the target data type; the pairwise comparison principle is used to compare the data type corresponding to the input tensor parameters and the data type corresponding to the output tensor parameters.

5. The method according to claim 4, characterized in that, The step of creating a CastHelper instance corresponding to the tensor parameters based on the target data type includes: Get data type conversion methods; Define the CastHelper module; The data type conversion method, the data type corresponding to the tensor parameter, the size of the data type, and the target data type are encapsulated in the CastHelper module to obtain the CastHelper instance.

6. The method according to claim 2, characterized in that, The step of transferring the TensorInfor variable and CastHelper instance to the unified computing device architecture to call the code of the unified computing device architecture to obtain the data to be processed, and performing type conversion and operations on the data to be processed to obtain the result data includes: The code of the unified computing device architecture is invoked to obtain the data to be processed based on the TensorInfor variable and the CastHelper instance; Transform the data to be processed into the target data type; Perform operations on the target data type to obtain the result data; The output of the result data includes: Obtain the data type corresponding to the output tensor parameters; The resulting data is then converted into the data type corresponding to the output tensor parameters and output.

7. The method according to claim 6, characterized in that, The unified computing device architecture includes multiple kernel functions, and the CastHelper instance includes data loading instructions; Accordingly, obtaining the data to be processed based on the TensorInfor variable and the CastHelper instance includes: The target data type is determined based on the CastHelper instance; Determine the target kernel function corresponding to the target data type; Execute the target kernel function to determine the shape, stride, and ndim information of the corresponding tensor parameters based on the TensorInfor variable; Based on the shape, stride, and ndim information in the tensor parameters, the corresponding offset is obtained; Obtain the data type corresponding to the input tensor parameters; Determine the first memory size occupied by the data type corresponding to the input tensor parameter; Based on the offset and the first memory size, the first memory location is determined; Based on the first memory location, obtain the data to be processed; Accordingly, the step of converting the result data into the data type corresponding to the output tensor parameters and then outputting it includes: Determine the second memory size occupied by the data type corresponding to the output tensor parameter; The second memory location is determined based on the offset and the second memory size; After converting the result data into the data type corresponding to the output tensor parameters, it is written to the second memory location to output the result data.

8. A data processing device, characterized in that, Applied to computing devices, including: The acquisition module is used to acquire the target operator; The determination module is used to determine the tensor parameters corresponding to the target operator; The processing module is used to define the TensorInfor structure and, based on the TensorInfor structure, encapsulate the basic information corresponding to the tensor parameters to obtain the TensorInfor variables corresponding to the tensor parameters. The acquisition module is also used to acquire the target data type corresponding to the tensor parameters; The processing module is also used to create a CastHelper instance corresponding to the tensor parameter based on the target data type; The processing module is also used to transfer the TensorInfor variable and CastHelper instance to the unified computing device architecture, so as to call the code of the unified computing device architecture to obtain the data to be processed, perform type conversion and operation on the data to be processed, and obtain the result data. The output module is used to output the result data.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the data processing method as described in any one of claims 1 to 7.

Images