Multi-dimensional data processing method and device, equipment and medium

Abstract

The invention relates to the technical field of data processing, in particular to a multi-dimensional data processing method and device, equipment and a medium, which are used for performing instantiation processing on a template according to an input tensor parameter to obtain an offset calculation instance. And performing initialization operation on the offset calculation instance to determine the step length and the shape of each input array on each output dimension, and recording the step length and the shape in a set storage space. And starting an offset calculation instance to carry out index calculation on the step length and the shape of each input array recorded in the storage space on each output dimension so as to obtain the offset corresponding to the output array. And calculating the data corresponding to the offset according to a set calculation rule to obtain an output result. By pre-calculating the step length and shape of each input array on each output dimension, unnecessary memory access can be reduced. The index calculation process is simplified by constructing the offset calculation instance, and data access and calculation can be efficiently and accurately achieved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, equipment and medium for processing multi-dimensional data. Background Art

[0002] With the rapid advancement of deep learning and artificial intelligence technologies, the demand for high-performance computing capabilities that can perform complex computing tasks has also increased dramatically. In the context of graphics processing units (GPUs), parallel computing using the Compute Unified Device Architecture (CUDA) has become a key means to increase computing speed.

[0003] In deep learning, tensors are basic data structures that describe the layout of multidimensional arrays through attribute parameters such as shape, strides, and dimension (ndim). When CUDA operators are executed, the data that needs to be processed is often not stored continuously in memory, or if the shapes of two tensors are not exactly the same, direct element-level operations are usually impossible because their elements cannot correspond one to one. This means that when performing calculations, the computer needs to constantly jump between different memory locations and cannot access data in sequence, otherwise it will cause out-of-bounds errors. The exact location of each element in memory, i.e., the index, needs to be calculated based on these attribute parameters such as shape, strides, and dimension to achieve correct reading and writing of data. Traditional index calculation methods often face problems such as out-of-bounds access, high computational complexity, and difficulty in parallelization, which limit the improvement of overall performance.

[0004] It can be seen that how to efficiently and accurately implement data access and calculation is a problem that technical personnel in this field need to solve. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a method, device, equipment and medium for processing multi-dimensional data, which can solve the problems of out-of-bounds access, high computational complexity and difficulty in parallelization faced by index calculation.

[0006] In order to solve the above technical problems, an embodiment of the present invention provides a method for processing multi-dimensional data, including:

[0007] Instantiate the template according to the input tensor parameters to obtain an offset calculation instance; wherein the tensor parameters include the shape, step size, dimension and data corresponding to the input array and the output array respectively;

[0008] Performing an initialization operation on the offset calculation instance to determine the step length and shape of each input array in each output dimension, and recording the step length and shape of each input array in each output dimension in a set storage space;

[0009] Starting the offset calculation instance to perform index calculation on the step length and shape of each input array recorded in the storage space in each output dimension, so as to obtain the offset corresponding to the output array;

[0010] The data corresponding to the offset is operated according to the set operation rules to obtain an output result.

[0011] On the one hand, starting the offset calculation instance to perform index calculation on the step length and shape of each input array recorded in the storage space on each output dimension to obtain the offset corresponding to the output array includes:

[0012] Initialize an offset register of all zeros; wherein the offset register is used to store the offset;

[0013] Traversing the dimensions of the output array, and reading the shape and stride of the current dimension from the storage space;

[0014] Determine whether the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have been traversed;

[0015] When the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have not been traversed, the shape and step size corresponding to the current index are modulo-operated using the built-in modulo function of the offset calculation instance, and the obtained quotient is used as the index of the next dimension, and the product of the remainder and the step size is used as the offset corresponding to the current dimension;

[0016] Storing the offset corresponding to the current dimension into the offset register;

[0017] Add one to the current dimension as the latest current dimension, and use the index of the next dimension as the latest current index, and return the step of reading the shape and step size of the current dimension from the storage space;

[0018] Until the current dimension is greater than the maximum dimension value or all dimensions of the output array are traversed, the traversal of the dimensions of the output array is terminated, and the offsets corresponding to all dimensions recorded in the offset register are read;

[0019] The accumulated value of the offsets corresponding to all dimensions is used as the offset corresponding to the output array.

[0020] On the one hand, after traversing the dimensions of the output array and reading the shape and step size of the current dimension from the storage space, it also includes:

[0021] In the case where the dimension of the input array is greater than or equal to the dimension of the output array, a copy operation is performed on the elements of the input array to match the dimension of the output array.

[0022] On the one hand, using the built-in modulo function of the offset calculation instance to perform a modulo operation on the shape and step size corresponding to the current index includes:

[0023] The divmod function included in the offset calculation example is used to perform a modulus operation on the shape and step size corresponding to the current index.

[0024] On the one hand, before instantiating the template according to the input tensor parameter to obtain the offset calculation instance, it also includes:

[0025] Keywords of the unified computing device architecture are deployed in the template so that data can be calculated on the graphics processor for multiple times without being transferred back to the central processor.

[0026] On the one hand, the data corresponding to the offset is operated according to the set operation rules to obtain an output result including:

[0027] The pointer arithmetic calculation formula is called to process the data corresponding to the offset to obtain an output result; wherein the pointer arithmetic calculation formula is as follows:

[0028] ;

[0029] in, Indicates the output result, Indicates the execution function. Represents the function corresponding to the operation rule, A pointer to the data corresponding to the offset, Indicates the offset.

[0030] On the one hand, after instantiating the template according to the input tensor parameter to obtain the offset calculation instance, it also includes:

[0031] When a configuration adjustment instruction is obtained, the initial template parameters contained in the offset calculation instance are adjusted according to the new template parameters carried in the configuration adjustment instruction to obtain a new offset calculation instance; wherein the template parameters include the number of input arrays, the type of index and / or the step type.

[0032] The embodiment of the present invention further provides a multi-dimensional data processing device, including an instantiation unit, an initialization unit, an index calculation unit and an operation unit;

[0033] The instantiation unit is used to instantiate the template according to the input tensor parameters to obtain an offset calculation instance; wherein the tensor parameters include the shape, step size, dimension and data corresponding to the input array and the output array respectively;

[0034] The initialization unit is used to perform an initialization operation on the offset calculation instance to determine the step length and shape of each input array in each output dimension, and record the step length and shape of each input array in each output dimension in a set storage space;

[0035] The index calculation unit is used to start the offset calculation instance to perform index calculation on the step length and shape of each input array recorded in the storage space in each output dimension, so as to obtain the offset corresponding to the output array;

[0036] The operation unit is used to operate the data corresponding to the offset according to the set operation rules to obtain an output result.

[0037] On the one hand, the index calculation unit includes an initialization subunit, a first reading subunit, a judgment subunit, a remainder subunit, a storage subunit, an addition subunit, a subunit, a second reading subunit and an accumulation subunit;

[0038] The initialization subunit is used to initialize an offset register of all zeros; wherein the offset register is used to store the offset;

[0039] The first reading subunit is used to traverse the dimensions of the output array and read the shape and step size of the current dimension from the storage space;

[0040] The judging subunit is used to judge whether the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have been traversed;

[0041] The modulo subunit is used to, when the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have not been traversed, use the modulo function built into the offset calculation instance to perform a modulo operation on the shape and step size corresponding to the current index, use the obtained quotient as the index of the next dimension, and use the product of the remainder and the step size as the offset corresponding to the current dimension;

[0042] The storage subunit is used to store the offset corresponding to the current dimension into the offset register;

[0043] The adding subunit is used to add one to the current dimension as the latest current dimension;

[0044] The subunit is used to take the index of the next dimension as the latest current index and return the step of reading the shape and step size of the current dimension from the storage space;

[0045] The second reading subunit is used to terminate the traversal of the dimensions of the output array until the current dimension is greater than the maximum dimension value or all dimensions of the output array are traversed, and read the offsets corresponding to all dimensions recorded in the offset register;

[0046] The accumulation subunit is used to use the accumulated values ​​of the offsets corresponding to all dimensions as the offsets corresponding to the output array.

[0047] On the one hand, it also includes a replication unit;

[0048] The copying unit is used to traverse the dimension of the output array, and after reading the shape and step size of the current dimension from the storage space, if the dimension of the input array is greater than or equal to the dimension of the output array, perform a copy operation on the elements of the input array to match the dimension of the output array.

[0049] On the one hand, the modulo subunit is used to use the divmod function included in the offset calculation instance to perform a modulo operation on the shape and step size corresponding to the current index.

[0050] On the one hand, it also includes deployment units;

[0051] The deployment unit is used to deploy keywords of the unified computing device architecture in the template before instantiating the template according to the input tensor parameters to obtain an offset calculation instance, so that the data can be calculated multiple times on the graphics processor without being transmitted back to the central processing unit.

[0052] On the one hand, the operation unit is used to call a pointer arithmetic calculation formula to process the data corresponding to the offset to obtain an output result; wherein the pointer arithmetic calculation formula is as follows:

[0053] ;

[0054] in, Indicates the output result, Indicates the execution function. Represents the function corresponding to the operation rule, A pointer to the data corresponding to the offset, Indicates the offset.

[0055] On the one hand, it also includes an adjustment unit;

[0056] The adjustment unit is used to instantiate the template according to the input tensor parameters to obtain an offset calculation instance, and then, when a configuration adjustment instruction is obtained, adjust the initial template parameters contained in the offset calculation instance according to the new template parameters carried in the configuration adjustment instruction to obtain a new offset calculation instance; wherein the template parameters include the number of input arrays, the type of index and / or the step type.

[0057] An embodiment of the present invention further provides a multi-dimensional data processing device, including:

[0058] Memory for storing computer programs;

[0059] A processor is used to execute the computer program to implement the steps of the multi-dimensional data processing method as described above.

[0060] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the multi-dimensional data processing method as described above are implemented.

[0061] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned multi-dimensional data processing method when executed by a processor.

[0062] It can be seen from the above technical solution that the template is instantiated according to the input tensor parameters to obtain an offset calculation instance; wherein the tensor parameters include the shape, step, dimension and data corresponding to the input array and the output array. In order to avoid repeatedly passing and copying the same data in each operation, the offset calculation instance can be initialized to determine the step and shape of each input array in each output dimension, and the step and shape of each input array in each output dimension are recorded in the set storage space. The offset calculation instance is started to perform index calculation on the step and shape of each input array recorded in the storage space in each output dimension to obtain the offset corresponding to the output array. The data corresponding to the offset is calculated according to the set operation rules to obtain the output result. The beneficial effect of the present invention is that by pre-calculating the step and shape of each input array in each output dimension, unnecessary memory access can be reduced and the memory access efficiency can be improved. And by constructing the offset calculation instance, the index calculation process is simplified, which greatly reduces the calculation burden on the graphics processor. The entire implementation process does not require complex index calculations and dynamic memory access, but directly uses the calculated offset to quickly locate data, efficiently and accurately implementing data access and calculation. In addition, this direct access mode significantly improves the efficiency of parallel processing, allowing large-scale data processing tasks to be completed faster. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 A method for processing multi-dimensional data provided by an embodiment of the present invention;

[0065] Figure 2 A flowchart of a method for performing index calculation on the step length and shape of each input array in each output dimension provided by an embodiment of the present invention;

[0066] Figure 3 A schematic diagram of the structure of a multi-dimensional data processing device provided by an embodiment of the present invention;

[0067] Figure 4 A structural diagram of a multi-dimensional data processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] The terms "including" and "having" in the specification of the present invention and the above-mentioned drawings, as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0070] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0071] As a parallel computing platform and programming model, CUDA is widely used in the field of deep learning because it can fully utilize the powerful computing power of GPU. Currently, in order to determine the exact location of each element in memory, a common practice is to transfer the shape and strides information of the Tensor from the CPU to the GPU and directly pass it to the CUDA kernel function as attribute parameters. Inside the kernel function, the offset of each element is dynamically calculated through complex arithmetic operations.

[0072] Although this method is simple to implement, it has a significant performance bottleneck. The shape and strides information of the Tensor is transferred from the CPU to the GPU, and complex index calculations are performed inside the CUDA kernel function. Because additional metadata (i.e., shape and strides) need to be transferred across devices (CPU to GPU), additional performance overhead is introduced.

[0073] Because the parallel computing advantage of GPU lies in large-scale simple operations, complex index calculations are more common in operators that require cross-element, cross-row or cross-column access or operations, such as slicing and transposition. Frequent complex index operations will significantly increase the computational burden of a single thread. These complex arithmetic operations usually involve multiple multiplication, division and modulus operations. Although these operations can be performed in parallel on the GPU, each thread needs to perform these operations, resulting in a huge overall amount of calculation and reducing parallel efficiency.

[0074] Therefore, the embodiments of the present invention provide a method, apparatus, device and medium for processing multi-dimensional data. By encapsulating high-performance offset calculation instances, it can support offset calculation of multi-dimensional arrays (tensors), thereby simplifying and optimizing broadcast operations in scenarios such as deep learning and scientific computing.

[0075] The solution provided by the embodiment of the present invention is not only applicable to the field of artificial intelligence (AI), but also can be applied to other fields that need to process multidimensional data and complex data formats, such as scientific computing, image processing, signal processing, etc. Through similar implementation methods, the efficiency of data processing and computing in these fields can be expanded and optimized.

[0076] Next, a multi-dimensional data processing method provided by an embodiment of the present invention is described in detail. Figure 1 A method for processing multi-dimensional data provided by an embodiment of the present invention includes:

[0077] S101: Instantiate the template according to the input tensor parameters to obtain an offset calculation instance.

[0078] Among them, tensor parameters can include the shape, stride, dimension, and data of the input array and output array respectively.

[0079] There are often multiple types of parameters involved in multi-dimensional data. The index calculation method for each type of parameter is similar, and the index calculation of one type of parameter is used as an example to explain. Considering that the number of parameters of each type is often large, the number of input arrays can be determined based on the number of input parameters of each type. For example, if the number of input parameters of the same type is 2, there can be two groups of input arrays, namely input array 1 and input array 2.

[0080] In an embodiment of the present invention, a high-performance offset calculation (OffsetCalculator) tool with good encapsulation and focusing on automatic broadcasting functions is designed, and this tool can support index and offset calculation of multi-dimensional arrays.

[0081] In the initial state, the offset calculation tool is stored in the form of a template, and the template cannot be used directly. It is necessary to instantiate the template in combination with actual application requirements to obtain an instance that can be used for offset calculation. This instance can be called an offset calculation instance.

[0082] In actual applications, you can use the MakeOffsetCalculator template function to construct an offset calculation instance, namely an OffsetCalculator instance, based on the input tensor parameters (such as x1, x2, y). Among them, x1 represents the tensor parameter of input array 1, x2 represents the tensor parameter of input array 2, and y represents the tensor parameter of the output array. These tensor parameters contain information such as shape, step size, dimension, and data.

[0083] S102: Performing an initialization operation on the offset calculation instance to determine the step length and shape of each input array in each output dimension, and recording the step length and shape of each input array in each output dimension in a set storage space.

[0084] In an embodiment of the present invention, in order to avoid repeatedly passing and copying the same data in each operation, the offset calculation instance may be initialized to determine the stride and shape of each input array in each output dimension, and the stride and shape of each input array in each output dimension may be recorded in a set storage space (all_sizes_).

[0085] The OffsetCalculator tool is presented in the form of an OffsetCalculator structure. The OffsetCalculator structure implements a tool for calculating the offset of elements in a multidimensional array during broadcasting or similar operations. The broadcasting mechanism can automatically expand a tensor of a smaller shape to match a tensor of a larger shape.

[0086] The OffsetCalculator structure contains the following parameters: input_dims1, input_dims2, output_dims, all_sizes_, and strides_. Among them, input_dims1 represents the dimension of input array 1, input_dims2 represents the dimension of input array 2, and output_dims represents the dimension of the output array. all_sizes_ is a two-dimensional array that stores the information of each input array on each output dimension, with IntDivider<index_t> IntDivider is a custom class for efficient integer division and modulo. strides_ is a two-dimensional array that stores the strides of each input array in each output dimension. The strides take into account the element size in order to correctly handle arrays of different data types.

[0087] In actual applications, the constructor in the OffsetCalculator structure can receive the number of dimensions of the input array, the number of dimensions of the output array, the shape (shapes) array, the stride (strides) array, and the optional element size (element_sizes) array. Initialize input_dims1, output_dims, and traverse the dimensions of the output array. The stride and shape of each input array in each output dimension can be calculated, and this information can be stored in all_sizes_.

[0088] S103: Start an offset calculation instance to perform index calculation on the step length and shape of each input array recorded in the storage space in each output dimension to obtain the offset corresponding to the output array.

[0089] The GetBroadCast function is the core of the offset calculation example. The input of the GetBroadCast function can be a linear index (linear_idx). Under the given linear index, the shape and step size of the input array corresponding to each output dimension can be read from all_sizes_. By performing modulo operations on the shape and step size under each dimension, the offset of each element of the input array in the output array can be output.

[0090] S104: Calculate the data corresponding to the offset according to the set calculation rules to obtain an output result.

[0091] After determining the offset corresponding to the output array, the corresponding data can be read based on the offset.

[0092] In practical applications, the pointer arithmetic calculation formula can be called to process the data corresponding to the offset to obtain the output result; wherein, the pointer arithmetic calculation formula is as follows:

[0093] ;

[0094] in, Indicates the output result, Indicates the execution function. Represents the function corresponding to the operation rule, Indicates the pointer to the data corresponding to the offset. Indicates the offset.

[0095] The function corresponding to the operation rule may be determined based on actual operation requirements. For example, the function corresponding to the operation rule may be an absolute value function or a comparison function.

[0096] It can be seen from the above technical solution that the template is instantiated according to the input tensor parameters to obtain an offset calculation instance; wherein the tensor parameters include the shape, step, dimension and data corresponding to the input array and the output array. In order to avoid repeatedly passing and copying the same data in each operation, the offset calculation instance can be initialized to determine the step and shape of each input array in each output dimension, and the step and shape of each input array in each output dimension are recorded in the set storage space. The offset calculation instance is started to perform index calculation on the step and shape of each input array recorded in the storage space in each output dimension to obtain the offset corresponding to the output array. The data corresponding to the offset is calculated according to the set operation rules to obtain the output result. The beneficial effect of the present invention is that by pre-calculating the step and shape of each input array in each output dimension, unnecessary memory access can be reduced and the memory access efficiency can be improved. And by constructing the offset calculation instance, the index calculation process is simplified, which greatly reduces the calculation burden on the graphics processor. The entire implementation process does not require complex index calculations and dynamic memory access, but directly uses the calculated offset to quickly locate data, efficiently and accurately implementing data access and calculation. In addition, this direct access mode significantly improves the efficiency of parallel processing, allowing large-scale data processing tasks to be completed faster.

[0097] Figure 2 A flowchart of a method for performing index calculation on the step length and shape of each input array in each output dimension provided by an embodiment of the present invention, the method comprising:

[0098] S201: Initialize an offset register to all zeros.

[0099] The offset register may be used to store the offset.

[0100] In practical applications, an offset (offset_type) object, namely an offset register (offsets), can be initialized with all zeros to store the offset.

[0101] S202: traverse the dimensions of the output array and read the shape and step size of the current dimension from the storage space.

[0102] In the embodiment of the present invention, the offset corresponding to each dimension can be determined by traversing the dimensions of the output array.

[0103] For the calculation of the offset in each dimension, you can use the shape and step information recorded in all_sizes_ to perform fast division to get the offset corresponding to each dimension. The offsets corresponding to all dimensions are accumulated and summed to get the final offset corresponding to the output array.

[0104] S203: Determine whether the current dimension is less than or equal to the maximum dimension value and whether all dimensions of the output array have been traversed.

[0105] In the embodiment of the present invention, traversing the dimensions of the output array is a loop iteration process, and each calculation of the offset under a dimension is a loop. In order to end the loop process in time, it can be determined whether the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have been traversed.

[0106] If the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have not been traversed, it means that there are dimensions whose offsets have not been calculated, and the operation of S204 can be performed. If the current dimension is greater than the maximum dimension value or all dimensions of the output array have been traversed, the operation of S207 is performed.

[0107] S204: Use the built-in modulo function of the offset calculation instance to perform a modulo operation on the shape and step size corresponding to the current index, use the obtained quotient as the index of the next dimension, and use the product of the remainder and the step size as the offset corresponding to the current dimension.

[0108] In the embodiment of the present invention, the divmod function included in the offset calculation example may be used to perform a modulo operation on the shape and step size corresponding to the current index.

[0109] The divmod function is used to perform a modulo operation on the shape and step size corresponding to the current index (linear_idx). The quotient is used to update linear_idx to continue the calculation of the next dimension, and the remainder is used to calculate the offset of each input array in the current dimension. The offset in the current dimension can be the product of the remainder and the step size in the current dimension.

[0110] S205: Store the offset corresponding to the current dimension into the offset register.

[0111] In the embodiment of the present invention, it is necessary to accumulate and sum the offsets in all dimensions, so each time an offset corresponding to a dimension is obtained, it can be stored in an offset register.

[0112] S206: Add one to the current dimension as the latest current dimension, and use the index of the next dimension as the latest current index, and return to the step of reading the shape and step size of the current dimension from the storage space.

[0113] According to the operations from S202 to S205, the offset corresponding to a dimension can be calculated. Each time the offset corresponding to a dimension is calculated, the next round of loop needs to be executed. At this time, the current dimension can be increased by one as the latest current dimension, and the index of the next dimension can be used as the latest current index, and return to S202 to execute the operation of reading the shape and step size of the current dimension from the storage space.

[0114] S207: End the traversal of the dimensions of the output array, and read the offsets corresponding to all dimensions recorded in the offset register.

[0115] When the current dimension is greater than the maximum dimension value or all dimensions of the output array are traversed, the traversal of the dimensions of the output array can be ended, and the offsets corresponding to all dimensions recorded in the offset register can be read.

[0116] S208: The accumulated value of the offsets corresponding to all dimensions is used as the offset corresponding to the output array.

[0117] In the embodiment of the present invention, by traversing the dimensions of the output array, the offset corresponding to each dimension can be calculated, and the final offset can be obtained by accumulating and summing the offsets of all dimensions. The entire implementation process does not require the execution of complex calculations, and the rapid determination of the offset of the multidimensional array is achieved.

[0118] Considering that in practical applications, the shapes of input arrays and output arrays are not exactly the same, in order to implement calculation operations between tensors of different shapes, the tensor of smaller shape can be automatically expanded to match the tensor of larger shape according to the broadcast mechanism.

[0119] Therefore, after traversing the dimensions of the output array and reading the shape and stride of the current dimension from the storage space, we can determine whether the dimension of the input array is greater than or equal to the dimension of the output array.

[0120] In the case where the dimension of the input array is greater than or equal to the dimension of the output array, the elements of the input array may be copied to match the dimension of the output array.

[0121] In deep learning frameworks, it is often necessary to perform operations on multiple tensors of different shapes, such as addition and multiplication. These operations usually require automatic processing of the shapes of tensors so that they can be operated element-wise, even if their shapes are not exactly the same. This is usually achieved through a broadcast mechanism, which automatically expands a tensor of smaller shape to match a tensor of larger shape.

[0122] For example, if we calculate tensors with shapes of (10) and (2, 10), (10) will be implicitly expanded to (2, 10), that is, copied twice. The expansion operation does not need to be reflected, and we only need to fetch data according to the shape of (2, 10).

[0123] In an embodiment of the present invention, OffsetCalculator has a built-in powerful GetBroadcast function, which realizes the automatic processing of the broadcast mechanism. In element-level operations, when encountering tensors with different shapes, GetBroadcast can intelligently identify and automatically expand the dimensions of the smaller tensor to match the shape of the larger tensor. This feature not only simplifies the code logic and improves the flexibility of the code, but also enables OffsetCalculator to be widely used in various tensor calculation scenarios that require broadcast operations, ensuring that no out-of-bounds access occurs during the broadcast process, greatly improving the usability of the code.

[0124] To speed up the code, before instantiating the template based on the input tensor parameters to obtain the offset calculation instance, the Unified Compute Device Architecture keyword can be deployed in the template so that the data can be calculated multiple times on the GPU without being transferred back to the CPU.

[0125] Keywords of the unified computing device architecture can include keywords in CUDA programming such as __host__ __device__ __forceinline__. __host__ __device__ means that they can run on both the CPU and the GPU (if CUDA is used). This allows the architecture to fully utilize the parallel computing capabilities of the GPU.

[0126] For example, after getting the OffsetCalculator instance, you can call the LaunchArithmeticKernel function to launch a parallel computing kernel on the specified CUDA stream. The kernel function is a __host__ __device__ lambda expression that uses the OffsetCalculator instance to calculate the offset and perform the actual arithmetic operation.

[0127] The lambda expression is marked as __host__ __device__, which means that it can be executed on the host (CPU) or device (GPU). However, in CUDA programming, lambda expressions are usually converted to CUDA Kernel and executed on the GPU. numel represents the total number of elements to be processed, which determines how many iterations the kernel of the parallel calculation will perform. In the lambda expression, the offset of all tensors under the given index (idx) is calculated by calling offset_calc.GetBroadcast(idx).

[0128] While __host__ __device__ itself does not directly reduce data transfer, it allows developers to operate directly on the GPU on data stored in GPU memory. This helps reduce unnecessary data transfer, because the data can be calculated multiple times on the GPU without being transferred back to the CPU. Reducing data transfer can significantly reduce the running time of the program, because data transfer is usually more time-consuming than calculation operations.

[0129] __forceinline__ flag, instructing the compiler to inline this method if possible.

[0130] Inlining means that it is expanded to each call site during compilation, rather than being called like a normal function. This can reduce the overhead of function calls and improve program execution efficiency, especially in the single instruction multiple data (SIMD) execution model of the GPU.

[0131] In the embodiment of the present invention, the OffsetCalculator tool provides flexible template parameter and constructor parameter setting options, and the user can easily adjust the configuration of the OffsetCalculator according to specific application scenarios and requirements.

[0132] When the configuration adjustment instruction is obtained, the initial template parameters included in the offset calculation instance may be adjusted according to the new template parameters carried in the configuration adjustment instruction to obtain a new offset calculation instance.

[0133] The template parameters may include the number of input arrays (NARGS), the type of index (index_t), and / or the stride type (signed_strides).

[0134] NARGS is a compile-time parameter that allows the OffsetCalculator to handle varying numbers of input arrays.

[0135] The default value of index_t is uint32_t. Users can choose integer types of different sizes as indexes as needed.

[0136] signed_strides is a boolean indicating whether strides should be a signed type. If true, the strides type will be a signed version of index_t; otherwise, it will be index_t itself.

[0137] By configuring template parameters, OffsetCalculator can flexibly support different numbers of inputs, different index types, and whether or not to handle signed step sizes, meeting different application scenarios and requirements.

[0138] In the embodiment of the present invention, offset calculation instances matching different application scenarios can be pre-established for common application scenarios. In practical applications, the template can be instantiated according to the tensor parameters corresponding to each application scenario to obtain the offset calculation instance corresponding to each application scenario.

[0139] In order to facilitate subsequent retrieval and use, the offset calculation instances matching each application scenario can be stored in the set memory space.

[0140] When a scene switching instruction is obtained, a target offset calculation instance matching the scene identifier carried in the scene switching instruction can be called from the set memory space, so as to use the target offset calculation instance to perform index calculation on the data in the current scene.

[0141] By pre-storing the offset calculation instances corresponding to various common application scenarios, the matching offset calculation instance can be directly called when the application scenario is switched, without the need to re-instantiate the template every time the scenario is switched.

[0142] Figure 3 A schematic diagram of the structure of a multi-dimensional data processing device provided by an embodiment of the present invention, comprising an instantiation unit 31, an initialization unit 32, an index calculation unit 33 and an operation unit 34;

[0143] An instantiation unit 31 is used to instantiate the template according to the input tensor parameters to obtain an offset calculation instance; wherein the tensor parameters include the shape, step size, dimension and data corresponding to the input array and the output array respectively;

[0144] An initialization unit 32, used to perform an initialization operation on the offset calculation instance to determine the step length and shape of each input array in each output dimension, and record the step length and shape of each input array in each output dimension in a set storage space;

[0145] An index calculation unit 33 is used to start an offset calculation instance to perform index calculation on the step length and shape of each input array recorded in the storage space in each output dimension to obtain an offset corresponding to the output array;

[0146] The operation unit 34 is used to operate the data corresponding to the offset according to the set operation rules to obtain an output result.

[0147] In some embodiments, the index calculation unit includes an initialization subunit, a first reading subunit, a judgment subunit, a remainder subunit, a storage subunit, an addition subunit, a subunit, a second reading subunit, and an accumulation subunit;

[0148] An initialization subunit, used for initializing an offset register of all zeros; wherein the offset register is used for storing the offset;

[0149] The first reading subunit is used to traverse the dimensions of the output array and read the shape and stride of the current dimension from the storage space;

[0150] A judgment subunit is used to judge whether the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have been traversed;

[0151] The remainder subunit is used to use the built-in remainder function of the offset calculation instance to perform a remainder operation on the shape and step size corresponding to the current index when the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have not been traversed. The obtained quotient is used as the index of the next dimension, and the product of the remainder and the step size is used as the offset corresponding to the current dimension;

[0152] A storage subunit, used for storing the offset corresponding to the current dimension into the offset register;

[0153] An add-one subunit is used to add one to the current dimension as the latest current dimension;

[0154] As a subunit, it is used to take the index of the next dimension as the latest current index and return the step of reading the shape and stride of the current dimension from the storage space;

[0155] The second reading subunit is used to terminate the traversal of the dimensions of the output array until the current dimension is greater than the maximum dimension value or all dimensions of the output array are traversed, and read the offsets corresponding to all dimensions recorded in the offset register;

[0156] The accumulation subunit is used to take the accumulated values ​​of the offsets corresponding to all dimensions as the offsets corresponding to the output array.

[0157] In some embodiments, further comprising a replication unit;

[0158] The copy unit is used to traverse the dimensions of the output array, read the shape and stride under the current dimension from the storage space, and then perform a copy operation on the elements of the input array to match the dimension of the output array if the dimension of the input array is greater than or equal to the dimension of the output array.

[0159] In some embodiments, the modulo subunit is used to perform a modulo operation on the shape and step size corresponding to the current index using the divmod function included in the offset calculation instance.

[0160] In some embodiments, further comprising a deployment unit;

[0161] A deployment unit is used to deploy a keyword of a unified computing device architecture in a template before instantiating the template according to an input tensor parameter to obtain an offset calculation instance, so that data can be calculated multiple times on a graphics processor without being transmitted back to a central processor.

[0162] In some embodiments, the operation unit is used to call a pointer arithmetic calculation formula to process the data corresponding to the offset to obtain an output result; wherein the pointer arithmetic calculation formula is as follows:

[0163] ;

[0164] in, Indicates the output result, Indicates the execution function. Represents the function corresponding to the operation rule, Indicates the pointer to the data corresponding to the offset. Indicates the offset.

[0165] In some embodiments, an adjustment unit is also included;

[0166] An adjustment unit is used to instantiate the template according to the input tensor parameters to obtain an offset calculation instance, and then, when a configuration adjustment instruction is obtained, adjust the initial template parameters contained in the offset calculation instance according to the new template parameters carried in the configuration adjustment instruction to obtain a new offset calculation instance; wherein the template parameters include the number of input arrays, the type of index and / or the step type.

[0167] Figure 3 The description of the features in the corresponding embodiments can be found in Figure 1 The relevant descriptions of the corresponding embodiments will not be repeated here one by one.

[0168] It can be seen from the above technical solution that the template is instantiated according to the input tensor parameters to obtain an offset calculation instance; wherein the tensor parameters include the shape, step, dimension and data corresponding to the input array and the output array. In order to avoid repeatedly passing and copying the same data in each operation, the offset calculation instance can be initialized to determine the step and shape of each input array in each output dimension, and the step and shape of each input array in each output dimension are recorded in the set storage space. The offset calculation instance is started to perform index calculation on the step and shape of each input array recorded in the storage space in each output dimension to obtain the offset corresponding to the output array. The data corresponding to the offset is calculated according to the set operation rules to obtain the output result. The beneficial effect of the present invention is that by pre-calculating the step and shape of each input array in each output dimension, unnecessary memory access can be reduced and the memory access efficiency can be improved. And by constructing the offset calculation instance, the index calculation process is simplified, which greatly reduces the calculation burden on the graphics processor. The entire implementation process does not require complex index calculations and dynamic memory access, but directly uses the calculated offset to quickly locate data, efficiently and accurately implementing data access and calculation. In addition, this direct access mode significantly improves the efficiency of parallel processing, allowing large-scale data processing tasks to be completed faster.

[0169] Figure 4 A structural diagram of a multi-dimensional data processing device provided by an embodiment of the present invention, such as Figure 4 As shown, the multi-dimensional data processing device includes: a memory 40 for storing a computer program;

[0170] The processor 41 is used to implement the steps of the multi-dimensional data processing method of the above embodiment when executing a computer program.

[0171] The multi-dimensional data processing device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.

[0172] Among them, the processor 41 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 41 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 41 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 41 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 41 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0173] The memory 40 may include one or more computer-readable storage media, which may be non-transitory. The memory 40 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 40 is at least used to store the following computer program 401, wherein the computer program is loaded and executed by the processor 41, and the relevant steps of the multi-dimensional data processing method disclosed in any of the above embodiments can be implemented. In addition, the resources stored in the memory 40 may also include an operating system 402 and data 403, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 402 may include Windows, Unix, Linux, etc. Data 403 may include but is not limited to the shape, step size, dimension and data corresponding to the input array and the output array, etc.

[0174] In some embodiments, the multi-dimensional data processing device may further include a display screen 42 , an input / output interface 43 , a communication interface 44 , a power supply 45 , and a communication bus 46 .

[0175] Those skilled in the art will understand that Figure 4 The structure shown in the figure does not constitute a limitation on the multi-dimensional data processing device, and may include more or fewer components than those shown in the figure.

[0176] It is understandable that if the multi-dimensional data processing method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or part of the contribution to the current technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, disk or optical disk, etc. Various media that can store program codes.

[0177] Based on this, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the multi-dimensional data processing method as described above are implemented.

[0178] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned multi-dimensional data processing method when executed by a processor.

[0179] The above is a detailed introduction to a multi-dimensional data processing method, device, equipment, computer-readable storage medium and computer program product provided by an embodiment of the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0180] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0181] The above is a detailed introduction to a multi-dimensional data processing method, device, equipment, computer-readable storage medium and computer program product provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A method for processing multi-dimensional data, characterized in that: include: Instantiate the template according to the input tensor parameters to obtain an offset calculation instance; wherein the tensor parameters include the shape, step size, dimension and data corresponding to the input array and the output array respectively; Performing an initialization operation on the offset calculation instance to determine the step length and shape of each input array in each output dimension, and recording the step length and shape of each input array in each output dimension in a set storage space; Starting the offset calculation instance to perform index calculation on the step length and shape of each input array recorded in the storage space in each output dimension, so as to obtain the offset corresponding to the output array; The data corresponding to the offset is operated according to the set operation rules to obtain an output result.

2. The method for processing multi-dimensional data according to claim 1, characterized in that: Starting the offset calculation instance to perform index calculation on the step length and shape of each input array recorded in the storage space on each output dimension to obtain the offset corresponding to the output array includes: Initialize an offset register of all zeros; wherein the offset register is used to store the offset; Traversing the dimensions of the output array, and reading the shape and stride of the current dimension from the storage space; Determine whether the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have been traversed; When the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have not been traversed, the shape and step size corresponding to the current index are modulo-operated using the built-in modulo function of the offset calculation instance, and the obtained quotient is used as the index of the next dimension, and the product of the remainder and the step size is used as the offset corresponding to the current dimension; Storing the offset corresponding to the current dimension into the offset register; Add one to the current dimension as the latest current dimension, and use the index of the next dimension as the latest current index, and return the step of reading the shape and step size of the current dimension from the storage space; Until the current dimension is greater than the maximum dimension value or all dimensions of the output array are traversed, the traversal of the dimensions of the output array is terminated, and the offsets corresponding to all dimensions recorded in the offset register are read; The accumulated value of the offsets corresponding to all dimensions is used as the offset corresponding to the output array.

3. The method for processing multi-dimensional data according to claim 2, characterized in that: After traversing the dimensions of the output array and reading the shape and stride of the current dimension from the storage space, it also includes: In the case where the dimension of the input array is greater than or equal to the dimension of the output array, a copy operation is performed on the elements of the input array to match the dimension of the output array.

4. The method for processing multi-dimensional data according to claim 2, characterized in that: Using the built-in modulo function of the offset calculation instance to perform a modulo operation on the shape and step size corresponding to the current index includes: The divmod function included in the offset calculation example is used to perform a modulo operation on the shape and step size corresponding to the current index.

5. The method for processing multi-dimensional data according to claim 1, characterized in that: Before instantiating the template according to the input tensor parameter to obtain the offset calculation instance, it also includes: The keywords of the unified computing device architecture are deployed in the template so that the data can be calculated on the graphics processor for multiple times without being transferred back to the central processor.

6. The method for processing multi-dimensional data according to claim 1, characterized in that: The data corresponding to the offset is operated according to the set operation rules to obtain output results including: The pointer arithmetic calculation formula is called to process the data corresponding to the offset to obtain an output result; wherein the pointer arithmetic calculation formula is as follows: ； in, Indicates the output result, Indicates the execution function. Represents the function corresponding to the operation rule, A pointer to the data corresponding to the offset, Indicates the offset.

7. The method for processing multi-dimensional data according to any one of claims 1 to 6, characterized in that: After instantiating the template according to the input tensor parameters to obtain the offset calculation instance, it also includes: When a configuration adjustment instruction is obtained, the initial template parameters contained in the offset calculation instance are adjusted according to the new template parameters carried in the configuration adjustment instruction to obtain a new offset calculation instance; wherein the template parameters include the number of input arrays, the type of index and / or the step type.

8. A multi-dimensional data processing device, characterized in that: It includes an instantiation unit, an initialization unit, an index calculation unit and an operation unit; The instantiation unit is used to instantiate the template according to the input tensor parameters to obtain an offset calculation instance; wherein the tensor parameters include the shape, step size, dimension and data corresponding to the input array and the output array respectively; The initialization unit is used to perform an initialization operation on the offset calculation instance to determine the step length and shape of each input array in each output dimension, and record the step length and shape of each input array in each output dimension in a set storage space; The index calculation unit is used to start the offset calculation instance to perform index calculation on the step length and shape of each input array recorded in the storage space in each output dimension, so as to obtain the offset corresponding to the output array; The operation unit is used to operate the data corresponding to the offset according to the set operation rules to obtain an output result.

9. A multi-dimensional data processing device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the multi-dimensional data processing method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for processing multi-dimensional data according to any one of claims 1 to 7 are implemented.

Images