A method, apparatus, device, and medium for processing multi-dimensional data

Abstract

This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and medium for processing multi-dimensional data. The method instantiates a template based on input tensor parameters to obtain an offset calculation instance. The offset calculation instance is initialized to determine the step size and shape of each input array in each output dimension, and these are recorded in a designated storage space. The offset calculation instance is then started to perform index calculations on the step size and shape of each input array recorded in the storage space in each output dimension to obtain the offset corresponding to the output array. The data corresponding to the offset is then processed according to a set calculation rule to obtain the output result. By pre-calculating the step size and shape of each input array in each output dimension, unnecessary memory accesses can be reduced. Constructing offset calculation instances simplifies the index calculation process, enabling efficient and accurate data storage, retrieval, and calculation.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and medium for processing multi-dimensional data. Background Technology

[0002] With the rapid advancement of deep learning and artificial intelligence technologies, the demand for high-performance computing capabilities capable of performing complex computational tasks has increased dramatically. In the context of Graphics Processing Units (GPUs), using Compute Unified Device Architecture (CUDA) for parallel computing has become a key means to improve computing speed.

[0003] In deep learning, tensors are a fundamental data structure that describes the layout of multidimensional arrays through attributes such as shape, strides, and ndim. During CUDA operator execution, the data often processed is not stored contiguously in memory, or if two tensors have different shapes, direct element-wise operations are usually impossible because their elements cannot be directly mapped. This means that during computation, the computer needs to constantly jump between different memory locations and cannot access data sequentially, otherwise out-of-bounds errors will occur. It is necessary to calculate the exact memory location (index) of each element based on attributes such as shape, strides, and ndim to achieve correct data reading and writing. Traditional index calculation methods often face problems such as out-of-bounds access, high computational complexity, and difficulty in parallelization, which limit the improvement of overall performance.

[0004] It is evident that how to efficiently and accurately store, retrieve, and compute data is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, device, and medium for processing multi-dimensional data, which can solve the problems of out-of-bounds access, high computational complexity, and difficulty in parallelization faced by index calculation.

[0006] To address the aforementioned technical problems, embodiments of the present invention provide a method for processing multi-dimensional data, comprising:

[0007] The template is instantiated based on the input tensor parameters to obtain the offset calculation instance; wherein, the tensor parameters include the shape, step size, dimension and data of the input array and the output array respectively;

[0008] The offset calculation instance is initialized to determine the step size and shape of each input array in each output dimension, and the step size and shape of each input array in each output dimension are recorded in the set storage space.

[0009] The offset calculation instance is started to perform index calculation on the step size and shape of each input array recorded in the storage space in each output dimension to obtain the offset corresponding to the output array;

[0010] The data corresponding to the offset is processed according to the set calculation rules to obtain the output result.

[0011] On one hand, the offset calculation instance is initiated to perform index calculations on the step size and shape of each input array recorded in the storage space in each output dimension, so as to obtain the offset corresponding to the output array, including:

[0012] Initialize an offset register filled with all zeros; wherein the offset register is used to store the offset.

[0013] Iterate through the dimensions of the output array and read the shape and step size of the current dimension from the storage space;

[0014] Determine if the current dimension is less than or equal to the maximum dimension value and if all dimensions of the output array have been traversed.

[0015] If the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have not been traversed, the modulo function built into the offset calculation instance is used to perform a modulo operation on the shape and step size corresponding to the current index. The resulting quotient is used as the index of the next dimension, and the product of the remainder and the step size is used as the offset corresponding to the current dimension.

[0016] Store the offset corresponding to the current dimension into the offset register;

[0017] Increment the current dimension by one to obtain the latest current dimension, and use the index of the next dimension as the latest current index. Return to the steps of reading the shape and step size under the current dimension from the storage space.

[0018] The traversal of the dimensions of the output array ends when the current dimension is greater than the maximum dimension value or all dimensions of the output array have been traversed, and the offsets corresponding to all dimensions recorded in the offset register are read.

[0019] The sum of the offsets corresponding to all dimensions is used as the offset of the output array.

[0020] On the one hand, after traversing the dimensions of the output array and reading the shape and step size in the current dimension from the storage space, the process also includes:

[0021] If the dimension of the input array is greater than or equal to the dimension of the output array, a copy operation is performed on the elements of the input array to match the dimension of the output array.

[0022] On the one hand, using the modulo function built into the offset calculation instance to perform a modulo operation on the shape and step size corresponding to the current index includes:

[0023] The divmod function included in the offset calculation instance is used to perform a modulo operation on the shape and step size corresponding to the current index.

[0024] On the one hand, before instantiating the template based on the input tensor parameters to obtain the offset calculation instance, the process also includes:

[0025] The template deploys keywords for a unified computing device architecture, enabling data to undergo multiple computations on the graphics processor without needing to be sent back to the central processing unit.

[0026] On the one hand, the data corresponding to the offset is processed according to the set calculation rules to obtain the output results, including:

[0027] The pointer arithmetic calculation formula is used to process the data corresponding to the offset to obtain the output result; wherein, the pointer arithmetic calculation formula is as follows:

[0028] ;

[0029] in, Indicates the output result. Indicates the function to be executed. This represents the function corresponding to the operation rule. A pointer to the data corresponding to the offset. This indicates the offset.

[0030] On the one hand, after instantiating the template based on the input tensor parameters to obtain the offset calculation instance, it also includes:

[0031] Upon receiving a configuration adjustment instruction, the initial template parameters contained in the offset calculation instance are adjusted according to the new template parameters carried in the configuration adjustment instruction to obtain a new offset calculation instance; wherein, the template parameters include the number of input arrays, the type of index, and / or the step type.

[0032] This invention also provides a multi-dimensional data processing device, including an instantiation unit, an initialization unit, an index calculation unit, and a computation unit;

[0033] The instantiation unit is used to instantiate the template according to the input tensor parameters to obtain the offset calculation instance; wherein, the tensor parameters include the shape, step size, dimension and data corresponding to the input array and the output array respectively;

[0034] The initialization unit is used to perform an initialization operation on the offset calculation instance to determine the step size and shape of each input array in each output dimension, and record the step size and shape of each input array in each output dimension in a set storage space.

[0035] The index calculation unit is used to initiate the offset calculation instance to perform index calculation on the step size and shape of each input array recorded in the storage space in each output dimension, so as to obtain the offset corresponding to the output array;

[0036] The calculation unit is used to perform calculations on the data corresponding to the offset according to the set calculation rules to obtain the output result.

[0037] On one hand, the index calculation unit includes an initialization subunit, a first reading subunit, a judgment subunit, a modulo subunit, a storage subunit, an increment subunit, an as-a subunit, a second reading subunit, and an accumulation subunit;

[0038] The initialization subunit is used to initialize an offset register with all zeros; wherein, the offset register is used to store the offset.

[0039] The first reading subunit is used to traverse the dimensions of the output array and read the shape and step size in the current dimension from the storage space;

[0040] The judgment subunit is used to determine whether the current dimension is less than or equal to the maximum dimension value and whether all dimensions of the output array have been traversed.

[0041] The modulo subunit is used to perform a modulo operation on the shape and step size corresponding to the current index using the modulo function built into the offset calculation instance when the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have not been traversed. The quotient value is used as the index of the next dimension, and the product of the remainder and the step size is used as the offset corresponding to the current dimension.

[0042] The storage subunit is used to store the offset corresponding to the current dimension into the offset register;

[0043] The increment subunit is used to increment the current dimension by one to obtain the latest current dimension;

[0044] The sub-unit is used to take the index of the next dimension as the latest current index and return the step of reading the shape and step size of the current dimension from the storage space;

[0045] The second reading subunit is used to end the traversal of the dimensions of the output array until the current dimension is greater than the maximum dimension value or all dimensions of the output array have been traversed, and to read the offsets corresponding to all dimensions recorded in the offset register.

[0046] The accumulation sub-unit is used to accumulate the offset values ​​corresponding to all dimensions as the offset values ​​corresponding to the output array.

[0047] On the one hand, it also includes a replication unit;

[0048] The copying unit is used to traverse the dimensions of the output array, read the shape and step size of the current dimension from the storage space, and then, if the dimension of the input array is greater than or equal to the dimension of the output array, perform a copying operation on the elements of the input array to match the dimension of the output array.

[0049] On the one hand, the remainder subunit is used to perform a remainder operation on the shape and step size corresponding to the current index using the divmod function contained in the offset calculation instance.

[0050] On the one hand, it also includes deployment units;

[0051] The deployment unit is used to deploy keywords of a unified computing device architecture in the template before instantiating the template according to the input tensor parameters to obtain the offset calculation instance, so that the data can be calculated multiple times on the graphics processor without being sent back to the central processing unit.

[0052] On one hand, the arithmetic unit is used to call the pointer arithmetic calculation formula to process the data corresponding to the offset in order to obtain the output result; wherein, the pointer arithmetic calculation formula is as follows:

[0053] ;

[0054] in, Indicates the output result. Indicates the function to be executed. This represents the function corresponding to the operation rule. A pointer to the data corresponding to the offset. This indicates the offset.

[0055] On the one hand, it also includes adjustment units;

[0056] The adjustment unit is used to, after instantiating the template according to the input tensor parameters to obtain an offset calculation instance, adjust the initial template parameters contained in the offset calculation instance according to the new template parameters carried in the configuration adjustment instruction when a configuration adjustment instruction is obtained, so as to obtain a new offset calculation instance; wherein, the template parameters include the number of input arrays, the type of index and / or the step type.

[0057] This invention also provides a multi-dimensional data processing device, comprising:

[0058] Memory, used to store computer programs;

[0059] A processor for executing the computer program to implement the steps of the multi-dimensional data processing method described above.

[0060] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the multi-dimensional data processing method described above.

[0061] This invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-described multi-dimensional data processing method.

[0062] As can be seen from the above technical solution, the template is instantiated based on the input tensor parameters to obtain an offset calculation instance. The tensor parameters include the shape, step size, dimension, and data corresponding to the input and output arrays, respectively. To avoid repeatedly transmitting and copying the same data in each operation, the offset calculation instance can be initialized to determine the step size and shape of each input array in each output dimension, and these dimensions are recorded in a designated storage space. The offset calculation instance is then started to perform index calculations on the step size and shape of each input array recorded in the storage space in each output dimension to obtain the offset corresponding to the output array. The data corresponding to the offset is then calculated according to the set calculation rules to obtain the output result. The beneficial effect of this invention is that by pre-calculating the step size and shape of each input array in each output dimension, unnecessary memory accesses can be reduced, improving memory access efficiency. Furthermore, constructing an offset calculation instance simplifies the index calculation process, greatly reducing the computational burden on the graphics processor. The entire implementation process eliminates the need for complex index calculations and dynamic memory access. Instead, it directly utilizes the calculated offsets to quickly locate data, achieving efficient and accurate data storage, retrieval, and computation. Furthermore, this direct access mode significantly improves the efficiency of parallel processing, enabling large-scale data processing tasks to be completed much faster. Attached Figure Description

[0063] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This invention provides a method for processing multi-dimensional data.

[0065] Figure 2 A flowchart illustrating a method for indexing the step size and shape of each input array in each output dimension, provided by an embodiment of the present invention;

[0066] Figure 3 This is a schematic diagram of the structure of a multi-dimensional data processing device provided in an embodiment of the present invention;

[0067] Figure 4 This is a structural diagram of a multi-dimensional data processing device provided in an embodiment of the present invention. Detailed Implementation

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0069] The terms "comprising" and "having," and any variations thereof, in the specification and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may include steps or units not listed.

[0070] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0071] CUDA, as a parallel computing platform and programming model, is widely used in the field of deep learning because it can fully utilize the powerful computing capabilities of GPUs. Currently, to determine the exact location of each element in memory, a common practice is to transfer the shape and strides information of the Tensor from the CPU to the GPU and pass it directly as attribute parameters to the CUDA kernel function. Inside the kernel function, the offset of each element is dynamically calculated through complex arithmetic operations.

[0072] While this method is simple to implement, it suffers from significant performance bottlenecks. Transferring the shape and strides information of the Tensor from the CPU to the GPU and performing complex index calculations within the CUDA kernel function introduces additional performance overhead due to the need to transfer extra metadata (i.e., shape and strides) across devices (CPU to GPU).

[0073] Because GPUs excel at parallel computing for large-scale, simple operations, complex indexing calculations are more common in operators that require accessing or manipulating across elements, rows, or columns, such as slicing and transposing. Frequent complex indexing operations significantly increase the computational burden on a single thread. These complex arithmetic operations typically involve multiple multiplications, divisions, and modulo operations. While these operations can be executed in parallel on a GPU, each thread still needs to perform these operations, resulting in a massive overall computational load and reduced parallel efficiency.

[0074] Therefore, embodiments of the present invention provide a method, apparatus, device and medium for processing multi-dimensional data. By encapsulating high-performance offset calculation instances, it can support offset calculation of multi-dimensional arrays (tensors), thereby simplifying and optimizing broadcast operations in scenarios such as deep learning and scientific computing.

[0075] The solutions provided in this invention are not only applicable to the field of Artificial Intelligence (AI), but can also be applied to other fields that require processing multidimensional data and complex data formats, such as scientific computing, image processing, and signal processing. Through similar implementation methods, the efficiency of data processing and computation in these fields can be expanded and optimized.

[0076] Next, we will describe in detail a method for processing multi-dimensional data provided by an embodiment of the present invention. Figure 1 A method for processing multi-dimensional data provided in this embodiment of the invention includes:

[0077] S101: Instantiate the template based on the input tensor parameters to obtain an offset calculation instance.

[0078] The tensor parameters can include the shape, stride, dimension, and data of the input and output arrays, respectively.

[0079] Multidimensional data often involves multiple parameter types, and the index calculation method for each type of parameter is similar. We will use the index calculation of one type of parameter as an example for explanation. Considering that the number of parameters of each type is often large, the number of input parameters of each type can determine the number of input arrays. For example, if the number of input parameters of the same type is 2, then there can be two input arrays: input array 1 and input array 2.

[0080] In this embodiment of the invention, a well-encapsulated, high-performance offset calculator tool focused on automatic broadcasting is designed, which can support index and offset calculation of multidimensional arrays.

[0081] Initially, the offset calculation tool is stored as a template. The template cannot be used directly. It needs to be instantiated according to the actual application requirements to obtain an instance that can be used for offset calculation. This instance can be called an offset calculation instance.

[0082] In practical applications, the `MakeOffsetCalculator` template function can be used to construct an offset calculation instance, i.e., an `OffsetCalculator` instance, based on the input tensor parameters (such as x1, x2, y). Here, x1 represents the tensor parameters of input array 1, x2 represents the tensor parameters of input array 2, and y represents the tensor parameters of the output array. These tensor parameters contain information such as shape, stride, dimensions, and data.

[0083] S102: Initialize the offset calculation instance to determine the step size and shape of each input array in each output dimension, and record the step size and shape of each input array in each output dimension in the set storage space.

[0084] In this embodiment of the invention, in order to avoid repeatedly transmitting and copying the same data in each operation, the offset calculation instance can be initialized to determine the step size and shape of each input array in each output dimension, and the step size and shape of each input array in each output dimension can be recorded in the set storage space (all_sizes_).

[0085] The OffsetCalculator tool is presented as an OffsetCalculator struct. The OffsetCalculator struct implements a tool for calculating the element offsets of multidimensional arrays during broadcasting or similar operations. The broadcasting mechanism allows for the automatic expansion of smaller tensors to match larger tensors.

[0086] The `OffsetCalculator` structure contains the following parameters: `input_dims1`, `input_dims2`, `output_dims`, `all_sizes_`, and `strides_`. `input_dims1` represents the number of dimensions of input array 1, `input_dims2` represents the number of dimensions of input array 2, and `output_dims` represents the number of dimensions of the output array. `all_sizes_` is a two-dimensional array that stores information about each input array in each output dimension, expressed as `IntDivider`.<index_t> The object takes the form of `IntDivider`, a custom class for efficient integer division and modulo operations. `strides_` is a two-dimensional array that stores the stride of each input array in each output dimension. The stride takes into account the element size to correctly handle arrays of different data types.

[0087] In practical applications, the constructor of the OffsetCalculator structure can accept the number of dimensions of the input array, the number of dimensions of the output array, the shapes array, the strides array, and an optional element sizes array. By initializing input_dims1 and output_dims, and iterating through the dimensions of the output array, the strides and shapes of each input array in each output dimension can be calculated, and this information can be stored in all_sizes_.

[0088] S103: Start the offset calculation instance to calculate the step size and shape of each input array recorded in the storage space on each output dimension to obtain the offset of the output array.

[0089] The GetBroadCast function is the core of the offset calculation example. The input to the GetBroadCast function can be a linear index (linear_idx). Given a linear index, the shape and step size of the input array for each output dimension can be read from all_sizes_. By performing a modulo operation on the shape and step size for each dimension, the offset of each element in the input array in the output array can be output.

[0090] S104: Perform calculations on the data corresponding to the offset according to the set calculation rules to obtain the output result.

[0091] After determining the offset of the output array, the corresponding data can be read based on the offset.

[0092] In practical applications, the pointer arithmetic calculation formula can be used to process the data corresponding to the offset to obtain the output result; the pointer arithmetic calculation formula is as follows:

[0093] ;

[0094] in, Indicates the output result. Indicates the function to be executed. This represents the function corresponding to the operation rule. A pointer to the data corresponding to the offset. This indicates the offset.

[0095] The function corresponding to the operation rule can be determined based on the actual operation requirements. For example, the function corresponding to the operation rule can be an absolute value function or a comparison function.

[0096] As can be seen from the above technical solution, the template is instantiated based on the input tensor parameters to obtain an offset calculation instance. The tensor parameters include the shape, step size, dimension, and data corresponding to the input and output arrays, respectively. To avoid repeatedly transmitting and copying the same data in each operation, the offset calculation instance can be initialized to determine the step size and shape of each input array in each output dimension, and these dimensions are recorded in a designated storage space. The offset calculation instance is then started to perform index calculations on the step size and shape of each input array recorded in the storage space in each output dimension to obtain the offset corresponding to the output array. The data corresponding to the offset is then calculated according to the set calculation rules to obtain the output result. The beneficial effect of this invention is that by pre-calculating the step size and shape of each input array in each output dimension, unnecessary memory accesses can be reduced, improving memory access efficiency. Furthermore, constructing an offset calculation instance simplifies the index calculation process, greatly reducing the computational burden on the graphics processor. The entire implementation process eliminates the need for complex index calculations and dynamic memory access. Instead, it directly utilizes the calculated offsets to quickly locate data, achieving efficient and accurate data storage, retrieval, and computation. Furthermore, this direct access mode significantly improves the efficiency of parallel processing, enabling large-scale data processing tasks to be completed much faster.

[0097] Figure 2 A flowchart of a method for indexing the step size and shape of each input array in each output dimension, provided by an embodiment of the present invention, is provided. The method includes:

[0098] S201: Initialize an offset register with all zeros.

[0099] The offset register can be used to store the offset.

[0100] In practical applications, an offset (offset_type) object with all zeros can be initialized, namely the offset register (offsets), to store the offset.

[0101] S202: Traverse the dimensions of the output array and read the shape and step size of the current dimension from the storage space.

[0102] In this embodiment of the invention, the offset corresponding to each dimension can be determined by traversing the dimensions of the output array.

[0103] To calculate the offset for each dimension, we can use the shape and step size information recorded in `all_sizes_` to perform fast division, thus obtaining the offset for each dimension. The offsets for all dimensions are then summed to obtain the final offset for the output array.

[0104] S203: Determine if the current dimension is less than or equal to the maximum dimension value and if all dimensions of the output array have been traversed.

[0105] In this embodiment of the invention, traversing the dimensions of the output array is a loop iteration process, and each calculation of the offset under a dimension constitutes one loop. In order to end the loop process in a timely manner, it can be determined whether the current dimension is less than or equal to the maximum dimension value and whether all dimensions of the output array have been traversed.

[0106] If the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have not been traversed, it indicates that there are still dimensions whose offsets have not been calculated. In this case, operation S204 can be executed. If the current dimension is greater than the maximum dimension value or all dimensions of the output array have been traversed, operation S207 is executed.

[0107] S204: Utilize the built-in modulo function of the offset calculation instance to perform a modulo operation on the shape and step size corresponding to the current index, use the resulting quotient as the index of the next dimension, and use the product of the remainder and the step size as the offset corresponding to the current dimension.

[0108] In this embodiment of the invention, the divmod function included in the offset calculation instance can be used to perform a modulo operation on the shape and step size corresponding to the current index.

[0109] The `divmod` function performs a modulo operation on the shape and step size corresponding to the current index (`linear_idx`). The quotient is used to update `linear_idx` to continue the calculation for the next dimension, and the remainder is used to calculate the offset of each input array in the current dimension. The offset in the current dimension can be the product of the remainder and the step size.

[0110] S205: Store the offset corresponding to the current dimension into the offset register.

[0111] In this embodiment of the invention, it is necessary to sum the offsets under all dimensions. Therefore, the offset corresponding to each dimension can be stored in the offset register.

[0112] S206: Increment the current dimension by one to get the latest current dimension, and use the index of the next dimension as the latest current index. Return the steps for reading the shape and step size of the current dimension from the storage space.

[0113] The offset corresponding to a dimension can be calculated by following the operations from S202 to S205. After calculating the offset for each dimension, the next round of the loop needs to be executed. At this time, the current dimension can be incremented by one to obtain the latest current dimension, and the index of the next dimension can be used as the latest current index. Then, return to S202 to perform the operation of reading the shape and step size of the current dimension from the storage space.

[0114] S207: End the traversal of the dimensions of the output array and read the offsets corresponding to all dimensions recorded in the offset register.

[0115] If the current dimension is greater than the maximum dimension value or all dimensions of the output array have been traversed, the traversal of the dimensions of the output array can be ended. At this time, the offsets corresponding to all dimensions recorded in the offset register can be read.

[0116] S208: The sum of the offsets corresponding to all dimensions is used as the offset of the output array.

[0117] In this embodiment of the invention, by traversing the dimensions of the output array, the offset corresponding to each dimension can be calculated. The final offset is obtained by summing the offsets of all dimensions. The entire implementation process does not require complex calculations, achieving rapid determination of the offset of a multi-dimensional array.

[0118] Considering that the shapes of the input array and the output array are not exactly the same in practical applications, in order to enable operations to be performed between tensors of different shapes, the smaller shape tensor can be automatically expanded according to the broadcast mechanism to match the larger shape tensor.

[0119] Therefore, after iterating through the dimensions of the output array and reading the shape and step size of the current dimension from the storage space, it is possible to determine whether the dimension of the input array is greater than or equal to the dimension of the output array.

[0120] If the dimension of the input array is greater than or equal to the dimension of the output array, the elements of the input array can be copied to match the dimension of the output array.

[0121] In deep learning frameworks, it is often necessary to perform operations on multiple tensors of different shapes, such as addition and multiplication. These operations typically require automatic handling of the tensor shapes so that they can be operated on element-wise, even if their shapes are not exactly the same. This is usually achieved through a broadcast mechanism, which automatically expands smaller tensors to match larger ones.

[0122] For example, if the shape is divided into (10) and (2,10) tensors for operation, (10) will be implicitly expanded to the size of (2,10), which means it is copied twice. This expansion operation does not need to be reflected. Just take the numbers according to the shape of (2,10).

[0123] In this embodiment of the invention, OffsetCalculator incorporates a powerful GetBroadcast function, which automates the broadcast mechanism. In element-level operations, when encountering tensors with different shapes, GetBroadcast intelligently identifies and automatically expands the dimensions of the smaller tensor to match the shape of the larger tensor. This feature not only simplifies the code logic and improves its flexibility but also enables OffsetCalculator to be widely applied in various tensor calculation scenarios requiring broadcast operations, ensuring that out-of-bounds access does not occur during broadcasting and greatly enhancing code usability.

[0124] To improve code execution speed, before instantiating the template based on the input tensor parameters to obtain the offset calculation instance, keywords for a unified computing device architecture can be deployed in the template, so that data can be calculated multiple times on the graphics processor without being sent back to the central processing unit.

[0125] Keywords for a unified computing device architecture can include CUDA programming keywords such as __host__, __device__, and __forceinline__. __host__ and __device__ indicate that they can run on either a CPU or a GPU (if using CUDA). This allows the architecture to fully utilize the parallel computing capabilities of GPUs.

[0126] For example, after obtaining an OffsetCalculator instance, you can call the LaunchArithmeticKernel function to start a parallel computing kernel on a specified CUDA stream. The kernel function is a __host__ __device__ lambda expression that uses the OffsetCalculator instance to calculate the offset and performs the actual arithmetic operations.

[0127] Lambda expressions are marked with `__host__` and `__device__`, meaning they can be executed on either the host (CPU) or the device (GPU). However, in CUDA programming, lambda expressions are typically translated into a CUDA kernel and executed on the GPU. `numel` represents the total number of elements to be processed, which determines how many iterations the parallel computation kernel will perform. Within a lambda expression, the offsets of all tensors at a given index (`idx`) are calculated by calling `offset_calc.GetBroadcast(idx)`.

[0128] While `__host__` and `__device__` themselves don't directly reduce data transfer, they allow developers to manipulate data stored in GPU memory directly on the GPU. This helps reduce unnecessary data transfers because data can be computed multiple times on the GPU without being sent back to the CPU. Reducing data transfer can significantly decrease program runtime, as data transfer is typically more time-consuming than computation.

[0129] The `__forceinline__` flag instructs the compiler to inline the method whenever possible.

[0130] Inlining refers to the function being expanded to each call point at compile time, rather than being called like a normal function. This reduces the overhead of function calls and improves program execution efficiency, especially in the GPU's Single Instruction Multiple Data (SIMD) execution model.

[0131] In this embodiment of the invention, the OffsetCalculator tool provides flexible options for setting template parameters and constructor parameters, allowing users to easily adjust the configuration of OffsetCalculator according to specific application scenarios and needs.

[0132] Upon receiving a configuration adjustment instruction, the initial template parameters contained in the offset calculation instance can be adjusted according to the new template parameters carried in the configuration adjustment instruction to obtain a new offset calculation instance.

[0133] The template parameters may include the number of input arrays (NARGS), the type of index (index_t), and / or the type of step (signed_strides).

[0134] NARGS is a compile-time parameter that allows OffsetCalculator to handle different numbers of input arrays.

[0135] The default value for `index_t` is `uint32_t`. Users can choose different integer types as the index as needed.

[0136] `signed_strides` is a boolean value indicating whether the strides should be a signed type. If true, the strides will be a signed version of `index_t`; otherwise, they will be `index_t` itself.

[0137] By configuring template parameters, OffsetCalculator can flexibly support different numbers of inputs, different index types, and whether signed step sizes need to be handled, thus meeting different application scenarios and requirements.

[0138] In this embodiment of the invention, offset calculation instances matching different application scenarios can be pre-established for common application scenarios. In practical applications, the template can be instantiated according to the tensor parameters corresponding to each application scenario to obtain the offset calculation instance corresponding to each application scenario.

[0139] To facilitate subsequent retrieval and use, the offset calculation instances matched for each application scenario can be stored in the designated memory space.

[0140] Upon receiving a scene switching command, a target offset calculation instance matching the scene identifier carried in the scene switching command can be retrieved from the designated memory space. This target offset calculation instance is then used to perform index calculations on the data in the current scene.

[0141] By pre-storing offset calculation instances corresponding to various common application scenarios, the matching offset calculation instance can be directly called when switching application scenarios, without having to re-instantiate the template every time a scenario is switched.

[0142] Figure 3 This is a schematic diagram of a multi-dimensional data processing device provided in an embodiment of the present invention, including an instantiation unit 31, an initialization unit 32, an index calculation unit 33, and a calculation unit 34.

[0143] Instantiation unit 31 is used to instantiate the template according to the input tensor parameters to obtain the offset calculation instance; wherein, the tensor parameters include the shape, step size, dimension and data of the input array and the output array respectively;

[0144] Initialization unit 32 is used to perform initialization operations on the offset calculation instance to determine the step size and shape of each input array in each output dimension, and record the step size and shape of each input array in each output dimension in the set storage space;

[0145] The index calculation unit 33 is used to initiate an offset calculation instance to perform index calculation on the step size and shape of each input array recorded in the storage space in each output dimension, so as to obtain the offset corresponding to the output array.

[0146] The arithmetic unit 34 is used to perform calculations on the data corresponding to the offset according to the set calculation rules to obtain the output result.

[0147] In some embodiments, the index calculation unit includes an initialization subunit, a first reading subunit, a judgment subunit, a remainder subunit, a storage subunit, an increment subunit, an assimilation subunit, a second reading subunit, and an accumulation subunit.

[0148] The initialization sub-unit is used to initialize an offset register with all zeros; the offset register is used to store the offset.

[0149] The first reading subunit is used to traverse the dimensions of the output array and read the shape and step size in the current dimension from the storage space;

[0150] The judgment sub-unit is used to determine whether the current dimension is less than or equal to the maximum dimension value and whether all dimensions of the output array have been traversed.

[0151] The modulo sub-unit is used when the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have not been traversed. It uses the modulo function built into the offset calculation instance to perform a modulo operation on the shape and step size corresponding to the current index, and uses the quotient as the index of the next dimension, and the product of the remainder and the step size as the offset corresponding to the current dimension.

[0152] The storage sub-unit is used to store the offset corresponding to the current dimension into the offset register;

[0153] Add one sub-unit, used to increment the current dimension by one to obtain the latest current dimension;

[0154] As a sub-unit, it is used to take the index of the next dimension as the latest current index and return the steps to read the shape and step size of the current dimension from the storage space;

[0155] The second reading subunit is used to end the traversal of the dimensions of the output array until the current dimension is greater than the maximum dimension value or all dimensions of the output array have been traversed, and to read the offsets corresponding to all dimensions recorded in the offset register.

[0156] The accumulator sub-unit is used to accumulate the offsets corresponding to all dimensions and use them as the offsets of the output array.

[0157] In some embodiments, a copying unit is also included;

[0158] The copy unit is used to traverse the dimensions of the output array. After reading the shape and step size of the current dimension from the storage space, it performs a copy operation on the elements of the input array to match the dimensions of the output array if the dimensions of the input array are greater than or equal to the dimensions of the output array.

[0159] In some embodiments, the remainder subunit is used to perform a remainder operation on the shape and step size corresponding to the current index using the divmod function contained in the offset calculation instance.

[0160] In some embodiments, a deployment unit is also included;

[0161] The deployment unit is used to deploy keywords of a unified computing device architecture in the template before instantiating the template based on the input tensor parameters to obtain the offset calculation instance, so that the data can be calculated multiple times on the graphics processor without being sent back to the central processing unit.

[0162] In some embodiments, the arithmetic unit is used to call the pointer arithmetic calculation formula to process the data corresponding to the offset in order to obtain the output result; wherein, the pointer arithmetic calculation formula is as follows:

[0163] ;

[0164] in, Indicates the output result. Indicates the function to be executed. This represents the function corresponding to the operation rule. A pointer to the data corresponding to the offset. This indicates the offset.

[0165] In some embodiments, an adjustment unit is also included;

[0166] The adjustment unit is used to adjust the initial template parameters contained in the offset calculation instance according to the new template parameters carried in the configuration adjustment instruction after instantiating the template based on the input tensor parameters to obtain the offset calculation instance, in order to obtain a new offset calculation instance; wherein, the template parameters include the number of input arrays, the type of index and / or the step type.

[0167] Figure 3 For a description of the features in the corresponding embodiments, please refer to Figure 1 The relevant descriptions of the corresponding embodiments will not be repeated here.

[0168] As can be seen from the above technical solution, the template is instantiated based on the input tensor parameters to obtain an offset calculation instance. The tensor parameters include the shape, step size, dimension, and data corresponding to the input and output arrays, respectively. To avoid repeatedly transmitting and copying the same data in each operation, the offset calculation instance can be initialized to determine the step size and shape of each input array in each output dimension, and these dimensions are recorded in a designated storage space. The offset calculation instance is then started to perform index calculations on the step size and shape of each input array recorded in the storage space in each output dimension to obtain the offset corresponding to the output array. The data corresponding to the offset is then calculated according to the set calculation rules to obtain the output result. The beneficial effect of this invention is that by pre-calculating the step size and shape of each input array in each output dimension, unnecessary memory accesses can be reduced, improving memory access efficiency. Furthermore, constructing an offset calculation instance simplifies the index calculation process, greatly reducing the computational burden on the graphics processor. The entire implementation process eliminates the need for complex index calculations and dynamic memory access. Instead, it directly utilizes the calculated offsets to quickly locate data, achieving efficient and accurate data storage, retrieval, and computation. Furthermore, this direct access mode significantly improves the efficiency of parallel processing, enabling large-scale data processing tasks to be completed much faster.

[0169] Figure 4 A structural diagram of a multi-dimensional data processing device provided in an embodiment of the present invention is shown below. Figure 4 As shown, the multi-dimensional data processing device includes: a memory 40 for storing computer programs;

[0170] The processor 41 is used to execute a computer program to implement the steps of the multi-dimensional data processing method as described in the above embodiments.

[0171] The multi-dimensional data processing device provided in this embodiment may include, but is not limited to, smartphones, tablets, laptops, or desktop computers.

[0172] The processor 41 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 41 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 41 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 41 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 41 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0173] The memory 40 may include one or more computer-readable storage media, which may be non-transitory. The memory 40 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 40 is used to store at least the following computer program 401, which, after being loaded and executed by the processor 41, is capable of implementing the relevant steps of the multi-dimensional data processing method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 40 may also include an operating system 402 and data 403, and the storage method may be temporary or permanent storage. The operating system 402 may include Windows, Unix, Linux, etc. The data 403 may include, but is not limited to, the shape, step size, dimensions, and data corresponding to the input array and output array, respectively.

[0174] In some embodiments, the multi-dimensional data processing device may further include a display screen 42, an input / output interface 43, a communication interface 44, a power supply 45, and a communication bus 46.

[0175] Those skilled in the art will understand that Figure 4 The structure shown does not constitute a limitation on the device for processing multidimensional data and may include more or fewer components than illustrated.

[0176] It is understood that if the multi-dimensional data processing methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the current technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, magnetic disks, or optical disks, and other media capable of storing program code.

[0177] Based on this, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the multi-dimensional data processing method described above.

[0178] This invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-described multi-dimensional data processing method.

[0179] The foregoing has provided a detailed description of a multi-dimensional data processing method, apparatus, device, computer-readable storage medium, and computer program product provided by embodiments of the present invention. The various embodiments are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0180] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0181] The foregoing has provided a detailed description of a multi-dimensional data processing method, apparatus, device, computer-readable storage medium, and computer program product provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A method of processing multi-dimensional data, characterized by, The method comprises the following steps: instantiating a template according to input tensor parameters to obtain an offset calculation instance; wherein the tensor parameters comprise shapes, strides, dimensions and data of input arrays and output arrays respectively; initializing the offset calculation instance to determine the stride and shape of each input array in each output dimension and record the stride and shape of each input array in each output dimension in a set storage space; starting the offset calculation instance to perform index calculation on the stride and shape of each input array in each output dimension recorded in the storage space to obtain the offset corresponding to the output array; performing operation on the data corresponding to the offset according to a set operation rule to obtain an output result.

2. The method of claim 1, wherein, starting the offset calculation instance to perform index calculation on the stride and shape of each input array in each output dimension recorded in the storage space to obtain the offset corresponding to the output array comprises: initializing an offset register full of zeros; wherein the offset register is used to store an offset; traversing the dimensions of the output array to read the shape and stride under the current dimension from the storage space; determining whether the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have been traversed; in the case that the current dimension is less than or equal to the maximum dimension value and all dimensions of the output array have not been traversed, performing modulo operation on the shape and stride corresponding to the current index by using a modulo function built in the offset calculation instance, taking the quotient as the index of the next dimension, and taking the product of the remainder and the stride as the offset corresponding to the current dimension; storing the offset corresponding to the current dimension to the offset register; taking the current dimension plus one as the latest current dimension, taking the index of the next dimension as the latest current index, and returning to the step of reading the shape and stride under the current dimension from the storage space; until the current dimension is greater than the maximum dimension value or all dimensions of the output array have been traversed, ending the traversal of the dimensions of the output array, and reading the offsets corresponding to all dimensions recorded in the offset register; taking the cumulative value of the offsets corresponding to all dimensions as the offset corresponding to the output array.

3. The method of claim 2, wherein, After traversing the dimensions of the output array to read the shape and stride under the current dimension from the storage space, the method further comprises the following steps: in the case that the dimension of the input array is greater than or equal to the dimension of the output array, performing copy operation on the elements of the input array to match the dimension of the output array.

4. The method of claim 2, wherein, performing modulo operation on the shape and stride corresponding to the current index by using a modulo function built in the offset calculation instance comprises: performing modulo operation on the shape and stride corresponding to the current index by using a divmod function contained in the offset calculation instance.

5. The method of claim 1, wherein, Before instantiating the template according to the input tensor parameters to obtain the offset calculation instance, the method further comprises the following steps: deploying a keyword of a unified computing device architecture in the template to enable data to perform multiple calculations on a graphics processing unit and not to be transmitted back to a central processing unit.

6. The method of claim 1, wherein, According to the set operation rule, the data corresponding to the offset is operated to obtain an output result, comprising: The pointer arithmetic calculation formula is called to process the data corresponding to the offset to obtain an output result; wherein the pointer arithmetic calculation formula is as follows: ； wherein, represents an output result, represents an execution function, represents a function corresponding to an operation rule, represents a pointer to data corresponding to the offset, represents the offset.

7. The method of claim 1 to 6, wherein After the template is instantiated according to the input tensor parameter to obtain the offset calculation instance, further comprising: In the case of obtaining the configuration adjustment instruction, the initial template parameter contained in the offset calculation instance is adjusted according to the new template parameter carried in the configuration adjustment instruction to obtain a new offset calculation instance; wherein the template parameter includes the number of input arrays, the type of index and / or the type of step length.

8. A processing device of multi-dimensional data, characterized by, Comprising an instantiation unit, an initialization unit, an index calculation unit and an operation unit; The instantiation unit is used for instantiating a template according to an input tensor parameter to obtain an offset calculation instance; wherein the tensor parameter includes the shape, step length, dimension and data of each input array and output array corresponding thereto; The initialization unit is used for initializing the offset calculation instance to determine the step length and shape of each input array on each output dimension, and records the step length and shape of each input array on each output dimension in a set storage space; The index calculation unit is used for starting the offset calculation instance to perform index calculation on the step length and shape of each input array on each output dimension recorded in the storage space to obtain the offset corresponding to the output array; The operation unit is used for operating the data corresponding to the offset according to the set operation rule to obtain an output result.

9. A processing device of multi-dimensional data, characterized by, Comprising: A memory for storing a computer program; A processor for executing the computer program to realize the steps of the multi-dimensional data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the steps of the multi-dimensional data processing method according to any one of claims 1 to 7.

Images