Data processing method, data processing apparatus, electronic device, and storage medium

Abstract

Embodiments of the present disclosure provide a data processing method, a data processing apparatus, an electronic device and a storage medium. The data processing method comprises: receiving a signal about an input tensor and an index tensor; reading target data from a source storage space corresponding to the input tensor based on the index tensor through a general tensor operation interface, and writing the target data into a destination storage space corresponding to an output tensor, wherein the general tensor operation interface supports unified tensor operations involving at least two data arrangement modes. The data processing method proposes a general and efficient index operation implementation scheme, which can adapt to different data arrangement modes and meet the complex index access requirements under different hardware architectures, thereby significantly improving the generalization ability and execution efficiency of the index operator and supporting efficient and low-latency index operations on heterogeneous computing devices.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of artificial intelligence technology, and in particular to a data processing method, a data processing apparatus, an electronic device, and a storage medium. Background Technology

[0002] Tensor indexing is a common operation in fields such as deep learning, image processing, and large-scale data computation. For example, in feature extraction, label mapping, or attention mechanisms, it is often necessary to extract data from specific positions in a source tensor (input tensor) based on a set of indices to form a new target tensor (output tensor).

[0003] The index operator is a high-level indexing operation used to extract data from input tensors of arbitrary dimensions. This operation supports data extraction along any dimension of the input tensor according to a given integer index tensor, and can be flexibly applied to data preprocessing and specific modeling tasks.

[0004] As the model size continues to expand and the complexity of operators continues to increase, optimizing the efficiency of indexing operations has become an urgent problem to be solved. Summary of the Invention

[0005] At least one embodiment of this disclosure provides a data processing method, which includes: receiving signals about an input tensor and an index tensor; reading target data from a source storage space corresponding to the input tensor based on the index tensor through a general tensor operation interface, and writing the target data into a destination storage space corresponding to an output tensor, wherein the general tensor operation interface supports unified tensor operations involving at least two data arrangement methods.

[0006] In at least one embodiment of the data processing method provided in this disclosure, the step of reading target data from the source storage space corresponding to the input tensor based on the index tensor through a general tensor operation interface and writing the target data into the destination storage space corresponding to the output tensor includes: performing the following operations using the general tensor operation interface: determining the output coordinates of the target data corresponding to the current thread based on the tensor attribute information of the output tensor; determining the index value offset corresponding to the current thread to obtain the input coordinates of the target data based on the tensor attribute information of the index tensor and the output coordinates; determining the address offset of the target data in the input tensor based on the tensor attribute information of the input tensor and the input coordinates and reading the target data from the source storage space; and writing the target data into the destination storage space based on the tensor attribute information of the output tensor, wherein the tensor attribute information includes a data arrangement method.

[0007] In at least one embodiment of the data processing method provided in this disclosure, the current thread is included in a thread bundle. The step of determining the output coordinates of the target data corresponding to the current thread based on the tensor attribute information of the output tensor includes: determining a task distribution method that supports continuous writing based on the data arrangement and shape of the output tensor, so that data blocks written by multiple threads in the same thread bundle are arranged continuously in the destination storage space; and determining the output coordinates of the target data corresponding to the current thread based on the task distribution method.

[0008] In at least one embodiment of the data processing method provided in this disclosure, the step of determining the index value offset corresponding to the current thread based on the tensor attribute information of the index tensor and the output coordinates to obtain the input coordinates of the target data includes: truncating the output coordinates according to the starting dimension of the index and the dimension of the index tensor to determine the coordinates of the index value corresponding to the current thread; determining the index value offset corresponding to the current thread according to the data arrangement of the index tensor and the coordinates of the index value; and determining the input coordinates of the target data according to the index value offset and the output coordinates.

[0009] In at least one embodiment of the data processing method provided in this disclosure, determining the input coordinates of the target data based on the index value offset and the output coordinates includes: starting from dimension 0, sequentially accessing each dimension of the input tensor; in response to the existence of an index tensor in the current dimension, determining the value of the input coordinates in the current dimension based on the index value offset and the number of index tensors before the current dimension; in response to the absence of an index tensor in the current dimension, determining the value of the input coordinates in the current dimension based on the value of the output coordinates in the corresponding dimension.

[0010] In at least one embodiment of the data processing method provided in this disclosure, determining the value of the input coordinate in the current dimension based on the index value offset and the number of index tensors preceding the current dimension includes: determining the index tensor corresponding to the current dimension based on the number of index tensors preceding the current dimension; reading the index value from the index tensor corresponding to the current dimension based on the index value offset, and using it as the value of the input coordinate in the current dimension.

[0011] In a data processing method provided in at least one embodiment of this disclosure, determining the value of the input coordinate in the current dimension based on the value of the output coordinate in the corresponding dimension includes: in response to the absence of an index tensor before the current dimension, using the value of the output coordinate in the current dimension as the value of the input coordinate in the current dimension; in response to the presence of an index tensor before the current dimension, using the value of the output coordinate in the Xth dimension as the value of the input coordinate in the current dimension, where X = current dimension + dimension of the index tensor - number of index tensors before the current dimension.

[0012] In the data processing method provided in at least one embodiment of this disclosure, for the same target data, the index value offset corresponding to each index tensor is the same.

[0013] In at least one embodiment of the data processing method provided in this disclosure, the step of determining the address offset of the target data in the input tensor and reading the target data from the source storage space based on the tensor attribute information of the input tensor and the input coordinates includes: determining the address offset of the target data in the input tensor based on the data arrangement of the input tensor and the input coordinates; and reading the target data from the source storage space based on the address offset.

[0014] In a data processing method provided in at least one embodiment of this disclosure, the current thread is included in a thread bundle, and the step of writing the target data into the destination storage space according to the tensor attribute information of the output tensor includes: merging the target data corresponding to the current thread with the target data corresponding to other threads in the thread bundle and writing them into the destination storage space according to the data arrangement of the output tensor.

[0015] In a data processing method provided in at least one embodiment of this disclosure, the current thread is included in a thread bundle. After determining the index value offset corresponding to the current thread, the method further includes: in response to the absence of an index tensor in the continuous dimension of the output tensor data arrangement storage, determining to broadcast the index value corresponding to the index value offset to multiple threads in the thread bundle.

[0016] In a data processing method provided in at least one embodiment of this disclosure, the step of determining to broadcast the index value corresponding to the index value offset to multiple threads in the thread bundle in response to the absence of an index tensor in the continuous dimension of the output tensor data arrangement storage includes: determining to broadcast the index value corresponding to the index value offset to multiple threads in the thread bundle in response to the absence of an index tensor dimension obtained by indexing covering the continuous dimension of the output tensor data arrangement storage.

[0017] In the data processing method provided in at least one embodiment of this disclosure, the general tensor operation interface includes at least one of a general tensor read / write interface or a general coordinate processing interface. The general tensor read / write interface includes a thread-granularity read / write interface and a thread-beam granularity read / write interface. The thread-granularity read / write interface is configured to support broadcast operations, and the thread-beam granularity read / write interface is configured to support merged read / write. The general coordinate processing interface includes at least one of a coordinate transformation interface or a coordinate calculation interface. The coordinate transformation interface is configured to implement coordinate transformation under different data arrangements, and the coordinate calculation interface is configured to calculate the address offset corresponding to the coordinate.

[0018] At least one embodiment of this disclosure provides a data processing apparatus, comprising: a receiving module configured to receive signals regarding an input tensor and an index tensor; and an execution module configured to read target data from a source storage space corresponding to the input tensor based on the index tensor via a general tensor operation interface, and write the target data into a destination storage space corresponding to an output tensor, wherein the general tensor operation interface supports unified tensor operations involving at least two data arrangement methods.

[0019] At least one embodiment of this disclosure provides an electronic device, the electronic device comprising: at least one processor; at least one memory including one or more computer program modules; wherein the one or more computer program modules are stored in the at least one memory and configured to be executed by the at least one processor, the one or more computer program modules being used to implement the data processing method provided in the at least one embodiment above.

[0020] At least one embodiment of this disclosure provides a non-transitory computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed by at least one processor, perform the data processing method provided in the at least one embodiment described above.

[0021] This disclosure provides at least one embodiment of a data processing method, data processing apparatus, electronic device, and non-transitory computer-readable storage medium, proposing a general and efficient indexing operation implementation scheme. This scheme can adapt to different data arrangement methods and is compatible with complex index access requirements under different hardware architectures, thereby significantly improving the generalization ability and execution efficiency of index operators. It can support high-efficiency, low-latency indexing operations on heterogeneous computing devices. This solves the problems of current lack of generalization schemes supporting indexing operations under multiple different data arrangement methods, and the high complexity of enumeration methods. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0023] Figure 1 A flowchart illustrating a data processing method provided in at least one embodiment of this disclosure;

[0024] Figure 2 A flowchart illustrating a data processing method provided in at least one embodiment of this disclosure;

[0025] Figure 3 This is a schematic diagram of the structure of a general-purpose graphics processor;

[0026] Figure 4 A schematic block diagram of a data processing apparatus provided in at least one embodiment of this disclosure;

[0027] Figure 5 A schematic block diagram of an electronic device provided for at least one embodiment of this disclosure;

[0028] Figure 6 A schematic block diagram of another electronic device provided for at least one embodiment of this disclosure;

[0029] Figure 7 This is a schematic block diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0031] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as “comprising” or “including” mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as “connected” or “linked” are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as “upper,” “lower,” “left,” and “right” are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0032] The present disclosure will now be described through several specific embodiments. To keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and known components may be omitted. When any component of an embodiment of the present disclosure appears in more than one drawing, the component is represented by the same or similar reference numerals in each drawing.

[0033] Tensors typically refer to multidimensional arrays capable of representing various data structures, including scalars (zero-dimensional tensors), vectors (one-dimensional tensors), matrices (two-dimensional tensors), and arrays of higher dimensions. A tensor consists of multiple elements. The specific positions of these elements within the tensor can be represented by coordinates. For example, a two-dimensional tensor with a shape of 1×8190 indicates that it has 1 row and 8190 columns. Any element can be accessed based on its coordinates (i, j), where i is the row coordinate and j is the column coordinate. For example, rows can represent the batch dimension, and columns can represent the height dimension. For multidimensional tensors, the number and order of coordinates depend on the tensor's dimensions and shape.

[0034] Tensor indexing is a common operation in fields such as deep learning, image processing, and large-scale data computation. For example, in feature extraction, label mapping, or attention mechanisms, it is often necessary to extract data from specific positions in a source tensor (input tensor) based on a set of indices to form a new target tensor (output tensor).

[0035] Indexing operators are high-level indexing operations used to extract data from input tensors of arbitrary dimensions. These operations support data extraction along any dimension of the input tensor, based on a given integer index tensor, and can be flexibly applied to data preprocessing and specific modeling tasks. In deep learning, indexing operators enable models to efficiently access specific elements or sub-tensors within a tensor without traversing the entire data structure, significantly improving computational efficiency and providing greater flexibility for complex data operations. In large-scale models such as Mixture-of-Experts (MoE) models, which typically contain hundreds or even thousands of expert networks, efficient indexing operators can achieve precise routing and data distribution between input samples and corresponding experts, supporting horizontal scaling of the model.

[0036] Taking a three-dimensional input tensor of shape (x, y, z) as an example, it means that the 0th dimension has x elements, the 1st dimension has y elements, and the 2nd dimension has z elements. Next, indexing operations are performed on the 0th and 2nd dimensions of the input tensor. The given (e.g., user-specified) shape of the index tensor is (2), that is, each index tensor is a one-dimensional tensor, including two elements.

[0037] The following is a code example that extracts elements with indices [0, 2] and [1, 3] from the 0th and 2nd dimensions of an input tensor, respectively:

[0038] indices_0 = [0, 2]

[0039] indices_2 = [1, 3]

[0040] output_tensor = input_tensor[indices_0, : ,indices_2]

[0041] Where indices_0 is the index tensor corresponding to the 0th dimension, indices_2 is the index tensor corresponding to the 2nd dimension, input_tensor is the input tensor, and output_tensor is the output tensor generated after extraction based on the specified index.

[0042] The output tensor, `output_tensor`, is a two-dimensional tensor with a shape of (2, y). Specifically, the first row of the output tensor, `output_tensor[0, :]`, corresponds to `input_tensor[0, : , 1]`, and the second row, `output_tensor[1, :]`, corresponds to `input_tensor[2, : , 3]`. Here, `input_tensor[0, : , 1]` represents the selection of the 0th element of the 0th dimension, all elements of the 1st dimension, and the 1st element of the 2nd dimension of `input_tensor`, forming a one-dimensional tensor of length y; `input_tensor[2, : , 3]` represents the selection of the 2nd element of the 0th dimension, all elements of the 1st dimension, and the 3rd element of the 2nd dimension of `input_tensor`, also forming a one-dimensional tensor of length y. Therefore, the shape of the output tensor, `output_tensor`, is determined to be (2, y).

[0043] The above describes the implementation at the logical level. In the underlying hardware implementation, which involves reading and writing data, the indexing operator can convert the index into a linear address offset based on the dimension and step size information of the input tensor, thereby realizing tensor indexing operations on the data.

[0044] The following is a low-level implementation of an indexing operator, including steps S1 to S5.

[0045] S1: Traverse all output index items and perform the following steps S2 to S5 for each output index item.

[0046] For example, an output index term refers to each addressable element in the output tensor. For example, in the previous example, if the output tensor has the shape (2, y), it contains 2×y output index terms. In the case of a single thread performing the indexing operation, the thread will process these 2×y output index terms one by one in sequence. Each output index term corresponds to a set of address offsets, including the output tensor write position offsets[0] (indicating where the current thread should write the indexed result to in the output tensor), the address offsets[1] of the non-indexed dimensions in the input tensor (e.g., the access offset of the first dimension in the input tensor in the previous example), and the address offsets[2] of the index tensor used to read the index value (indicating where the current thread should read the index value from in the index tensor). The above "position" refers to the address location in the storage space.

[0047] S2: Calculate the input address and output address based on the address offset corresponding to the output index item.

[0048] For example, the output address is obtained by adding offsets[0] to the starting address of the output tensor's data, and the input address is obtained by adding offsets[1] to the starting address of the input tensor's data. The input address points to the starting access position of the non-indexed dimension in the input tensor, and the output address points to the position in the output tensor where the current thread should write the result.

[0049] S3: Calculate the cumulative address offset of the input tensor along the index dimension.

[0050] For example, traverse all index dimensions (such as the 0th and 2nd dimensions in the example above), read the index value of the output index item in the index dimension from the index tensor according to offsets[2], calculate the address offset in the storage space according to the step size corresponding to the index dimension, and sum the address offsets of each index dimension to obtain the accumulated address offset.

[0051] For example, for a three-dimensional tensor of shape (x, y, z), the step size of the 0th dimension is y×z, the step size of the 1st dimension is z, and the step size of the 2nd dimension is 1. If the current output index item has an index value of 2 in the 0th dimension and an index value of 3 in the 2nd dimension, then the calculated cumulative address offset is 2×y×z + 3×1.

[0052] S4: Calculate the total offset based on the input address and the accumulated address offset.

[0053] For example, assuming the input address is denoted as in_data and the accumulated address offset is denoted as idx_offset, the total offset can be represented as in_data[idx_offset].

[0054] S5: Read data from the input tensor based on the total offset and write it back to the corresponding address of the output tensor.

[0055] For example, reading the data located at in_data[idx_offset] in the input tensor and writing it to the position pointed to by the output address completes the indexing operation of an element.

[0056] The arrangement of elements in a tensor within a storage space (such as memory or video memory) is called the tensor layout, or the tensor data arrangement in this embodiment. The tensor layout indicates the storage order and dimensional arrangement of tensors in the storage space, which can affect data access efficiency and the throughput of hardware computing units, thereby affecting the efficiency of hardware parallel computing.

[0057] Heterogeneous operators can be tensor operations executed on different types of hardware devices (such as Graphics Processing Units (GPUs), Tensor Processing Units (TPUs), Deep Learning Processing Units (DPUs), Accelerated Processing Units (APUs), Neural Network Processing Units (NPUs), etc.). They can also be tensor operations executed on similar hardware devices from different vendors, which may differ in architecture, memory hierarchy, and computational unit layout. These operations are assigned to the most suitable device based on their computational characteristics. A well-designed layout can effectively improve the data read / write bandwidth of heterogeneous operators, thereby achieving higher performance. Theoretically, the layout can be format-converted at the software layer, allowing the hardware to handle input data with different layouts by supporting only one basic format. However, with the continuous expansion of deep learning model scale and the increasing demands for inference and training efficiency, relying on software conversion is no longer sufficient to meet application requirements. Therefore, to fully utilize the bandwidth potential of the hardware and achieve higher read / write bandwidth, it is necessary to design and be aware of the tensor layout at the hardware level. In other words, by directly supporting different data arrangement methods through hardware, unnecessary memory access overhead and data transfer delays can be reduced.

[0058] Common tensor layout types include, but are not limited to, the following:

[0059] Row-major layout: Consecutive rows of data are stored sequentially in contiguous storage locations (e.g., memory). For example, in a two-dimensional array, all data within a row is stored contiguously, while data between different rows may have significant address jumps.

[0060] Column-major layout: Consecutive columns of data are stored sequentially in contiguous memory locations. For example, in a two-dimensional array, all data within a single column is stored contiguously, while data between different columns may have significant memory address jumps.

[0061] Stride Layout: By specifying the stride for each dimension, the jump distance of tensor elements in the storage space is defined. In other words, the stride for each dimension defines how many bytes to jump to the next element, which can flexibly support custom layouts of any dimension.

[0062] Dimensional order layout: This determines the layout of a tensor in storage space by specifying the order in which its dimensions are arranged. For example, a four-dimensional tensor includes batch (B) dimension, channel (C) dimension, height (H) dimension, and width (W) dimension. Two common dimensional order layouts are as follows:

[0063] NCHW layout: Batch, Channel, Height, Width are arranged in dimensional order, which is suitable for mathematical operations on the CPU and facilitates channel-level operations.

[0064] NHWC layout: arranged in dimensional order of Batch, Height, Width, and Channel, suitable for GPU memory access patterns.

[0065] The tensor layout types described above are merely illustrative examples. The tensor layouts (i.e., data arrangement methods) involved in this disclosure are not limited to the aforementioned examples. Various tensor layouts can be flexibly defined and extended according to different application scenarios and hardware characteristics. This disclosure does not impose any restrictions on this.

[0066] In hardware architectures that support multiple data arrangement methods, the calculation of address offsets cannot simply follow the traditional step-size accumulation method. Instead, it must be remapped and calculated based on the current tensor's data arrangement and specific coordinate information. Therefore, the aforementioned traditional implementation method cannot be directly applied to tensor processing scenarios with specific data arrangement methods.

[0067] Furthermore, for index operators that support multiple data arrangement methods, the inputs they need to process not only include the index representations under different data arrangement methods, but also the combination relationships between multiple index dimensions. Due to the lack of a unified and universal implementation scheme and the high complexity of the operators, it is difficult to cover all possible combinations of data arrangement methods and index dimensions if an enumeration-based implementation method is used. It can only deal with some typical scenarios, resulting in weak generalization ability of index operators.

[0068] Furthermore, hardware and software architectures with layout design often perform well in computationally intensive tasks but poorly in memory-intensive operators, especially indexing operators which involve non-contiguous access. The powerful flexibility of indexing operators relies on complex processing logic, and layout design increases this complexity by nearly N cubed (where N is the number of layouts). Therefore, a universal indexing scheme is urgently needed in hardware and software architectures based on layout design.

[0069] In summary, there is currently no universal implementation mechanism that can effectively support the implementation of index operators in hardware environments where multiple data arrangement methods coexist. Furthermore, as the complexity of operators increases, the enumeration implementation method faces significant challenges in practical applications.

[0070] At least one embodiment of this disclosure provides a data processing method, the method comprising: receiving signals about an input tensor and an index tensor; reading target data from a source storage space corresponding to the input tensor based on the index tensor through a general tensor operation interface, and writing the target data into a destination storage space corresponding to the output tensor, wherein the general tensor operation interface supports unified tensor operations involving at least two data arrangement methods.

[0071] This disclosure provides a data processing method with at least one embodiment, proposing a general and efficient index operation implementation scheme that can adapt to different data arrangement methods and be compatible with complex index access requirements under different hardware architectures. This significantly improves the generalization ability and execution efficiency of index operators, enabling high-efficiency, low-latency index operations on heterogeneous computing devices. It solves the problems of current lack of generalization schemes supporting index operations under multiple different data arrangement methods and the high complexity of enumeration methods.

[0072] Furthermore, at least one embodiment of the data processing method provided in this disclosure proposes a broadcast strategy and a merge write strategy based on data arrangement, which reduces redundant tensor read and write operations, effectively improves memory access efficiency and the overall performance of the operator, fully leverages the performance advantages of modern computing devices in large-scale parallel computing, and realizes support for index operators in high-performance deep learning scenarios.

[0073] For example, the data processing method provided in at least one embodiment of this disclosure can be applied to a processor, which may include a graphics processing unit (GPU), a tensor processing unit (TPU), a general-purpose graphics processing unit (GPGPU), a deep learning processing unit (DPU), an accelerated processing unit (APU), a neural network processing unit (NPU), etc. Of course, the embodiments of this disclosure are not limited to this, and the processor may also be any other type of processor.

[0074] The data processing method provided in at least one embodiment of this disclosure can be used for the implementation of index operators.

[0075] Figure 1 This is a flowchart illustrating a data processing method provided in at least one embodiment of the present disclosure.

[0076] For example, such as Figure 1 As shown, the data processing method provided in at least one embodiment of this disclosure includes the following steps S101-S102. For example, the data processing method provided in at least one embodiment of this disclosure can be applied to a processor, which may include a graphics processor or a general-purpose graphics processor. Of course, the embodiments of this disclosure are not limited to this, and the processor may also be any other type of processor.

[0077] Step S101: Receive signals about the input tensor and the index tensor.

[0078] For example, in step S101, the input tensor refers to the source tensor participating in the indexing operation, containing the data elements to be processed, and is the target object to be accessed by the indexing operation. For example, the input tensor may include at least one of voice data, image data, text data, video data, and biomedical data. The index tensor is used to specify which elements to read from the input tensor; its content represents the index position information on a certain dimension of the input tensor to be indexed. Specific examples of input tensors and index tensors have been described above and will not be repeated here.

[0079] For example, a signal about an input tensor can be a generalized representation of the input tensor, and a signal about an index tensor can be a generalized representation of the index tensor. Generalized representations include, but are not limited to, attributes such as the tensor's dimension, shape, data type, and data arrangement. For instance, a generalized representation can be implemented using a struct, with the aforementioned attributes as member variables of the struct. It should be noted that the implementation of generalized representations is not limited to structs; other data structures or object models (such as classes) can also be used. This generalized representation helps to shield differences in underlying implementations, enabling unified processing and interface compatibility for various data arrangement methods.

[0080] For example, the input to an indexing operator may include the tensor to be indexed (the input tensor), one or more index tensors (whose dimensions are less than or equal to the dimension of the tensor to be indexed), and the index dimension corresponding to each index tensor. Each index tensor has the same shape and the same data arrangement.

[0081] Step S102: Using a general tensor operation interface, read the target data from the source storage space corresponding to the input tensor based on the index tensor, and write the target data into the destination storage space corresponding to the output tensor. The general tensor operation interface supports unified tensor operations involving at least two data arrangement methods.

[0082] In some examples, at least two of the input tensor, index tensor, and output tensor have different data arrangements in storage space. For example, the input tensor uses a row-major layout, while the index tensor and output tensor use a column-major layout. Furthermore, the input tensor, index tensor, and output tensor can all have different data arrangements in storage space.

[0083] In other examples, the input tensor, index tensor, and output tensor can be arranged in exactly the same way in storage space. For example, the input tensor, index tensor, and output tensor can all use a row-first layout.

[0084] The above are merely examples. This disclosure does not limit the specific data arrangement of the input tensor, index tensor, and output tensor. Each tensor can adopt any applicable layout form based on the actual application scenario.

[0085] For example, in step S102, the output tensor is the target tensor of the indexing operation, used to store the data indexed from the input tensor. Its shape typically depends on the shape of the index tensor, the dimension of the index tensor, and the shape of the input tensor. Storage space can be located in memory, video memory, cache, or other types of storage media. Source storage space refers to the storage area storing the input tensor, such as a segment of system memory; destination storage space refers to the storage area storing the output tensor. The source and destination storage spaces can reside in the same storage medium or different storage media, depending on the actual application scenario and system architecture. Target data refers to the specific data extracted from the input tensor through the indexing operation and ultimately written into the output tensor.

[0086] The data arrangement of tensors is referred to as tensor layout as described above. It defines the rules for arranging multidimensional tensors in storage space, including but not limited to row-first layout, column-first layout, step-size layout, and dimension-order layout. A detailed description of tensor layout has been provided earlier and will not be repeated here. Different data arrangement methods differ in data access patterns, memory address mapping rules, and hardware access characteristics. To adapt to diverse storage structures and hardware architectures, the data arrangement methods for input tensors, index tensors, and output tensors may vary in practical applications.

[0087] For example, a general tensor operation interface can provide tensor operation capabilities to upper-level callers through functions, providing unified operation semantics under at least two different data arrangement methods. The general tensor operation interface is configured to adapt to multiple data arrangement methods, possessing compatibility with various data arrangement methods and shielding the underlying data arrangement differences, thereby achieving support for different data arrangement methods.

[0088] For example, unified tensor operations mean that in the upper-level logic, regardless of the data arrangement of the input (e.g., input tensor, index tensor, and output tensor) of the general tensor operation interface, the external operation form remains consistent. When users call the general tensor operation interface, they do not need to care about the underlying data arrangement of the tensor to complete the required data processing tasks. In other words, users do not need to use different operation interfaces or adjust operation logic for different data arrangements, nor do they need to write different code for different data arrangements.

[0089] It should be noted that the "through the general operation interface" mentioned in step S102 refers to the unified access and operation of tensor data from the perspective of the upper-level caller (that is, reading the target data from the source storage space corresponding to the input tensor based on the index tensor and writing the target data into the destination storage space corresponding to the output tensor). It reflects the consistency of the operation semantics provided by the interface, rather than the specific underlying implementation details.

[0090] Figure 2 This is a flowchart illustrating a data processing method provided in at least one embodiment of the present disclosure.

[0091] An example of step S102 includes performing the following operations using a general tensor operation interface. For ease of description, each operation is referred to as steps S201 to S204, as follows: Figure 2 As shown.

[0092] It should be noted that in this example, "perform the following operations using the general tensor operation interface" refers to the specific implementation of this interface at the underlying level; that is, the following operations are encapsulated within the interface. In other words, the general tensor operation interface encapsulates the actual read and write logic for different data arrangement methods, including specific implementations such as address offset calculation and coordinate calculation.

[0093] Step S201: Determine the output coordinates of the target data corresponding to the current thread based on the tensor attribute information of the output tensor.

[0094] Step S202: Based on the tensor attribute information and output coordinates of the index tensor, determine the index value offset corresponding to the current thread to obtain the input coordinates of the target data.

[0095] Step S203: Based on the tensor attribute information and input coordinates of the input tensor, determine the address offset of the target data in the input tensor and read the target data from the source storage space.

[0096] Step S204: Write the target data into the destination storage space according to the tensor attribute information of the output tensor.

[0097] As data volume increases, the efficiency of single-threaded indexing operations gradually becomes a performance bottleneck. To improve the speed of indexing operations, multi-threading techniques can be used for parallel acceleration. That is, the indexing task can be divided into multiple consecutive intervals, with each thread responsible for processing the indexing task of one interval. Since each indexing operation is independent of each other and there is no cross-thread data dependency, it can be split into multiple threads for concurrent execution. The indexing operations provided in this embodiment can be executed serially by a single thread or processed in parallel through a multi-threading mechanism.

[0098] For example, in a single-threaded execution scenario, the "current thread" refers to the thread currently executing the indexing task; in a multi-threaded parallel execution scenario, the "current thread" refers to the thread that is currently being described among multiple concurrent threads. The target data corresponding to the current thread refers to the data elements that this thread is responsible for processing, which are read from the source storage space where the input tensor resides and written to the destination storage space where the output tensor resides.

[0099] For example, tensor attribute information may include the data arrangement of the tensor. Since this embodiment can support cases where at least two of the input tensor, index tensor, and output tensor have different data arrangement methods in the storage space, the underlying operations described in steps S201 to S204 above all require determining the corresponding processing method based on the data arrangement method to achieve support and compatibility with different data arrangement methods. In some examples, tensor attribute information may also include the tensor's dimension, shape, data type, etc., which is not limited in this embodiment.

[0100] For example, in step S201, the output coordinates are logical coordinates independent of physical storage. That is, the logical coordinates of the target data being processed by the current thread in the output tensor can be determined based on the tensor attribute information of the output tensor. For example, in a two-dimensional tensor of shape (2, 3), the logical coordinates [1, 2] are used to represent the elements in the first row and second column, regardless of their actual storage method in the storage space. This use of logical coordinates helps to shield the differences caused by different data arrangement methods and achieve unified support for multiple data arrangement methods. For example, in a single-threaded execution scenario, each data item can be processed sequentially according to the actual arrangement order of the output tensor in the storage space; in a multi-threaded execution scenario, a task partitioning method that supports memory access merging can be adopted according to the data arrangement method of the output tensor to improve memory access efficiency and overall execution performance.

[0101] For example, in step S202, the index value offset corresponding to the current thread can be determined based on the tensor attribute information of the index tensor and the output coordinates obtained in step S201. The index value offset is an address offset used to read the corresponding index value from the index tensor. Given that the starting address of the index tensor is known, the storage address of the index value can be determined through the index value offset. The index tensor can be stored in any storage medium; it can be stored in the same storage medium as the input tensor or the output tensor, or it can be stored in a different storage medium. This embodiment of the disclosure does not impose any restrictions on this. The input coordinates of the target data can be determined based on the index value offset and the output coordinates. The input coordinates are also logical coordinates independent of physical storage, that is, the logical coordinates of the target data being processed by the current thread within the input tensor.

[0102] For example, in step S203, the address offset of the target data within the input tensor can be determined based on the tensor attribute information of the input tensor and the input coordinates obtained in step S202. Given that the starting address of the input tensor is known, the storage address of the target data in the source storage space can be determined using the address offset. Based on this storage address, the target data can be read from the source storage space.

[0103] For example, in step S204, the target data read from the source storage space can be written into the destination storage space in the correct format and order according to the tensor attribute information of the output tensor.

[0104] The data processing method provided in at least one embodiment of this disclosure maps and calculates based on the data arrangement and specific coordinate information of each tensor in the underlying implementation of the general tensor operation interface, thereby realizing an efficient generalization scheme for index operators that support different data arrangement methods.

[0105] In the data processing method provided in at least one embodiment of this disclosure, an indexing operation can be implemented using a warp-level parallel mechanism, thereby fully leveraging the parallel computing capabilities of a parallel computing processor (e.g., a general-purpose graphics processor).

[0106] Figure 3 This is a schematic diagram of the structure of a general-purpose graphics processor.

[0107] like Figure 3 As shown, a general-purpose graphics processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster (SPC), such as including... Figure 3The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In a general-purpose graphics processor, one streaming processor cluster handles one computational task, or multiple streaming processor clusters handle one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0108] like Figure 3 As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 3 The system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit includes multiple Cores (also called computational cores), each of which includes an Arithmetic Logic Unit (ALU), a floating-point unit, etc. These Cores are used to execute specific computational tasks. Furthermore, the Computation Unit also includes registers (e.g., ...). Figure 3 The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0109] like Figure 3 As shown, each streaming processor cluster also provides a buffer for caching data from the N computing units within the streaming processor cluster.

[0110] In high-performance parallel computing using parallel computing processors such as Graphics Processing Units (GPUs) or general-purpose graphics processors, a large number of threads are used to achieve high concurrency. For example, computational tasks are typically executed through multiple threads. These threads are divided into multiple thread blocks before execution in the parallel computing processor, and then dispatched via a thread block distribution module. Figure 3(Not shown in the diagram) Multiple thread blocks are distributed to various computational units. All threads in a thread block are assigned to the same computational unit for execution. Multiple thread blocks can execute in the same computational unit or in different computational units. Simultaneously, thread blocks are broken down into minimal execution thread bundles (or simply warps), which are the basic units of hardware scheduling. Each warp contains a fixed number (or less than this fixed number) of threads, for example, 32 threads. These threads execute the same instructions in a single-instruction multithreaded (SIMT) manner, only processing different data. Within a warp, each thread can share some register resources with other threads, while also possessing its own independent register resources for loading or computing different data.

[0111] In each computing unit, the thread beam scheduling / distribution module ( Figure 3 (Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can execute concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory-executed instructions are issued to shared memory within the computing unit or further issued to intermediate-level caches, global caches, or global memory (e.g., [example cache]). Figure 3 High Bandwidth Memory (HBM) is used for read and write operations.

[0112] In a scenario where indexing operations are implemented using a thread-beam parallel mechanism, an example of step S201 above may include the following steps S2011 to S2012.

[0113] Step S2011: Based on the data arrangement and shape of the output tensor, determine the task distribution method that supports continuous writing, so that the data blocks written by multiple threads in the same thread bundle are arranged continuously in the destination storage space.

[0114] For example, in step S2011, multiple threads within the same thread bundle need to process consecutively arranged data elements, thereby maximizing the memory access consistency of the thread bundle, reducing memory access latency, and improving bandwidth utilization. Since the shapes of the input tensor and the index tensor are known, the shape of the output tensor can also be uniquely determined.

[0115] Taking a thread bundle containing 32 threads as an example, with the output tensor being a two-dimensional tensor of shape (2, 64) and stored using a row-major layout, the output tensor can be divided into four consecutive data segments, each containing 32 consecutive elements. The 32 threads in the same thread bundle are each responsible for processing (writing) one element from their respective data segment, and adjacent threads process consecutive elements. This task distribution method matches the hardware characteristics of the thread bundle, fully leveraging the advantages of memory access merging and improving overall execution efficiency.

[0116] Different data arrangement methods can lead to differences in the order and distribution of data elements in the storage space. For example, under some data arrangement methods, elements with adjacent logical coordinates may have a large address gap in the storage space. Therefore, when performing thread-beam parallel processing, it is necessary to dynamically determine the task distribution strategy suitable for the data arrangement method based on the data arrangement method and shape information of the output tensor in order to support the continuity of write operations.

[0117] The above method allows data blocks written by multiple threads within the same thread bundle to be arranged consecutively in the destination storage space, thereby improving memory access efficiency and fully leveraging the hardware's parallel capabilities.

[0118] Step S2012: Determine the output coordinates of the target data corresponding to the current thread based on the task distribution method.

[0119] For example, in step S2012, the task distribution method can indicate the storage area where the current thread ultimately needs to write the target data. Based on the data arrangement of the output tensor, the mapping relationship between the physical storage address offset and logical coordinates of each element in the output tensor can be obtained. Based on this mapping relationship, the logical coordinates of the target data handled by the current thread in the output tensor can be further determined.

[0120] In the data processing method provided in at least one embodiment of this disclosure, an example of step S202 includes the following steps S2021 to S2023.

[0121] Step S2021: Truncate the output coordinates according to the starting dimension of the index and the dimension of the index tensor to determine the coordinates of the index value corresponding to the current thread.

[0122] For example, the coordinates of an index value refer to its logical coordinates within the index tensor. The output coordinates can be truncated based on the starting dimension of the index and the dimensions of the index tensor to determine the logical coordinates of the index value that the current thread needs to read within the index tensor. The number of dimensions of the index value's coordinates is equal to the number of dimensions of the index tensor.

[0123] For example, the starting dimension of the index refers to the dimension on which the indexing operation is performed first. If the indexing operation is performed on the 0th and 1st dimensions of the input tensor, then the starting dimension is 0. If the shape of the index tensor is (x, y), then the dimension of the index tensor is 2. Two consecutive dimensions starting from the starting dimension can be extracted from the output coordinates as the logical coordinates of the index value within the index tensor.

[0124] For example, in the example above, if the output coordinates corresponding to the current thread are [o1, o2, o3, o4], since the starting dimension of the index is 0 and the dimension of the index tensor is 2, the coordinates of the index value corresponding to the current thread can be determined to be [o1, o2].

[0125] Step S2022: Determine the index value offset corresponding to the current thread based on the data arrangement of the index tensor and the coordinates of the index value.

[0126] For example, based on the data arrangement of the index tensor, the mapping relationship between the physical storage address offset and logical coordinates of each element in the index tensor can be obtained. The corresponding address offset, or index value offset, can be determined based on the logical coordinates of the index value. The corresponding index value can then be read from the index tensor using the index value offset.

[0127] In the data processing method provided in at least one embodiment of this disclosure, for the same target data, the index value offset corresponding to each index tensor is the same. Therefore, the current thread can use the same index value offset to read the required index value from different index tensors, effectively improving computational efficiency.

[0128] Step S2023: Determine the input coordinates of the target data based on the index value offset and the output coordinates.

[0129] An example of step S2023 includes: starting from dimension 0, sequentially accessing each dimension of the input tensor and performing the following steps S2023a to S2023b.

[0130] Step S2023a: In response to the existence of an index tensor in the current dimension, determine the value of the input coordinates in the current dimension based on the index value offset and the number of index tensors preceding the current dimension.

[0131] For example, in step S2023a, if the current dimension requires indexing, it is assumed that an index tensor exists for the current dimension. The number of index tensors preceding the current dimension indicates how many dimensions before the current dimension require indexing.

[0132] For example, a binary representation can be used to predefine which dimensions require indexing. Each bit corresponds to a dimension of the input tensor; a value of 1 indicates that the dimension requires indexing (i.e., an indexed tensor exists for that dimension), while a value of 0 indicates that the dimension does not require indexing (i.e., an indexed tensor does not exist for that dimension). For instance, if the binary representation is set to dims=0b011, it means that dimensions 0 and 1 require indexing, while dimension 2 does not. This approach provides a flexible and compact multidimensional index configuration, facilitating the rapid parsing and determination of the current dimension's indexing requirements, and further determining the corresponding indexed tensor. In step S2023a, the existence of an indexed tensor for the current dimension can be determined by analyzing the aforementioned binary representation.

[0133] For example, suppose the indexing operation is performed on the 0th and 2nd dimensions of a 3D input tensor. If the current dimension is the 0th or 2nd dimension, then it is determined that an index tensor exists in the current dimension; if the current dimension is the 1st dimension, then it is determined that no index tensor exists in the current dimension.

[0134] For example, suppose the indexing operation is performed on the 0th and 2nd dimensions of the four-dimensional input tensor. If the current dimension is the 0th dimension, then the number of index tensors before the current dimension is 0. If the current dimension is the 1st or 2nd dimension, then the number of index tensors before the current dimension is 1 (referring to the index tensor corresponding to the 0th dimension). If the current dimension is the 3rd dimension, then the number of index tensors before the current dimension is 2 (referring to the index tensors corresponding to the 0th and 2nd dimensions).

[0135] An example of "determining the value of the input coordinate in the current dimension based on the index value offset and the number of index tensors before the current dimension" in step S2023a may include the following steps S301 to S302.

[0136] Step S301: Determine the index tensor corresponding to the current dimension based on the number of index tensors preceding the current dimension.

[0137] For example, the number of index tensors preceding the current dimension indicates how many dimensions need to be indexed before the current dimension. Based on this, it can be determined from multiple index tensors (each corresponding to a different dimension to be indexed) which index tensor the current dimension should read the index value from. Multiple index tensors can be organized sequentially into a set of index tensors to achieve ordered access to the index tensors.

[0138] For example, assuming the number of index tensors preceding the current dimension is 2, the index tensor corresponding to the current dimension can be the 3rd index tensor among multiple index tensors, that is, the 3rd item in the set of index tensors.

[0139] Step S302: Read the index value from the index tensor corresponding to the current dimension based on the index value offset, and use it as the value of the input coordinate in the current dimension.

[0140] For example, since step S301 has determined which index tensor the current thread should read the index value from, and since the current thread can use the same index value offset to read the required index value from different index tensors, the index value can be read from the determined index tensor based on the index value offset obtained in step S2022. Furthermore, the read index value is used as the value of the input coordinate in the current dimension.

[0141] For example, assuming the current dimension is the first dimension, the index value read through step S302 above is idx_value, and the input coordinates corresponding to the current thread are [i1, i2, i3], then the value of the input coordinates in the current dimension (the first dimension) is i2 = idx_value.

[0142] Step S2023b: In response to the absence of an index tensor in the current dimension, determine the value of the input coordinates in the current dimension based on the value of the output coordinates in the corresponding dimension.

[0143] The above text has already explained how to determine if an index tensor exists in the current dimension, so it will not be repeated here.

[0144] An example of "determining the value of the input coordinate in the current dimension based on the value of the output coordinate in the corresponding dimension" in step S2023b may include the following steps S303 to S304.

[0145] Step S303: In response to the absence of an index tensor in the current dimension, use the output coordinates in the current dimension as the input coordinates in the current dimension.

[0146] For example, in step S303, if there is no index tensor in the current dimension and no index tensor is involved before the current dimension, that is, no indexing is required in the current dimension and the dimensions before the current dimension, then the value of the output coordinate in the current dimension is used as the value of the input coordinate in the current dimension.

[0147] For example, assuming the indexing operation is performed on the first dimension of a 3D input tensor, if the current dimension is the 0th dimension, then it is assumed that there are no index tensors in the current dimension or the dimensions preceding it; if the current dimension is the 2nd dimension, then it is assumed that there are no index tensors in the current dimension, but there are index tensors preceding it (referring to the index tensor corresponding to the 1st dimension). If the output coordinates corresponding to the current thread are [o1, o2, o3, o4], and the input coordinates corresponding to the current thread are [i1, i2, i3], then the value of the input coordinate in the 0th dimension, i1, is equal to the value of the output coordinate in the 0th dimension, o1.

[0148] Step S304: In response to the existence of an index tensor before the current dimension, use the value of the output coordinate in the X-th dimension as the value of the input coordinate in the current dimension, where X = current dimension + dimension of the index tensor - number of index tensors before the current dimension.

[0149] For example, in step S304, if there is no index tensor in the current dimension, but there is an index tensor before the current dimension, that is, no indexing is required in the current dimension but indexing is required in the dimensions before the current dimension, then the value of the output coordinate in the X-th dimension is used as the value of the input coordinate in the current dimension.

[0150] For example, we can assume that the indexing operation is performed on the first dimension of the 3D input tensor. If the current dimension is the second dimension, it is assumed that there is no index tensor in the current dimension, but there is an index tensor before the current dimension (referring to the index tensor corresponding to the first dimension). Assuming the dimension of the index tensor is 2, since the current dimension is 2, the number of index tensors before the current dimension is 1 (referring to the index tensor corresponding to the first dimension), then X = current dimension + dimension of the index tensor - number of index tensors before the current dimension = 2 + 2 - 1 = 3. If the output coordinates corresponding to the current thread are [o1, o2, o3, o4], and the input coordinates corresponding to the current thread are [i1, i2, i3], then the value of the input coordinate in the second dimension, i3, is equal to the value of the output coordinate in the X-dimensional dimension (i.e., the third dimension), o4.

[0151] For example, we can assume that the indexing operation is performed on the 0th and 2nd dimensions of the four-dimensional input tensor.

[0152] If the current dimension is the first dimension, then it is assumed that there is no index tensor in the current dimension, but there is an index tensor before the current dimension (referring to the index tensor corresponding to the 0th dimension). Assuming the dimension of the index tensor is 2, since the current dimension is 1, the number of index tensors before the current dimension is 1 (referring to the index tensor corresponding to the 0th dimension). Therefore, X = current dimension + dimension of the index tensor - number of index tensors before the current dimension = 1 + 2 - 1 = 2. If the output coordinates corresponding to the current thread are [o1, o2, o3, o4], and the input coordinates corresponding to the current thread are [i1, i2, i3, i4], then the value of the input coordinate in the first dimension, i2, is equal to the value of the output coordinate in the Xth dimension (i.e., the second dimension), o3.

[0153] If the current dimension is the 3rd dimension, it is assumed that there is no index tensor in the current dimension, but there are index tensors before the current dimension (referring to the index tensors corresponding to the 0th and 2nd dimensions). Assuming the dimension of the index tensor is 2, since the current dimension is 3, the number of index tensors before the current dimension is 2 (referring to the index tensors corresponding to the 0th and 2nd dimensions). Therefore, X = current dimension + dimension of the index tensor - number of index tensors before the current dimension = 3 + 2 - 2 = 3. If the output coordinates corresponding to the current thread are [o1, o2, o3, o4], and the input coordinates corresponding to the current thread are [i1, i2, i3, i4], then the value of the input coordinate in the 3rd dimension, i4, is equal to the value of the output coordinate in the Xth dimension (i.e., the 3rd dimension), o4.

[0154] Referring to the above formula, the total output dimension can also be calculated, that is, the total output tensor dimension = the total input tensor dimension + the dimension of the index tensor - the number of index tensors before the total input tensor dimension.

[0155] An example of step S203 includes the following steps S2031 to S2032.

[0156] Step S2031: Determine the address offset of the target data in the input tensor based on the data arrangement and input coordinates of the input tensor.

[0157] For example, based on the data arrangement of the input tensor, the mapping relationship between the physical storage address offset and logical coordinates of each element in the input tensor can be obtained. Based on this mapping relationship, the address offset of the target data corresponding to the current thread in the input tensor can be further determined according to the input coordinates.

[0158] Step S2032: Read the target data from the source storage space according to the address offset.

[0159] For example, given that the starting address of the input tensor is known, the storage address of the target data in the source storage space can be determined by the aforementioned address offset, and the target data can be read from the source storage space based on this storage address.

[0160] In a scenario where indexing operations are implemented using a thread bundle-level parallel mechanism, one example of step S204 above may include: merging the target data corresponding to the current thread with the target data corresponding to other threads in the thread bundle and writing them into the destination storage space according to the data arrangement of the output tensor.

[0161] If a task distribution method supporting continuous writes has been adopted in step S2011, then the data that multiple threads within the current thread bundle need to write is contiguous in the destination storage space. Correspondingly, in this example, these target data can be merged into a single thread bundle-level batch write operation, thereby significantly improving memory access efficiency. This approach not only enhances the support capability of the general tensor operation interface for heterogeneous hardware architectures but also effectively improves the execution performance under large-scale parallel tasks.

[0162] In scenarios where indexing operations are implemented using a thread-beam parallel mechanism, after determining the index value offset corresponding to the current thread in step S203, the following step S205 can be executed.

[0163] Step S205: In response to the absence of an index tensor in the continuous dimension of the output tensor data arrangement storage, determine to broadcast the index value corresponding to the index value offset to multiple threads in the thread bundle.

[0164] For example, in step S205, the dimension of continuous data arrangement and storage of the output tensor refers to which dimension of the output tensor's data is stored continuously in the storage space. This dimension can be determined according to the data arrangement method of the output tensor; for different data arrangement methods, the dimension of continuous data arrangement and storage may be different.

[0165] For example, if it is determined that broadcasting is possible, then thread 0 in the thread bundle can read the index values ​​of all index tensors and broadcast them to the other threads in the thread bundle.

[0166] Taking an output tensor of shape (n, h, w) with a row-first layout as an example, the output tensor is stored continuously in the w-dimensional space.

[0167] If no index tensors exist in the consecutive dimensions of the output tensor data arrangement, meaning no index tensor exists in the input dimension corresponding to dimension w, then thread 0 in the thread bundle can read the index values ​​of all index tensors and broadcast them to other threads within the bundle. This is because threads within the same thread bundle process a contiguous block of output memory, and adjacent threads will access data in the consecutive w-dimensional space. Therefore, thread 0 can read the corresponding index values ​​in all n-dimensional and h-dimensional spaces and broadcast them to other threads within the bundle, thereby reducing repeated read requests for index tensors and improving memory access efficiency.

[0168] Conversely, if an index tensor exists on the input dimension corresponding to dimension w, then each thread needs to read the index tensor to determine the coordinates on dimension w. Since each thread has a different index value on dimension w, they cannot share the index value, and therefore no broadcast operation is performed.

[0169] By broadcasting the index value, multiple threads within a thread bundle can share the index value, effectively reducing repeated read requests for the index tensor. This method significantly reduces memory access bandwidth pressure and improves parallel execution efficiency.

[0170] An example of step S205 includes: in response to the output tensor obtained by indexing not covering the dimensions of the output tensor data arrangement storage contiguously, determining to broadcast the index value corresponding to the index value offset to multiple threads in the thread bundle.

[0171] For example, assuming the input tensor has a shape of (3, 5) and the 2D index tensor has a shape of (2, 2), performing an indexing operation on the first dimension will yield a 3D output tensor with a shape of (3, 2, 2). In this example, the initial index dimension is 1, and the index tensor dimension size is 2. Therefore, both the first and second dimensions of the output tensor are dimensions obtained through the index. Assuming the output tensor is stored in row-major order, the second dimension of the output tensor is the dimension of the data arrangement and storage. At this point, it can be determined that the dimensions of the output tensor obtained through the index cover the dimensions of the output tensor's data arrangement and storage, thus determining not to broadcast the index value.

[0172] For example, assuming the input tensor has a shape of (3, 5) and the 2D index tensor has a shape of (2, 2), performing an indexing operation on the 0th dimension will yield a 3D output tensor with a shape of (2, 2, 5). In this example, the initial index dimension is 0, and the index tensor dimension size is 2. Therefore, both the 0th and 1st dimensions of the output tensor are dimensions obtained through the index. Assuming the output tensor is stored in row-major order, the 2nd dimension of the output tensor is the dimension where the data is stored continuously. At this point, it can be determined that the dimension of the output tensor obtained through the index does not cover the dimension where the output tensor's data is stored continuously, thus determining the broadcast index value.

[0173] In the data processing method provided in at least one embodiment of this disclosure, the general tensor operation interface can be encapsulated in the generalized representation described above. The generalized representation described above is a data structure or object model capable of describing tensor properties and operational behaviors. For example, a class can be defined, which includes data members describing tensor properties and functions for manipulating these data, thereby providing better scalability and compatibility.

[0174] Therefore, the data processing method provided in this disclosure can support all permutations and combinations (including combinations of different data permutations and combinations of multiple index dimensions) through, for example, template functions. This can solve the problem of low support rate of index operators in enumeration schemes, while reducing the investment of a large amount of development manpower and reducing the learning cost of complex operators.

[0175] For example, a general tensor operation interface may include at least one of a general tensor read / write interface or a general coordinate processing interface. The general tensor read / write interface includes a thread-granular read / write interface and a thread-beam granular read / write interface. The thread-granular read / write interface is configured to support broadcast operations, and the thread-beam granular read / write interface is configured to support merged read / write.

[0176] Among them, the thread-beam granular read-write interface supports read-write merging, which helps improve memory access efficiency under large-scale parallel computing, and can be dynamically adapted according to the read-write characteristics of different hardware platforms; the thread-beam granular interface can implement flexible read-write modes, such as supporting broadcasting mechanisms, and usually adopts the traditional memory access mode based on the starting address pointer plus address offset to implement read and write.

[0177] For example, a general coordinate processing interface may include at least one of a coordinate transformation interface or a coordinate calculation interface. The coordinate transformation interface is configured to implement coordinate transformation under different data arrangements, and the coordinate calculation interface is configured to calculate the address offset corresponding to the coordinate.

[0178] The following is a pseudocode example of a data processing method provided in at least one embodiment of this disclosure, which corresponds to step S202 above:

[0179] int idx_count = 0;

[0180] int out_count = 0;

[0181] size _t pos = index.toAbsPos(&coord_out[index_at_dim])

[0182] for (int i = 0; i < dims_in; ++i) {

[0183] if ((dims & (1 << i))) {

[0184] coord_in[i] = index.template loadPos <broadcastidx>(pos, idx_count);

[0185] if (i == index_at_dim) {

[0186] out_count += dims_idx;

[0187] }

[0188] idx_count += 1;

[0189] } else {

[0190] coord_in[i] = coord_out[out_count];

[0191] out_count += 1;

[0192] }

[0193] }

[0194] The variable index in the above pseudo code is an example of a general representation of an index tensor, which encapsulates the properties of the index tensor (such as data arrangement, dimension, shape, data type, etc.) and a general tensor operation interface. toAbsPos() is an example of a coordinate calculation interface, which can calculate the index value offset according to the data arrangement and the coordinate parameters. loadPos() is a data arrangement-independent encapsulation for reading index values from the index tensor, which can implement the broadcast function. The variables idx_count and out_count are used for counting to implement the scheme of step S2023, and the variable idx_count indicates the number of index tensors before the current dimension; the variable pos represents the index value offset; the variable dims_in represents the dimension of the input tensor; the variable dims is a binary representation introduced in the related content of step S2023a, which is used to indicate which dimension needs to be indexed; the variable coord_in represents the input coordinate; the variable coord_out represents the output coordinate; and the variable index_at_dim indicates the starting dimension of the index.

[0195] The following is a specific example of a general tensor operation interface underlying operation provided by the data processing method and the above pseudo code based on at least one embodiment of the present disclosure.

[0196] In this example, it is assumed that the shape of the input tensor Input is (3, 4, 5), the two index tensors index_0 and index_1 correspond to indexing in the 0th dimension and the 1st dimension respectively, and the shape of each of the two index tensors is (2, 2), and the shape of the output tensor Output is (2, 2, 5). Correspondingly, the value of the binary representation dims is 0b011, indicating that there is indexing in the 0th dimension and the 1st dimension. In this example, the data arrangement manners of any two of the input tensor, the index tensor, and the output tensor are different. It should be noted that in some examples, the data arrangement manners of all of the input tensor, the index tensor, and the output tensor can also be the same.

[0197] Step S2011 is performed to determine a distribution manner supporting continuous writing according to the data arrangement manner and the shape of the output tensor, so that the data blocks written by the multiple threads in the same thread bundle are arranged continuously in the destination storage space. In this example, it is assumed that the output tensor adopts row-major storage, and each thread bundle includes 32 threads, and then the task can be allocated to multiple thread bundles, and each thread bundle processes 32 continuous data. For example, in a thread bundle, the output coordinate corresponding to a thread can be [0, 0, 0], and the output coordinate corresponding to the next adjacent thread can be [0, 0, 1]. It should be noted that since the output tensor can have various data arrangement manners, the output coordinate corresponding to the next adjacent thread can also be other coordinates besides [0, 0, 1], for example, [0, 1, 0] or [1, 0, 0], etc.

[0198] Step S2012 is performed to determine the output coordinate of the target data corresponding to the current thread according to the task distribution manner. It can be assumed here that the output coordinate coord_out corresponding to the current thread is determined according to the task distribution manner, which is [o1, o2, o3], that is, the current thread needs to store the target data into the corresponding address of the destination storage space corresponding to the output coordinate [o1, o2, o3].

[0199] Step S2021 is performed to intercept the output coordinate [o1, o2, o3] according to the index start dimension 0 and the dimension 2 of the index tensor, that is, to intercept the two continuous dimensions starting from the index start dimension 0 from the output coordinate to obtain the index value coordinate [o1, o2] corresponding to the current thread.

[0200] Step S2022 is performed to determine the index value offset corresponding to the current thread according to the arrangement manner of the index tensor and the coordinate of the index value.

[0201] For example, steps S2021-S2022 can be implemented by the toAbsPos() interface in the above pseudo code example.

[0202] Next, each dimension of the input tensor is accessed in turn from the 0th dimension. In this example, it is assumed that the shape of the input tensor Input is (3, 4, 5), the two index tensors index_0 and index_1 correspond to indexing in the 0th dimension and the 1st dimension respectively, and the shape of each of the two index tensors is (2, 2), and the shape of the output tensor Output is (2, 2, 5). Correspondingly, the value of the binary representation dims is 0b011, indicating that there is indexing in the 0th dimension and the 1st dimension. In this example, the data arrangement manners of any two of the input tensor, the index tensor, and the output tensor are different. It should be noted that in some examples, the data arrangement manners of all of the input tensor, the index tensor, and the output tensor can also be the same.

[0203] Since there is an index tensor in the 0th dimension, step S2023a is performed, and the index tensor corresponding to the 0th dimension, that is, index 0, is determined according to the number of index tensors before the 0th dimension, and the index value idx value 0 is read in index 0 according to the index value offset, as the value of the input coordinate in the 0th dimension. Assuming that the input coordinate coord in corresponding to the current thread is [i1, i2, i3], the value of the input coordinate in the 0th dimension coord in[0]=i1=idx value 0.

[0204] Since there is an index tensor in the 1st dimension, step S2023a is performed, and the index tensor corresponding to the 1st dimension, that is, index 1, is determined according to the number of index tensors before the 1st dimension, and the index value idx value 1 is read in index 1 according to the index value offset, as the value of the input coordinate in the 1st dimension, that is, coord in[1]=i2=idx value 1.

[0205] For example, the operation of step S2023a can be implemented by the loadPos() encapsulation of the above pseudo code example. In some examples, before step S2023a is performed, step S205 can be performed first, and if it is determined that there is no index tensor in the dimension on which the output tensor data is arranged to be stored continuously, it is determined that the index value corresponding to the index value offset can be broadcast to multiple threads in the thread bundle. At this time, the index values of all index tensors can be read by the 0th thread in the thread bundle and broadcast to other threads in the thread bundle.

[0206] Since there is no index tensor in the 2nd dimension, step S2023b is performed, and there is an index tensor before the 2nd dimension, step S304 is performed, and the value of the output coordinate in the Xth dimension is taken as the value of the input coordinate in the 2nd dimension, X=current dimension+dimension of index tensor-current dimension-number of index tensors before current dimension=2+2-2=2, thus coord in[2]=i3=coord out[2]=o3.

[0207] Therefore, the input coordinate coord in=[i1, i2, i3]=[idx value 0, idx value 1, o3] can be obtained. That is, in the destination storage space corresponding to the output coordinate coord out=[o1, o2, o3], the target data read from the source storage space corresponding to the input coordinate coord in=[idx value 0, idx value 1, o3] is to be written.

[0208] Next, step S2031 is performed to determine the address offset of the target data in the input tensor according to the data arrangement manner of the input tensor and the input coordinates [idx_value_0, idx_value_1, o3]; and step S2032 is performed to read the target data from the source storage space according to the address offset.

[0209] Next, step S204 is performed to merge the target data required to be written by the plurality of threads in the current thread bundle into a batch write operation of once thread bundle granularity according to the data arrangement manner of the output tensor, and write the target data into the corresponding destination storage space.

[0210] The data processing method provided by the at least one embodiment of the present disclosure provides a general and efficient index operation implementation scheme, which can adapt to different data arrangement manners, and is compatible with complex index access requirements under different hardware architectures, thereby significantly improving the generalization ability and execution efficiency of the index operator, and supporting efficient and low-delay index operations on heterogeneous computing devices. The problem that there is no generalization scheme to support index operations under multiple different data arrangement manners, and the implementation complexity of the enumeration manner is high is solved.

[0211] Further, in combination with the broadcast strategy and the merge write strategy based on the data arrangement manner, the method effectively reduces redundant tensor read and write operations, improves memory access efficiency and overall performance of the operator, fully plays the performance advantage of modern computing devices in large-scale parallel computing, and realizes support for the index operator in a high-performance deep learning scenario.

[0212] It should be further noted that in each of the embodiments of the present disclosure, the execution order of each step of the data processing method is not limited, and although the execution process of each step is described in a specific order above, this does not constitute a limitation on the embodiments of the present disclosure. Each step in the data processing method can be executed in series or in parallel, which can be determined according to actual needs.

[0213] For example, compared with the above description, the data processing method provided by at least one embodiment of the present disclosure can further include more or fewer steps, and the embodiments of the present disclosure do not limit this.

[0214] Figure 4 A schematic block diagram of a data processing apparatus provided by at least one embodiment of the present disclosure.

[0215] The data processing apparatus may, for example, be or be arranged in a processor, which may include a Graphics Processing Unit (GPU), a Tensor Processing Unit (TPU), a General-purpose Graphics Processing Unit (GPGPU), a Deep learning Processing Unit (DPU), an Accelerated Processing Unit (APU), a Neural network Processing Unit (NPU), etc. Of course, embodiments of the present disclosure are not limited thereto, and the processor may also be any other type of processor.

[0216] The data processing apparatus provided by at least one embodiment of the present disclosure may be suitable for implementation of an index operator in a deep learning network.

[0217] For example, as shown in Figure 4 The data processing apparatus provided by at least one embodiment of the present disclosure may include a receiving module 401 and an executing module 402.

[0218] For example, the receiving module 401 is configured to receive signals about an input tensor and an index tensor.

[0219] For example, the executing module 402 is configured to read target data from a source storage space corresponding to the input tensor based on the index tensor through a general tensor operation interface, and write the target data into a destination storage space corresponding to an output tensor, where the general tensor operation interface supports unified tensor operations involving at least two data arrangement modes.

[0220] For example, in at least one embodiment of the present disclosure, the executing module 402 includes an output determining unit, an input determining unit, a reading unit, and a writing unit. The output determining unit is configured to determine an output coordinate of target data corresponding to a current thread according to tensor attribute information of the output tensor; the input determining unit is configured to determine an index value offset amount corresponding to the current thread to obtain an input coordinate of the target data according to the tensor attribute information of the index tensor and the output coordinate; the reading unit is configured to determine an address offset amount of the target data in the input tensor according to the tensor attribute information of the input tensor and the input coordinate and read the target data from the source storage space; and the writing unit is configured to write the target data into the destination storage space according to the tensor attribute information of the output tensor, where the tensor attribute information includes a data arrangement mode. The output determining unit, the input determining unit, the reading unit, and the writing unit are a bottom layer implementation of the general tensor operation interface.

[0221] For example, in at least one embodiment of the present disclosure, the output determination unit is further configured to: determine a task distribution manner supporting continuous writing according to the data arrangement manner and the shape of the output tensor, so that the data blocks written by the plurality of threads in the same thread bundle are arranged continuously in the target storage space; and determine the output coordinates of the target data corresponding to the current thread according to the task distribution manner.

[0222] For example, in at least one embodiment of the present disclosure, the input determination unit is further configured to: intercept the output coordinates according to the index start dimension and the dimension of the index tensor to determine the coordinates of the index value corresponding to the current thread; determine the index value offset corresponding to the current thread according to the data arrangement manner of the index tensor and the coordinates of the index value; and determine the input coordinates of the target data according to the index value offset and the output coordinates.

[0223] For example, in at least one embodiment of the present disclosure, the input determination unit is further configured to: sequentially access each dimension of the input tensor from the 0th dimension: in response to the current dimension existing an index tensor, determine the value of the input coordinates in the current dimension according to the index value offset and the number of index tensors before the current dimension; and in response to the current dimension not existing an index tensor, determine the value of the input coordinates in the current dimension according to the value of the output coordinates in the corresponding dimension.

[0224] For example, in at least one embodiment of the present disclosure, the input determination unit is further configured to: determine the index tensor corresponding to the current dimension according to the number of index tensors before the current dimension; and read the index value in the index tensor corresponding to the current dimension according to the index value offset as the value of the input coordinates in the current dimension.

[0225] For example, in at least one embodiment of the present disclosure, the input determination unit is further configured to: in response to no index tensor existing before the current dimension, take the value of the output coordinates in the current dimension as the value of the input coordinates in the current dimension; and in response to an index tensor existing before the current dimension, take the value of the output coordinates in the Xth dimension as the value of the input coordinates in the current dimension, where X = current dimension + dimension of the index tensor - number of index tensors before the current dimension.

[0226] For example, in at least one embodiment of the present disclosure, the index value offset corresponding to each index tensor is the same for the same target data.

[0227] For example, in at least one embodiment of the present disclosure, the reading unit is further configured to: determine the address offset of the target data in the input tensor according to the data arrangement manner of the input tensor and the input coordinates; and read the target data from the source storage space according to the address offset.

[0228] For example, in at least one embodiment of this disclosure, the current thread is included in a thread bundle, and the writing unit is further configured to: merge the target data corresponding to the current thread with the target data corresponding to other threads in the thread bundle and write them into the destination storage space according to the data arrangement of the output tensor.

[0229] For example, in at least one embodiment of this disclosure, the current thread is included in a thread bundle, and the execution module further includes a broadcast unit configured to, after determining the index value offset corresponding to the current thread, determine to broadcast the index value corresponding to the index value offset to multiple threads in the thread bundle in response to the absence of an index tensor on a continuous dimension of the output tensor data arrangement storage.

[0230] For example, in at least one embodiment of this disclosure, the broadcast unit is further configured to determine, in response to the output tensor obtained by indexing not covering the dimensions of the output tensor data arrangement stored in a continuous manner, to broadcast the index value corresponding to the index value offset to multiple threads in the thread bundle.

[0231] For example, in at least one embodiment of this disclosure, the general tensor operation interface includes at least one of a general tensor read / write interface or a general coordinate processing interface. The general tensor read / write interface includes a thread-granular read / write interface and a thread bundle granular read / write interface. The thread-granular read / write interface is configured to support broadcast operations, and the thread bundle granular read / write interface is configured to support merged read / write. The general coordinate processing interface includes at least one of a coordinate transformation interface or a coordinate calculation interface. The coordinate transformation interface is configured to implement coordinate transformation under different data arrangements, and the coordinate calculation interface is configured to calculate the address offset corresponding to the coordinate.

[0232] It should be noted that the above-mentioned modules and units can be implemented by software, hardware, firmware or any combination thereof. For example, the receiving module and the execution module can be implemented as receiving circuit and execution circuit respectively. The embodiments of this disclosure do not limit their specific implementation methods.

[0233] It should be understood that the data processing apparatus 400 provided in at least one embodiment of this disclosure can be used to implement the aforementioned data processing method and can also achieve similar technical effects as the aforementioned data processing method, which will not be elaborated here.

[0234] It should be noted that in the embodiments of this disclosure, the data processing device 400 may include more or fewer modules or units, and the connection relationship between the various modules or units is not limited and can be determined according to actual needs. The specific configuration of each module or unit is not limited; it can be constructed from analog devices according to circuit principles, or from digital chips, or in other applicable ways.

[0235] Figure 5 This is a schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure.

[0236] For example, such as Figure 5 As shown, the electronic device 500 includes at least one processor 501 and at least one memory 502. The at least one memory 502 includes one or more computer program modules. These computer program modules are stored in the memory 502 and configured to be executed by the at least one processor 501. The one or more computer program modules include instructions for performing the data processing method described above. When executed by the at least one processor 501, they can perform one or more steps of the data processing method provided in at least one embodiment of this disclosure. The memory 502 and the processor 501 can be interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0237] For example, processor 501 can be a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a tensor processor (TPU), an artificial intelligence (AI) accelerator, or other processing units with data processing and / or program execution capabilities, such as a field-programmable gate array (FPGA). For example, the CPU can be based on x86, ARM, or RISC-V architectures. Processor 501 can be a general-purpose processor or a special-purpose processor, capable of controlling other components in electronic device 500 to perform desired functions.

[0238] For example, memory 502 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc.

[0239] Figure 6 This is a schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure.

[0240] The electronic devices in at least one embodiment of this disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0241] The electronic device includes at least one processor and a memory. The processor may be referred to as processing device 601 below, and the memory may include at least one of read-only memory (ROM), random access memory (RAM), and storage device 608 below. The memory is used to store programs for performing the methods described in the various method embodiments above; the processor is configured to execute the programs stored in the memory. The processor may include a central processing unit (CPU) or other processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0242] like Figure 6 As shown, electronic device 600 may include processing device 601 (e.g., central processing unit, graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) or a program loaded from storage device 608 into random access memory (RAM). RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interfaces are also connected to bus 604.

[0243] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, displays, speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0244] In particular, according to at least one embodiment of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, at least one embodiment of this disclosure includes a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by a processing device 601, it performs the functions defined in the methods of at least one embodiment of this disclosure.

[0245] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In at least one embodiment of this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In at least one embodiment of this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, radio frequency (RF), etc., or any suitable combination thereof.

[0246] The aforementioned computer-readable medium may be included in the aforementioned electronic device 600; or it may exist independently and not assembled into the electronic device 600.

[0247] Figure 7 This is a schematic block diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure.

[0248] For example, such as Figure 7 As shown, a non-transitory computer-readable storage medium 700 stores computer-readable instructions 701, which, when executed by at least one processor, perform one or more steps of the data processing method described above.

[0249] For example, the storage medium may include a memory card for a smartphone, a storage component for a tablet computer, a hard drive for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media. For example, the readable storage medium may also be... Figure 5 The memory 502 in the memory is described in the foregoing content and will not be repeated here.

[0250] Although the present disclosure has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to the embodiments of the present disclosure, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present disclosure are within the scope of protection claimed by the present disclosure.

[0251] The following points should be noted regarding this disclosure:

[0252] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0253] (2) For clarity, the thickness of layers or regions in the drawings used to describe embodiments of the present disclosure is enlarged or reduced, i.e., these drawings are not drawn to actual scale.

[0254] (3) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0255] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.< / broadcastidx>

Claims

1. A data processing method, characterized by, The data processing method comprises: receiving signals about an input tensor and an index tensor; reading target data from a source storage space corresponding to the input tensor based on the index tensor through a general tensor operation interface, and writing the target data into a destination storage space corresponding to an output tensor, wherein the general tensor operation interface supports unified tensor operations involving at least two data arrangement modes, wherein the reading of the target data from the source storage space corresponding to the input tensor based on the index tensor through the general tensor operation interface and the writing of the target data into the destination storage space corresponding to the output tensor comprise: using the general tensor operation interface to perform the following operations: determining an output coordinate of target data corresponding to a current thread according to tensor attribute information of the output tensor; determining an index value offset corresponding to the current thread to obtain an input coordinate of the target data according to tensor attribute information of the index tensor and the output coordinate; determining an address offset of the target data in the input tensor and reading the target data from the source storage space according to tensor attribute information of the input tensor and the input coordinate; writing the target data into the destination storage space according to the tensor attribute information of the output tensor, wherein the tensor attribute information comprises a data arrangement mode.

2. The data processing method according to claim 1, characterized in that, The current thread is included in a thread bundle, and the determining of the output coordinate of the target data corresponding to the current thread according to the tensor attribute information of the output tensor comprises: determining a task distribution mode supporting continuous writing according to a data arrangement mode and a shape of the output tensor, so that data blocks written by multiple threads in the same thread bundle are arranged continuously in the destination storage space; determining the output coordinate of the target data corresponding to the current thread according to the task distribution mode.

3. The data processing method of claim 1, wherein, The determining of the index value offset corresponding to the current thread to obtain the input coordinate of the target data according to the tensor attribute information of the index tensor and the output coordinate comprises: determining a coordinate of an index value corresponding to the current thread by intercepting the output coordinate according to an index starting dimension and a dimension of the index tensor; determining the index value offset corresponding to the current thread according to a data arrangement mode of the index tensor and the coordinate of the index value; determining the input coordinate of the target data according to the index value offset and the output coordinate.

4. The data processing method according to claim 3, characterized in that, The determining of the input coordinate of the target data according to the index value offset and the output coordinate comprises: starting from the 0th dimension, sequentially accessing each dimension of the input tensor: in response to the current dimension having an index tensor, determining a value of the input coordinate in the current dimension according to the index value offset and a number of index tensors before the current dimension; in response to the current dimension not having an index tensor, determining the value of the input coordinate in the current dimension according to a value of the corresponding dimension of the output coordinate.

5. The data processing method according to claim 4, characterized in that, The determining of the value of the input coordinate in the current dimension according to the index value offset and the number of index tensors before the current dimension comprises: determining an index tensor corresponding to the current dimension according to a number of index tensors before the current dimension; reading an index value in the index tensor corresponding to the current dimension according to the index value offset, and taking the index value as a value of the input coordinate in the current dimension.

6. The data processing method according to claim 4, characterized in that, The determining the value of the input coordinate in the current dimension according to the value of the output coordinate in the corresponding dimension comprises: in response to no index tensor existing before the current dimension, taking the value of the output coordinate in the current dimension as the value of the input coordinate in the current dimension; in response to an index tensor existing before the current dimension, taking the value of the output coordinate in the Xth dimension as the value of the input coordinate in the current dimension, where X = current dimension + dimension of the index tensor - number of index tensors before the current dimension.

7. The data processing method of claim 1, wherein, For the same target data, the index value offset corresponding to each index tensor is the same.

8. The data processing method of claim 1, wherein, The determining the address offset of the target data in the input tensor according to the tensor attribute information of the input tensor and the input coordinate and reading the target data from the source storage space comprises: determining the address offset of the target data in the input tensor according to the data arrangement mode of the input tensor and the input coordinate; reading the target data from the source storage space according to the address offset.

9. The data processing method of claim 1, wherein, The writing the target data into the destination storage space according to the tensor attribute information of the output tensor comprises: merging the target data corresponding to the current thread and the target data corresponding to other threads in the thread bundle and writing the merged target data into the destination storage space according to the data arrangement mode of the output tensor.

10. The data processing method of claim 1, wherein, The current thread is included in a thread bundle, and after the index value offset corresponding to the current thread is determined, the method further comprises: in response to no index tensor existing on a dimension in which the output tensor data is arranged and stored continuously, determining that the index value corresponding to the index value offset is broadcast to multiple threads in the thread bundle.

11. The data processing method according to claim 10, characterized in that, The determining that the index value corresponding to the index value offset is broadcast to multiple threads in the thread bundle in response to no index tensor existing on a dimension in which the output tensor data is arranged and stored continuously comprises: in response to a dimension of the output tensor obtained through indexing not covering the dimension in which the output tensor data is arranged and stored continuously, determining that the index value corresponding to the index value offset is broadcast to multiple threads in the thread bundle.

12. The data processing method of claim 1, wherein, The general tensor operation interface comprises at least one of a general tensor read-write interface or a general coordinate processing interface, the general tensor read-write interface comprises a thread granularity read-write interface and a thread bundle granularity read-write interface, the thread granularity read-write interface is configured to support a broadcast operation, and the thread bundle granularity read-write interface is configured to support merged read-write, the general coordinate processing interface comprises at least one of a coordinate conversion interface or a coordinate calculation interface, the coordinate conversion interface is configured to implement coordinate conversion under different data arrangements, and the coordinate calculation interface is configured to calculate an address offset corresponding to a coordinate.

13. A data processing apparatus, characterized by: The data processing apparatus comprises: a receiving module configured to receive signals about an input tensor and an index tensor; An execution module is configured to read target data from a source storage space corresponding to the input tensor based on the index tensor and write the target data into a destination storage space corresponding to the output tensor through a general tensor operation interface, The general tensor operation interface supports unified tensor operations involving at least two data arrangement modes, The execution module is further configured to: perform the following operations using the general tensor operation interface: determine an output coordinate of target data corresponding to a current thread according to tensor attribute information of the output tensor; determine an index value offset of the current thread to obtain an input coordinate of the target data according to tensor attribute information of the index tensor and the output coordinate; determine an address offset of the target data in the input tensor and read the target data from the source storage space according to tensor attribute information of the input tensor and the input coordinate; write the target data into the destination storage space according to tensor attribute information of the output tensor, The tensor attribute information includes a data arrangement mode.

14. An electronic device, comprising: The electronic device includes: at least one processor; at least one memory including one or more computer program modules; The one or more computer program modules are stored in the at least one memory and configured to be executed by the at least one processor, and the one or more computer program modules are used to implement the data processing method of any one of claims 1-12.

15. A non-transitory computer-readable storage medium, comprising: The non-transitory computer readable storage medium stores computer instructions, wherein the computer instructions are executed by at least one processor to perform the data processing method of any one of claims 1-12.

Images