Data processors, data processing methods, electronic devices, storage media

Abstract

This disclosure provides a data processor, a data processing method, an electronic device, and a non-temporary computer-readable storage medium. The data processor includes a tensor operation unit and N computing units. The tensor operation unit is configured to perform tensor calculations on input data to obtain tensor calculation results. The N computing units are configured to perform at least one of vector operations on the tensor calculation results and the generation of input data, and a first data transfer channel is provided between the tensor operation unit and at least some of the N computing units. The first data transfer channel is configured to directly supply tensor calculation results to the computing units and to directly supply input data from the computing units to the tensor operation unit. The data processor can reduce access to global memory, reduce resource waste, reduce data transfer delays, significantly improve the overall efficiency of the operators, effectively utilize the powerful computing capabilities of the tensor operation unit itself, and improve the computational efficiency of the data processor.

Description

Technical Field

[0001] This application claims the benefit of priority of Chinese Patent Application No. 202410289769.3 filed on March 14, 2024, and incorporates by reference in its entirety the content disclosed in the above Chinese Patent Application.

[0002] Embodiments of the present disclosure relate to a data processor, a data processing method, an electronic device, and a non - transient computer - readable storage medium.

Background Art

[0003] A tensor is a multi - linear map defined by the Cartesian product of several vector spaces and several dual spaces. For example, a scalar can be regarded as a 0 - dimensional tensor, a vector can be regarded as a 1 - dimensional tensor, and a matrix can be regarded as a 2 - dimensional tensor. Tensor operations are widely used in processors such as parallel processors.

[0004] With the development of artificial intelligence and machine learning, new requirements have been raised for a number of parallel processor devices represented by parallel processors (such as multi - core processors, digital signal processors, etc.). With the development of artificial intelligence and machine learning, new requirements have been raised for a number of parallel processor devices represented by parallel processors (such as multi - core processors, graphics processors, digital signal processors, etc.). In neural network processing frequently used in fields such as artificial intelligence, for example, in a convolutional neural network, it is often necessary to perform tensor operations, such as matrix multiplication or convolution operations, and furthermore, it is also often necessary to perform vector calculations, such as cumulative addition, reduction, ordinary addition, subtraction, multiplication, division, etc.

Summary of the Invention

[0005] At least one embodiment of the present disclosure provides a data processor comprising a tensor operation unit and N computing units, where N is a positive integer greater than 1, wherein the tensor operation unit is configured to perform a tensor calculation on input data to obtain a tensor calculation result, the tensor calculation including matrix multiplication or convolution operations, the N computing units are configured to perform at least one of performing a vector operation on the tensor calculation result and generating the input data, a first data transfer channel is provided between the tensor operation unit and at least some of the N computing units, the first data transfer channel is configured to directly supply the tensor calculation result to the corresponding computing unit and to directly supply the input data from the computing unit generating the input data to the tensor operation unit.

[0006] For example, in a data processor provided by at least one embodiment of the present disclosure, each computing unit includes a register configured to store data related to the vector operation and data related to the generation of the input data, the tensor operation unit includes a cache block configured to cache the input data, and the first data transfer channel connects the registers in at least some of the computing units to the cache block.

[0007] For example, in a data processor provided by at least one embodiment of the present disclosure, each computing unit includes a shared cache and a plurality of computing cores, the shared cache is configured to share data among the plurality of computing cores included in the computing unit, a logical operation unit is provided in the shared cache, the logical operation unit is configured to perform logical operations between computing units, and the logical operations between computing units include performing arithmetic logical operations on data from different computing units.

[0008] For example, in a data processor provided by at least one embodiment of the present disclosure, the arithmetic logic operation includes cumulative addition, and the logic operation unit includes an accumulator.

[0009] For example, in a data processor provided by at least one embodiment of the present disclosure, each computing unit includes a shared cache and a plurality of computing cores, the shared cache is configured to share data among the plurality of computing cores included in the computing unit, a second data transfer channel is provided between the shared caches of some or all of the N computing units, and the second data transfer channel is configured to transfer data between the shared caches of some or all of the computing units.

[0010] For example, in a data processor provided by at least one embodiment of the present disclosure, the data processor includes a plurality of streaming processor clusters, each streaming processor cluster includes a plurality of compute units and one tensor operation unit, and the N compute units include compute units in different streaming processor clusters, or the N compute units are in the same streaming processor cluster.

[0011] For example, in a data processor provided by at least one embodiment of the present disclosure, the data processor is further configured to perform operator fusion, the operator fusion including deep fusion of arbitrary operators, and in the operator fusion process, the tensor operation unit and the vector operation unit can both function as a task producer or a task consumer, the vector operation unit includes the N computation units, the task producer is configured to generate computation data, and the task consumer is configured to receive and process the computation data.

[0012] For example, in a data processor provided by at least one embodiment of the present disclosure, the shared cache of some or all of the computing units is abstracted as a single intermediate-level cache layer in the abstraction hierarchy of the data processor.

[0013] At least one embodiment of the present disclosure provides a data processing method for a data processor. The data processor includes a tensor operation unit and a plurality of computing units, the tensor operation unit being configured to perform a tensor calculation on input data to obtain a tensor calculation result, the tensor calculation including matrix multiplication or convolution operations, the plurality of computing units being configured to perform at least one of performing a vector operation on the tensor calculation result and generating the input data, the data processing method including supplying the tensor calculation result directly to the corresponding computing unit using a first data transfer channel, and supplying the input data directly to the tensor operation unit from a computing unit that generates the input data.

[0014] For example, in a data processing method provided by at least one embodiment of the present disclosure, each computing unit includes a register, the register is configured to store data related to the vector operation and data related to the generation of the input data, The tensor operation unit includes a cache block, and the cache block is configured to cache the input data. The first data transfer channel connects registers in at least some of the plurality of computing units to the cache blocks.

[0015] For example, in a data processing method provided by at least one embodiment of the present disclosure, each computing unit includes a shared cache and a plurality of computing cores, the shared cache is configured to share data among the plurality of computing cores included in the computing unit, and the data processing method further includes performing logical operations between computing units in the shared cache, the logical operations between computing units performing arithmetic logical operations on data from different computing units.

[0016] For example, in a data processing method provided by at least one embodiment of the present disclosure, each computing unit includes a shared cache and a plurality of computing cores, the shared cache is configured to share data among the plurality of computing cores included in the computing unit, and the data processing method further includes using a second data transfer channel to transfer data between the shared caches of at least some of the plurality of computing units.

[0017] For example, a data processing method provided by at least one embodiment of the present disclosure further includes abstracting the shared cache of at least some of the computing units as a whole as one intermediate-level cache layer in the abstraction hierarchy of the data processor.

[0018] For example, a data processing method provided by at least one embodiment of the present disclosure further includes performing operator fusion, wherein the operator fusion includes performing a deep fusion of arbitrary operators, and in the process of operator fusion, the tensor operation unit and the vector operation unit can both function as a task producer or a task consumer, wherein the vector operation unit includes the plurality of computation units, the task producer is configured to generate computation data, and the task consumer is configured to receive and process the computation data.

[0019] At least one embodiment of the present disclosure provides an electronic device including a data processor described in any embodiment of the present disclosure.

[0020] At least one embodiment of the present disclosure provides a non - transient computer - readable storage medium storing instructions, which, when executed by a processor, implement the data - processing method described in any embodiment of the present disclosure.

Brief Description of the Drawings

[0021] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only related to some embodiments of the present disclosure and do not limit the present disclosure.

[0022] [Figure 1] A schematic structural diagram of a general - purpose graphics processing unit (GPGPU) is shown. [Figure 2] A schematic structural diagram of a tensor operation unit is shown. [Figure 3] It is a schematic block diagram of a data processor provided by at least one embodiment of the present disclosure. [Figure 4] It is a schematic block diagram of a data processor provided by at least one embodiment of the present disclosure. [Figure 5A] It is a schematic data - flow diagram of logical operations between computing units. [Figure 5B] It is a schematic data - flow diagram of logical operations between computing units provided by at least one embodiment of the present disclosure. [Figure 6] It is a schematic structural diagram of a data processor provided by at least one embodiment of the present disclosure. [Figure 7] It is a diagram showing a software - abstract data - flow provided by at least one embodiment of the present disclosure. [Figure 8] It is a diagram showing a schematic flowchart of a tensor calculation method provided by at least one embodiment of the present disclosure. [Figure 9] It is a diagram showing a schematic flowchart of a tensor calculation method provided by at least one embodiment of the present disclosure. [Figure 10]Schematic diagram of a non - transient computer - readable storage medium provided by at least one embodiment of the present disclosure. [Figure 11] Schematic structural diagram of an electronic device provided by at least one embodiment of the present disclosure.

Embodiments for Carrying Out the Invention

[0023] In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, hereinafter, in combination with the accompanying drawings of the embodiments of the present disclosure, the technical solutions of the embodiments of the present disclosure will be clearly and completely described. Obviously, the described embodiments are some but not all of the embodiments of the present disclosure. Based on the described embodiments of the present disclosure, all other embodiments that can be obtained by those skilled in the art without creative labor belong to the protection scope of the present disclosure.

[0024] Unless otherwise defined, technical terms or scientific terms used in the present disclosure shall have the ordinary meanings understood by those of ordinary skill in the field to which the present disclosure pertains. The "first", "second" and similar terms used in the present disclosure do not represent any order, quantity or importance, but are merely used to distinguish different components. Similar terms such as "include" or "comprise" mean that the elements or objects appearing before that term include the elements or objects listed after that term and their equivalents, and do not exclude other elements or objects. Similar terms such as "connect" or "associate" are not limited to physical or mechanical connections, and can include electrical connections whether direct or indirect. "Upper", "lower", "left", "right", etc. are merely used to represent relative positional relationships, and after the absolute position of the described object changes, the relative positional relationships can also change accordingly.

[0025] In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits the detailed descriptions of some known functions and known components.

[0026] GPUs (Graphics Process Units) and GPGPUs (General-Purpose Graphics Process Units) are widely used in training deep learning models. For common tensor operations in deep learning models, GPU manufacturers implement special optimizations in their software and hardware designs to accelerate computations. For example, some GPU manufacturers provide dedicated tensor units to optimize tensor calculations. These tensor units can include tensor cores, and the use of tensor cores significantly increases data throughput and improves computational efficiency.

[0027] Currently, most accelerators for artificial intelligence generally employ a combination of tensor and vector units to optimize and accelerate computations. Tensor units have relatively high computational power, but can only perform operations such as matrix multiplication or convolution. On the other hand, many operators used in artificial intelligence computations require not only matrix multiplication or convolution, but also additional calculations such as matrix addition, cumulative addition, and reduction operations, and these additional calculations are usually performed by vector units.

[0028] Figure 1 shows a schematic diagram of a general-purpose graphics processing unit (GPGPU).

[0029] As shown in Figure 1, a general-purpose graphics processor is actually an array of Streaming Processor Clusters (SPCs), including, for example, Streaming Processor Cluster 1, ..., Streaming Processor Cluster M shown in Figure 1, where M is a positive integer greater than 1. In a graphics processor, one Streaming Processor Cluster handles one computation task, or multiple Streaming Processor Clusters handle one computation task. Data is shared between multiple Streaming Processor Clusters through a global cache or global memory.

[0030] As shown in Figure 1, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple compute units. For example, compute units 1, 2, ..., N in Figure 1, where N is a positive integer. Each compute unit (abbreviated as CU, e.g., a streaming processor) is configured to perform arithmetic logic operations other than matrix calculations such as matrix multiplication and convolution, such as cumulative addition, reduction, normal addition, subtraction, multiplication, division, etc. A single compute unit contains multiple cores (also called compute nuclei or compute cores). Each compute nucleus includes an arithmetic logic unit (ALU), a floating-point arithmetic unit, etc., and is configured to perform specific computation tasks. Furthermore, a compute unit also includes registers (e.g., a register file in Figure 1) and a shared cache, configured to hierarchically store source data and destination data related to computation tasks, and the shared cache in a single compute unit is configured to share data among the cores of the compute unit.

[0031] In parallel computing, computation tasks are generally executed through multiple threads. These threads are divided into multiple thread blocks before being executed on a general-purpose graphics processor (or parallel computing processor), and then these thread blocks are distributed to each computing unit via a thread block distribution module (not shown in Figure 1). All threads within a single thread block must be assigned to run on the same computing unit. Simultaneously, a thread block is divided into minimum execution thread warps (or simply thread warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks may run on the same computing unit or on different computing units.

[0032] In each computing unit, a thread warp scheduling / distribution module (not shown in Figure 1) schedules and allocates thread warps, enabling multiple computing cores in the computing unit to execute the thread warps. Based on the number of computing cores in the computing unit, multiple thread warps within a single thread block are executed simultaneously or in a time-sharing manner. Multiple threads within each thread warp execute the same instruction. Memory execution instructions are issued to the shared cache in the computing unit, or further to an intermediate-level cache, global cache, or global memory to perform read / write operations, etc.

[0033] As shown in Figure 1, the streaming processor cluster 1 further includes a tensor operation unit. The tensor operation unit is configured to perform tensor calculations, which may include, for example, matrix multiplication, convolution operations, and so on.

[0034] Figure 2 shows a schematic diagram of the tensor operation unit.

[0035] As shown in Figure 2, a tensor operation unit includes cache blocks and computation kernels. For example, in some tensor operation units, two cache blocks are provided, configured to cache two source operands as source data for tensor computations. Input data is transferred from memory or a cache (e.g., an intermediate-level cache, global cache, or global memory in Figure 1) along at least one data transfer channel to the cache blocks of the tensor operation unit, and then transferred from the cache blocks to the computation kernel, where a tensor computation such as matrix multiplication or convolution is completed to obtain the tensor computation result. The tensor computation result is then transferred as output data to memory, a cache, or a vector operation unit.

[0036] In at least one embodiment of the present disclosure, the computation units are collectively referred to as vector operation units, which are configured to perform arithmetic logic operations such as cumulative addition, normal addition, subtraction, multiplication, and division, and reduction, while the tensor operation units are configured to perform tensor-related operations such as matrix multiplication and convolution.

[0037] Furthermore, as shown in Figure 1, an intermediate-level cache can be installed in the streaming processor cluster 1, and the intermediate-level cache is configured to share data among the computing units within a single streaming processor cluster. Of course, in some general-purpose graphics processors, an intermediate-level cache may not be installed, and the computing units may share data via a global cache or global memory.

[0038] The structure of the other streaming processor clusters is the same as that of streaming processor cluster 1 and will not be described in detail here. The general-purpose graphics processor further includes a global cache / memory configured to share data across each streaming processor cluster.

[0039] As shown in Figures 1 and 2, the high-speed cache blocks in the tensor operation unit are configured solely for the tensor operation unit, and all ordinary multiplication, addition, reduction, etc., performed after the tensor operation are carried out by the vector operation unit. In a situation where only the vector operation unit performs calculations, the cache blocks in the tensor operation unit cannot demonstrate their value.

[0040] Furthermore, as shown in Figure 1, the output data of the tensor arithmetic unit can only be transferred unidirectionally to registers in the compute unit, and the compute unit cannot directly write data to the tensor arithmetic unit. When the compute unit writes data to the tensor arithmetic unit, it must first transfer the input data to the compute unit's shared cache, intermediate-level cache (or global cache / memory), and then transfer it from the intermediate-level cache (or global cache / memory) to the tensor arithmetic unit. This is the first time the input data can be written to the tensor arithmetic unit, which increases the path for the tensor arithmetic unit to load the input data into the cache block, leading to resource waste and increased data latency.

[0041] At least one embodiment of the present disclosure provides a data processor, a data processing method, an electronic device, and a non-temporary computer-readable storage medium. The data processor comprises a tensor operation unit and N computing units, where N is a positive integer greater than 1. The tensor operation unit is configured to perform tensor calculations on input data and to obtain tensor calculation results, where the tensor calculations include matrix multiplication or convolution operations. The N computing units are configured to perform at least one of vector operations on the tensor calculation results and the generation of input data, wherein a first data transfer channel is provided between the tensor operation unit and at least some of the N computing units, the first data transfer channel directly supplies tensor calculation results to the corresponding computing units and directly supplies input data from the computing units that generate the input data to the tensor operation unit.

[0042] In the data processor provided in at least one embodiment of this disclosure, an interconnection channel is established between a tensor operation unit and a computing unit, enabling bidirectional connectivity between the computing unit and the tensor operation unit. The tensor operation unit can directly write tensor calculation results to the computing unit, and the computing unit can directly supply input data to the tensor operation unit to perform calculations. This eliminates the need to perform data flow through storage components such as global cache, intermediate-level cache, shared cache, and global memory. This reduces access to global memory, reduces idle time of hardware units, improves hardware utilization, reduces data transfer delay, significantly improves the overall efficiency of the operators, effectively utilizes the powerful computing capabilities of the tensor operation unit itself, and improves the computational efficiency of the data processor.

[0043] The embodiments of this disclosure will be described in detail below with reference to the attached drawings, but this disclosure is not limited to these specific embodiments.

[0044] Figure 3 is a schematic block diagram of a data processor provided in at least one embodiment of the present disclosure.

[0045] As shown in Figure 3, the data processor 100 includes a tensor operation unit 101 and N computation units 102, the N computation units including computation unit 102_1, computation unit 102_2, ..., computation unit 102_N, etc.

[0046] For example, the tensor operation unit 101 is configured to perform tensor calculations on input data and obtain tensor calculation results. For example, tensor calculations include matrix multiplication or convolution operations.

[0047] For example, the N computation units 102 are configured to perform at least one of the following: vector operations on the tensor computation results and generation of input data.

[0048] For example, vector operations include ordinary addition, subtraction, multiplication, and division operations other than matrix multiplication and convolution operations, and may further include cumulative addition and reduction operations, and this disclosure does not limit the specific operations of vector operations.

[0049] For example, the calculation unit 102 can perform reduction operations, cumulative addition, activation function calculations, pooling, etc., on the tensor calculation results obtained through matrix multiplication or convolution operations. For example, the calculation unit 102 can further perform some vector operations on the data to generate input data for tensor calculation, for example, by performing cumulative addition and pooling on the tensor calculation results and providing this as input data to the tensor calculation unit again to continue executing the tensor calculation. For example, the calculation unit 102 can perform either the vector operations on the tensor calculation results and the generation of input data as described above, or the calculation unit 102 can perform the vector operations on the tensor calculation results and generate input data as described above.

[0050] For example, a first data transfer channel is provided between the tensor operation unit 101 and at least some of the N computing units 102.

[0051] For example, as shown in Figure 3, a first data transfer channel 103 is provided between the tensor operation unit and each computing unit. For example, in other embodiments, the tensor operation unit may provide the first data transfer channel 103 between some of the computing units, and the tensor operation unit may not provide the first data transfer channel 103 between other computing units, and this disclosure is not specifically limited thereto.

[0052] For example, the first data transfer channel 103 is configured to directly supply the tensor calculation result to the corresponding computing unit 102 and to directly supply the input data from the computing unit 102 that generates the input data to the tensor operation unit 101.

[0053] In at least one embodiment of this disclosure, by adding a bidirectional interconnection channel between the tensor operation unit and the computing unit, i.e., a first data transfer channel 103, to the data processor, the computing unit and the tensor operation unit can establish a direct communication connection through the first data transfer channel 103. The computing unit can directly access the tensor operation unit, and input data can be directly transferred to the tensor operation unit for tensor calculations. As shown in Figure 1, there is no need to perform data flow through the computing unit's shared cache, intermediate level cache, etc., reducing access to global memory, global cache, etc., reducing resource waste, lowering data transfer delay, significantly improving the overall efficiency of the operators, effectively utilizing the powerful computing capabilities of the tensor operation unit itself, and improving the computing efficiency of the data processor.

[0054] For example, in some embodiments, the data processor may be a general-purpose graphics processor or a graphics processor, and the structure of the data processor can be found in the relevant description in Figure 1 above.

[0055] For example, in some embodiments, the tensor operation unit 102 may be a tensor computation core. The structure of the tensor operation unit 102 can be found in the related description in Figure 2 above. Of course, it should be noted that the tensor operation unit shown in Figure 2 is a schematic structure, and this disclosure is not limited thereto. The structure of the tensor operation unit can be set as needed, as long as it can realize the corresponding functions. For example, in some embodiments, the tensor operation unit may have an additional output unit for caching, outputting, etc., the tensor computation results obtained by the computation core, and this disclosure is not specifically limited thereto. For example, in other embodiments, the computation core in the tensor operation unit may not only complete tensor computations but also output tensor computation results from the tensor operation unit.

[0056] Figure 4 is a schematic block diagram of a data processor provided by at least one embodiment of the present disclosure.

[0057] For example, each computing unit 102 includes a register. For example, the register is configured to store data related to vector operations and data related to the generation of input data.

[0058] For example, the registers in a computing unit may be the register file shown in Figure 4, which is, for example, the register file in a general-purpose graphics processor.

[0059] The register file is the most important part of on-chip memory, providing data access speeds that match those of the computing cores. Unlike, for example, a CPU (Central Processing Unit) core has only a small number of general-purpose registers, each computing unit in a general-purpose graphics processor has a large number of register resources. The graphics processor statically allocates these registers to each thread, and the large register file allows more threads to remain active simultaneously without the need for time-consuming context switching like that of a CPU.

[0060] The tensor operation unit includes a cache block, which is configured to cache input data. For example, as shown in Figure 4, the tensor operation unit may include a cache block, for example, two cache blocks, which are configured to cache two source operands for performing tensor calculations. For example, the two source operands could be tensor A and tensor B, the dimensions of tensor A and tensor B may be 2 or higher, and the matrix multiplication result or convolution result of tensor A and tensor B is computed through the tensor operation unit.

[0061] For example, the first data transfer channel connects the registers of at least some of the computing units to the cache blocks of the tensor arithmetic unit.

[0062] For example, as shown in Figure 4, the first data transfer channel 103 connects the registers of each computing unit to the cache block of the tensor operation unit.

[0063] This enables direct data exchange between the tensor operation unit and the computation unit via the first data transfer channel 103, allowing the tensor operation unit to directly transfer tensor calculation results to registers for subsequent processing (e.g., vector operations), and to directly transfer input data in registers to the cache block of the tensor operation unit for tensor operations. Since data transfer no longer needs to flow through the global cache / memory, resource utilization is improved, data access efficiency is enhanced, data access overhead consumption is avoided, the number of accesses to the global cache, global memory, or intermediate-level cache is reduced, data transfer delay is reduced, and tensor operation efficiency is improved.

[0064] Furthermore, since the registers are closest to the core of the computing unit, have the fastest data transfer speed, and the highest bandwidth, installing the first data transfer channel 103 to connect the registers of the computing unit with the cache blocks of the tensor operation unit can further improve data flow efficiency and thus improve computing efficiency.

[0065] Furthermore, the establishment of the first data transfer channel 103 allows the computing unit to use the memory resources in the tensor operation unit, and the high-speed cache blocks provided to the tensor operation unit become available to the computing unit, fully demonstrating their value in scenarios where computations are performed solely by the vector operation unit.

[0066] Artificial neural networks, also abbreviated as neural networks, are algorithmic mathematical models that mimic the behavioral characteristics of animal neural networks and perform distributed parallel information processing. These networks achieve information processing objectives by adjusting the interconnections between a large number of internal nodes, depending on the complexity of the system. Regardless of the type, common features include large-scale parallel processing, distributed memory, elastic topology, high redundancy, and nonlinear computation, giving them capabilities in terms of computational speed, associative ability, adaptability, fault tolerance, and self-organization.

[0067] To improve the computational efficiency of neural networks, it is common practice to merge multiple adjacent operators that satisfy certain conditions or rules within the neural network to form a fused operator. A single fused operator, or a single operator that cannot be fused, can be represented as a fusion layer, or simply referred to as a layer. The computational process of a neural network is carried out layer by layer, with fusion layers as the unit. Under normal circumstances, the output of the previous layer (or the previous few layers) is used as the input of the subsequent layer (or the subsequent few layers), thereby creating data dependencies between fused layers.

[0068] Currently, fusion layers typically terminate with vector-type operators and use tensor-type operators as the driving force. This is because, although the tensor calculation results output by the tensor unit can be output to the computation unit, the computation unit cannot input the data after the vector operation to the tensor unit for subsequent tensor calculations unless it has passed through the cache at each level and even external storage resources. Therefore, when fusing operators, it is only possible to terminate with vector-type operators. This constitutes a limitation on the operator fusion strategy, making it impossible to achieve deep fusion when fusing operators.

[0069] For example, a data processor provided by at least one embodiment of the present disclosure is further configured to perform operator fusion.

[0070] For example, operator fusion includes performing deep fusion of arbitrary operators, and in the operator fusion process, both the tensor operation unit and the vector operation unit can function as task producers or task consumers. The vector operation unit includes the N computation units.

[0071] Here, the vector operation unit includes, for example, N calculation units 102 in Figure 4. For example, all of the calculation units 102 can be referred to as a vector operation unit.

[0072] For example, a task producer is configured to generate computation data, and a task consumer is configured to receive and process the computation data. For example, when a tensor operation unit functions as a task producer, it can generate tensor computation results as computation data, and a vector operation unit, acting as a task consumer, receives and processes the tensor computation results. For example, when a vector operation unit functions as a task producer, it can generate input data as computation data, and a tensor operation unit, acting as a task consumer, receives and processes the input data and generates tensor computation results.

[0073] The disclosure provides a first data transfer channel, which increases the interconnection between registers and cache blocks built into the tensor operation unit, thereby increasing the potential for deep fusion of fusion operators. The fusion strategy for fusion operators is no longer limited to having the tensor computation core as the driver and the vector computation core as the co-consumer, and can achieve deep fusion of any operator. Depending on the fusion strategy, the tensor operation unit can also drive operators as a task producer and receive and process computation data as a task consumer. Similarly, the vector operation unit can also drive operators as a task producer and receive and process computation data as a task consumer. This results in greater memory access efficiency than loading data through a cache or external storage area.

[0074] Logical operations between computing units include performing arithmetic logic operations on data from different computing units. For example, arithmetic logic operations are performed on data in computing unit 102_1 and computing unit 102_2. For example, arithmetic logic operations can include various arithmetic operations, logical operations, etc., such as addition, subtraction, multiplication, division, bitwise operations, comparison, cumulative addition, reduction, etc., and this disclosure does not limit the specific types of arithmetic logic operations.

[0075] For example, when performing arithmetic logic operations on data in computing unit 102_1 and computing unit 102_2 in computing unit 102_1, the data in computing unit 102_2 can be transferred to the shared cache in computing unit 102_1 via the second data transfer channel (see below), or the data in computing unit 102_2 can be transferred to the shared cache of computing unit 102_1 via an intermediate-level cache, global cache, or global memory, etc.

[0076] Figure 5A is a schematic data flow diagram of logical operations between computing units.

[0077] Currently, as shown in Figure 5A, when data performs logical operations between computing units, the source data for the logical operations must first be transferred from the computing unit's shared cache to the computing core via registers. This source data includes data from other computing units, and the computing core then performs specific arithmetic logical operations on this source data. Subsequently, as shown in Figure 5A, the calculation results are again transferred to the shared cache via registers, and the calculation results are either stored in the shared cache or transferred to other computing units via the shared cache.

[0078] Figure 5B is a schematic data flow diagram of logical operations between computing units provided in at least one embodiment of the present disclosure.

[0079] For example, as shown in Figure 5B, in at least one embodiment of the present disclosure, a logical operation unit is provided in the shared cache, and the logical operation unit is configured to perform logical operations between computing units.

[0080] For example, as shown in Figure 5B, in this disclosure, a logical operation unit is provided in the shared cache of each computing unit, and the logical operation unit can perform some arithmetic logic operations (ALU). For example, the logical operation unit can provide a cumulative addition function. Of course, the logical operation unit can also support arithmetic logic operations such as addition and subtraction as needed, and this disclosure does not impose any specific limitations on this.

[0081] For example, a logical operation unit can support atomic operations, which are one or a series of operations that cannot be interrupted; that is, operations that are not interrupted by the thread scheduling mechanism, do not involve any context switches during execution, and atomic operations either complete all steps or do not execute any steps at all; it is not possible to execute only a portion of all steps.

[0082] For example, after data between computing units enters the shared cache, it is not necessary to directly execute the corresponding arithmetic logic operation in the logical operation unit, transfer it to the computing core via registers, and then perform the logical calculation. Furthermore, it is not necessary to transfer the data to the shared cache via registers, avoiding repeated reading and writing of data to the shared cache, improving computational efficiency, and reducing data access overhead by circulating data within the shared cache as much as possible.

[0083] A group of threads running on a single computing unit is called a thread group, and data transfer between thread groups is primarily performed via an intermediate-level cache or a global cache. As shown in Figure 1, when computing units 1 and 2 of streaming processor cluster 1 perform data transfer, the data is first transferred to the intermediate-level cache via the shared cache, and computing unit 2 then obtains the data via the intermediate-level cache. Alternatively, when computing unit 1 performs data transfer with computing unit 1 in streaming processor cluster M, the data is first transferred to the intermediate-level cache via computing unit 1's shared cache, and then transferred from the intermediate-level cache to the global cache, and computing unit 1 of streaming processor cluster M then obtains the data via the global cache.

[0084] In at least one embodiment of the present disclosure, a second data transfer channel is provided between the shared caches of some or all of the N computing units, and the second data transfer channel is configured to transfer data between the shared caches of the some or all of the computing units, that is, the shared caches of these computing units share data via the second data transfer channel.

[0085] Figure 6 is a schematic diagram of a data processor provided by at least one embodiment of the present disclosure.

[0086] As shown in Figure 6, each computing unit includes a shared cache and multiple computing cores, and the shared cache is configured to share data among the multiple computing cores.

[0087] As shown in Figure 6, a second data transfer channel 104 is provided that is interconnected bidirectionally between the shared caches of the computing units.

[0088] For example, as shown in Figure 6, the registers in the computing unit 102_1 can transmit the calculation results to the shared cache in the computing unit 102_1. The shared cache in computing unit 102_1 transmits the calculation results to the shared cache in computing unit 102_2 via the second data transfer channel 104, and the shared cache in computing unit 102_2 then transmits the calculation results back to the registers in computing unit 102_1 to perform subsequent calculations.

[0089] For example, in some embodiments, a second data transfer channel can be established between the shared caches of all computing units, and data sharing among all computing units can be achieved using this second data transfer channel. For example, in some other embodiments, a second data transfer channel can be established between the shared caches of some computing units, and data sharing among these computing units can be achieved using this second data transfer channel.

[0090] This allows data sharing between computing units to be achieved directly via a shared cache, and data can flow between the shared caches of each computing unit via a second data transfer channel 104, eliminating the need for data transfer to flow through a global cache / memory. This improves resource utilization, enhances the efficiency of data access, avoids the consumption of overhead associated with data access, reduces the number of accesses to the global cache, global memory, or intermediate-level cache, and reduces data transfer latency.

[0091] The shared cache is closer to the computing core, and memory resources closer to the computing core have higher data access efficiency and higher data transfer bandwidth. By providing a second data transfer channel 104, a high-speed interconnection channel can be provided to the shared cache of the computing unit, further improving data flow efficiency, increasing data transfer speed, reducing access overhead, and improving computing efficiency.

[0092] Furthermore, for any computing unit, its operable shared cache is expanded by N times, allowing memory resources to perform at their full potential and enabling full utilization of the shared cache closer to the computing core. In addition, in some embodiments, some arithmetic and logical operations of the computing unit (e.g., cumulative addition) can be performed by the logical operation unit of the shared cache, so these operations can be performed in advance from the global cache, etc., to the shared cache (especially when data accumulator capacity is insufficient), thereby improving system bandwidth efficiency.

[0093] For example, as shown in Figure 1, a data processor may include multiple streaming processor clusters, each streaming processor cluster containing multiple compute units and one tensor operation unit.

[0094] For example, a second data transfer channel may be configured to transfer data between shared caches of compute units in multiple streaming processor clusters. For example, the second data transfer channel may connect all shared caches in all compute units of the entire data processor, and the shared caches in all compute units may perform data transfers via the second data transfer channel. For example, the second data transfer channel may connect the shared caches of some compute units of the data processor, and these compute units may belong to one streaming processor cluster or different streaming processor clusters. For example, the shared caches in these compute units may perform data transfers via the second data transfer channel, while the other compute units still perform data transfers via the global cache, global memory, or intermediate-level cache.

[0095] For example, the aforementioned N computing units may belong to a single streaming processor cluster, and the provided second data transfer channel is configured to share data between the shared caches of the N computing units within the streaming processor cluster.

[0096] For example, the aforementioned N computing units may include computing units in different streaming processor clusters, for instance, all computing units in multiple streaming processor clusters. For example, the aforementioned N computing units may be computing units in different streaming processor clusters, and the provided second data transfer channel may be configured to perform data sharing between shared caches of computing units within different streaming processor clusters.

[0097] In other words, in this disclosure, the second data transfer channel can be configured to perform data sharing between shared caches of compute units belonging to the same streaming processor cluster. Alternatively, if resources allow, it can be configured to perform data sharing between shared caches of all compute units in the entire data processor. N compute units can refer to at least some compute units within a single streaming processor cluster, or to at least some compute units in the entire data processor (all streaming processor clusters).

[0098] Currently, shared caches within computing units are used only within that unit, and computing units cannot access and use shared caches in other computing units. Therefore, when abstracting the software layer, it is not possible to abstract the shared cache as a single level. In other words, it cannot be encapsulated as a single module on its own, so it is not possible to use the shared cache in a unified manner across different operators. This prevents the software abstraction layer from properly introducing hierarchical partitioning, resulting in a large number of experimental methods with different approaches for the operators being designed.

[0099] When establishing data interconnection channels between shared caches, the shared caches of multiple computing units can be abstracted as a single intermediate-level cache layer within the data processor's abstraction hierarchy.

[0100] For example, in the software development process, it is common practice to abstract the system into layers. Abstracting each layer requires focusing only on the information related to that layer, and since this is the only possible approach, it simplifies the overall system design. Currently, since data interconnection channels (second data transfer channels) are established between all shared caches, the shared caches of the computing units can be abstracted as a single high-speed intermediate-level cache layer, allowing for development and encapsulation as a single whole during software development.

[0101] Furthermore, currently, the high-speed cache blocks in the tensor operation unit are used exclusively by the tensor operation unit, and all other computation units cannot access and use the said cache blocks. Therefore, when abstracting the software layer, it is not possible to abstract the cache blocks in the tensor operation unit as a single level.

[0102] For example, in at least one embodiment of the present disclosure, the cache blocks in the tensor operation unit can also be abstracted as an intermediate-level cache layer in the abstraction hierarchy of the data processor.

[0103] Therefore, in at least one embodiment of this disclosure, by providing a first data transfer channel and a second data transfer channel, from a software perspective, it is possible to abstract better software memory access and computation hierarchy structures, further abstract computation task partitioning and adjustment logic, maximize the use of hardware resources such as tensor operation units, and achieve performance generalization.

[0104] Figure 7 shows a software abstract data flow provided by at least one embodiment of the present disclosure.

[0105] Figure 7 shows the data flow at the software layer over time. The two tensor operation units in the figure are not two different tensor operation units, but simply indicate that data can be directly re-inputted back into tensor operation unit 101 after being output from calculation unit 102.

[0106] As shown in Figure 7, multiple thread groups that process the data output by the tensor operation unit can be abstracted, and each thread group is executed within a single computing unit. As shown in Figure 7, the computing unit's execution of the thread groups involves multiple stages, such as a first stage, a second stage, a third stage, and so on. In each stage, data is transferred from the shared cache to a register, then from the register to the computing core where vector operations are performed, and then the results are transferred back to the shared cache.

[0107] Since a second data transfer channel is provided between shared caches, data can be exchanged directly between shared caches, and computing units can directly share data via the shared cache, as shown by the arrows between shared caches in Figure 7. This allows thread groups to communicate with each other via the computing units' shared cache, reducing access to the global cache or global memory as much as possible, circulating data in the fastest possible shared cache, improving resource utilization, enhancing the efficiency of data memory access, avoiding the consumption of data memory access overhead, reducing the number of accesses to the global cache, global memory, or intermediate-level cache, and reducing data transfer latency.

[0108] Furthermore, a first data transfer channel is provided between the tensor operation unit 101 and the computation unit 102, allowing thread groups to directly write data to the tensor operation unit 102. This avoids the need to go through the global cache or global memory, improving resource utilization, enhancing data memory access efficiency, avoiding the consumption of data memory access overhead, reducing the number of accesses to the global cache, global memory, or intermediate-level cache, reducing data transfer delay, and improving tensor operation efficiency.

[0109] In the data processor provided in at least one embodiment of this disclosure, a cache hierarchy is constructed from high to low in bandwidth according to the memory hierarchy, and high-speed interconnection communication is formed between different hierarchies. For example, a first data transfer channel is provided between the compute unit and the tensor operation unit, and a second data transfer channel is provided between the shared caches of the compute units. This solves the utilization problem of high-speed cache units on general-purpose graphics processors of tensor operation architectures, moves data as much as possible between registers, shared memory, and the built-in high-speed cache blocks of the tensor operation unit, reduces access to global memory, improves data locality, maximizes the parallelism of computations, and reduces data transfer delay. In addition, increasing the interconnection between the registers in the compute unit and the high-speed cache area of ​​the tensor operation unit enhances the possibility of deep fusion of fusion operators. From a software perspective, the cache blocks in the tensor operation unit can be abstracted as a single-layer memory structure, and the shared cache of the compute unit can be abstracted as a whole as a single-layer memory structure, maximizing the use of hardware resources such as the tensor operation unit, maximizing the advantages of cache capacity and bandwidth at each hierarchical level, and achieving performance generalization.

[0110] At least one embodiment of this disclosure further provides a data processing method. Figure 8 is a schematic flowchart of the data processing method provided in at least one embodiment of this disclosure.

[0111] For example, the data processing method can be applied to a data processor. The data processor includes a tensor operation unit and a plurality of computing units.

[0112] For example, a tensor operation unit is configured to perform tensor calculations on input data and obtain the tensor calculation results. For example, tensor calculations include matrix multiplication or convolution operations.

[0113] For example, multiple computing units may be configured to perform vector operations on the tensor calculation results, or multiple computing units may be configured to generate input data, or multiple computing units may be configured to perform vector operations on the tensor calculation results and generate input data.

[0114] For example, in some embodiments, the data processor may be a general-purpose graphics processor, and the structure of the data processor can be found in the related description in Figure 1 above. For example, in some embodiments, the tensor operation unit 102 may be a tensor core. The structure of the tensor operation unit 102 can be found in the related description in Figure 2 above.

[0115] The structure and function of the computation unit and tensor operation unit can be found in the related descriptions of computation unit 102 and tensor operation unit 101 in the data processor mentioned above, and will not be described in detail here.

[0116] For example, as shown in Figure 8, the data processing method provided in the embodiments of this disclosure includes at least step S10.

[0117] In step S10, the first data transfer channel is used to directly supply the tensor calculation result to the corresponding computing unit, and the input data is supplied directly from the computing unit that generates the input data to the tensor calculation unit.

[0118] For example, a first data transfer channel is provided between the tensor operation unit and each or some of the calculation units. The first data transfer channel is configured to directly supply the tensor calculation results calculated by the tensor operation unit 101 to the corresponding calculation unit 102, and to directly supply input data from the calculation unit 102 that generates the input data to the tensor operation unit 101.

[0119] In at least one embodiment of this disclosure, by adding a bidirectional interconnection channel, i.e., a first data transfer channel, between the tensor operation unit and the compute unit to the data processor, the compute unit and the tensor operation unit can establish a direct communication connection via the first data transfer channel. Using the first data transfer channel, tensor computation results can be directly supplied to the compute unit, and input data can be directly supplied from the compute unit to the tensor operation unit. As shown in Figure 1, this eliminates the need for data flow through the compute unit's shared cache and intermediate-level cache, reducing access to global memory, minimizing resource waste, reducing data transfer delay, significantly improving the overall efficiency of the operators, effectively utilizing the powerful computing capabilities of the tensor operation unit itself, and improving the computational efficiency of the data processor.

[0120] For example, each computing unit includes registers, which are configured to store data related to vector operations and data related to the generation of input data.

[0121] For example, a tensor operation unit includes a cache block, and the cache block is configured to cache input data.

[0122] For example, the first data transfer channel connects registers in each or some of the computing units to cache blocks in the tensor operation unit.

[0123] As shown in Figure 4, the first data transfer channel 103 connects the registers in each computing unit to the cache blocks in the tensor arithmetic unit. This enables direct data exchange between the tensor arithmetic unit and the computing unit via the first data transfer channel 103, and the tensor arithmetic unit can directly transfer the tensor calculation results to the corresponding registers for subsequent processing (e.g., vector operations). Furthermore, input data in the registers can be directly transferred to the cache blocks of the tensor arithmetic unit for tensor operations. As a result, data transfer no longer needs to flow through the global cache / memory, improving resource utilization, enhancing data access efficiency, avoiding the consumption of data access overhead, reducing the number of accesses to the global cache, global memory, or intermediate-level cache, reducing data transfer delay, and improving tensor arithmetic efficiency.

[0124] Furthermore, since the registers are located closest to the core of the computing unit, have the fastest data transfer speed, and the highest bandwidth, installing a first data transfer channel 103 to connect the registers in the computing unit with the cache blocks in the tensor operation unit can further improve data flow efficiency and thus improve computing efficiency.

[0125] Furthermore, the establishment of the first data transfer channel 103 allows the computing unit to use the memory resources in the tensor operation unit, and the high-speed cache blocks provided in the tensor operation unit can be used by the computing unit, fully demonstrating their value in scenarios where computations are performed solely by the vector operation unit.

[0126] Figure 9 is a schematic flowchart of a data processing method provided by at least one embodiment of the present disclosure.

[0127] For example, as shown in the attached diagrams such as Figure 1, each computing unit includes a shared cache and multiple computing cores, and the shared cache is configured to share data among the multiple computing cores.

[0128] For example, as shown in Figure 9, the data processing method provided by the embodiment of the present disclosure further includes step S20.

[0129] In step S20, the second data transfer channel is used to transfer data between the shared caches of at least some of the computing units among the multiple computing units.

[0130] For example, a second data transfer channel is provided between the shared caches of the computing units, and the second data transfer channel is configured to transfer data between the shared caches of the computing units, and the shared caches of the computing units share data via the second data transfer channel.

[0131] This allows data sharing between computing units to be achieved directly via a shared cache, and data can flow between the shared caches of each computing unit via a second data transfer channel. As a result, data transfer no longer needs to flow through a global cache / memory, improving resource utilization, enhancing data access efficiency, avoiding the consumption of data access overhead, reducing the number of accesses to the global cache, global memory, or intermediate-level cache, and reducing data transfer latency.

[0132] The shared cache is closer to the computing core, and memory resources closer to the computing core have higher data access efficiency and higher data transfer bandwidth. Therefore, by providing a second data transfer channel 104, a high-speed interconnection channel can be provided to the computing units, further improving data flow efficiency, increasing data transfer speed, reducing access overhead, and improving computing efficiency.

[0133] Furthermore, for any computing unit, its operable shared cache is expanded by N times, allowing memory resources to perform at an even greater rate and enabling full utilization of the shared cache closer to the computing core. In some embodiments, some arithmetic and logical operations of a computing unit (e.g., cumulative addition) can be performed by logical operation units in the shared cache. These operations can be performed in advance from the global cache, etc., to the shared cache (especially when data accumulator capacity is insufficient), thereby improving system bandwidth efficiency.

[0134] For example, as shown in Figure 9, the data processing method provided by at least one embodiment of the present disclosure further includes step S30.

[0135] In step S30, logical operations are performed between computing units in the shared cache.

[0136] For example, a logical operation unit is provided within the shared cache, and the logical operation unit is configured to perform logical operations between computing units.

[0137] For example, a logical operation unit can provide an accumulator function. For example, a logical operation unit can include an accumulator. Of course, a logical operation unit can also support arithmetic logic operations such as addition and subtraction as needed, and this disclosure does not make any specific limitations on this.

[0138] For example, after data between computing units enters the shared cache, the logical operation unit directly executes the corresponding arithmetic and logical operations. As a result, there is no need to transfer the data to the computing core via registers to perform arithmetic and logical operations, nor is there a need to transfer it to the shared cache via registers. This avoids repeated reading and writing of data to the shared cache, improves computing efficiency, and reduces data access overhead by circulating data within the shared cache as much as possible.

[0139] For example, as shown in Figure 9, the data processing method provided by at least one embodiment of the present disclosure further includes step S40.

[0140] In step S40, operator fusion is performed.

[0141] For example, operator fusion includes deep fusion of arbitrary operators, and in the operator fusion process, both the tensor operation unit and the vector operation unit can function as task producers or task consumers. The vector operation unit includes the N computation units.

[0142] Here, the vector operation unit includes, for example, N calculation units 102 as shown in Figure 4.

[0143] For example, a task producer is configured to generate computation data, and a task consumer is configured to receive and process the computation data. For example, when a tensor operation unit functions as a task producer, it can generate tensor computation results as computation data, and a vector operation unit, acting as a task consumer, receives and processes the tensor computation results. For example, when a vector operation unit functions as a task producer, it can generate input data as computation data, and a tensor operation unit, acting as a task consumer, receives and processes the input data and generates tensor computation results.

[0144] In at least one embodiment of this disclosure, the first data transfer channel can be used to directly supply tensor computation results to the computation unit and input data directly from the computation unit to the tensor computation unit, increasing the interconnection between registers and the cache blocks built into the tensor computation unit and enhancing the possibility of deep fusion of fusion operators. The fusion strategy for fusion operators is no longer limited to the tensor computation core acting as the driver and the vector computation core as the co-consumer, and deep fusion of any operator can be achieved. Depending on the fusion strategy, the tensor computation unit can drive operators as a task producer and receive and process computation data as a task consumer, and similarly, the vector computation unit can drive operators as a task producer and receive and process computation data as a task consumer. This results in greater access efficiency than loading data via a cache or external storage area.

[0145] For example, as shown in Figure 9, the data processing method provided by at least one embodiment of the present disclosure further includes step S50.

[0146] In step S50, the shared cache of at least some of the computing units is abstracted as a single intermediate-level cache layer in the data processor's abstraction hierarchy.

[0147] When establishing data interconnection channels between shared caches, the shared caches of multiple computing units establishing the data interconnection channels can be abstracted as a single intermediate-level cache layer within the data processor's abstraction hierarchy, allowing for development and implementation as a single, integrated unit during software development.

[0148] For example, a data processing method provided by at least one embodiment of the present disclosure may further include abstracting a cache block in a tensor operation unit as an intermediate-level cache layer in the abstraction hierarchy of a data processor.

[0149] In at least one embodiment of this disclosure, by providing a first data transfer channel and a second data transfer channel, it is possible to abstract a better software access and computation hierarchy structure from the software layer. Furthermore, it is possible to abstract the partitioning and scheduling logic of computation tasks, maximize the use of hardware resources such as tensor operation units, and achieve performance generalization.

[0150] For more detailed information on the first data transfer channel, second data transfer channel, logical operation unit, etc., please refer to the relevant descriptions of the data processor mentioned above. Duplicate sections will not be elaborated again.

[0151] At least one embodiment of the present disclosure further provides a non-temporary computer-readable storage medium. Figure 10 is a schematic diagram of a non-temporary computer-readable storage medium provided by at least one embodiment of the present disclosure.

[0152] For example, as shown in Figure 10, the storage medium 200 may be a non-temporary computer-readable storage medium, and one or more computer-readable instructions 201 can be stored non-temporarily on the storage medium 200. For example, when a computer-readable instruction 201 is executed by a processor, one or more steps of the data processing method described above can be executed.

[0153] For example, the storage medium 200 can be applied to the data processor described above.

[0154] For example, the storage medium may include any combination of one or more computer program products. The computer program products may include various types of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache). Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB storage device, flash memory, etc. The computer-readable storage medium may store one or more computer-readable instructions, and the data processor may execute the computer-readable instructions to realize various functions of the data processor. The storage medium may further store various application programs and various data.

[0155] For example, the storage medium may include a smartphone memory card, a tablet computer memory component, a personal computer hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above-mentioned storage mediums, or other applicable storage mediums.

[0156] At least one embodiment of the present disclosure further provides an electronic device. Figure 11 is a schematic diagram of an electronic device provided by at least one embodiment of the present disclosure.

[0157] The electronic devices in the embodiments of this disclosure include, but are not limited to, mobile devices such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. The electronic devices shown in Figure 11 are merely examples and do not impose any limitations on the functionality and scope of use of the embodiments of this disclosure.

[0158] For example, as shown in Figure 11, in some examples the electronic device 300 includes a data processor 301, and the structure, function, and technical effects of the data processor 301 are described in detail below, with reference to the data processor 100 mentioned earlier.

[0159] For example, the data processor can perform various appropriate operations and processes based on programs stored in read-only memory (ROM) 302 or programs loaded from storage device 308 into random access memory (RAM) 303. RAM 303 also stores various programs and data necessary for the operation of the computer system. The data processor 301, ROM 302, and RAM 303 are interconnected via bus 304. An input / output (I / O) interface 305 is also connected to bus 304.

[0160] For example, the data processor 301 may be a general-purpose graphics processor or a graphics processor, and the electronic device 300 may be any hardware device that uses or includes the general-purpose graphics processor, for example, a computing device that includes the data processor, as required by the actual circumstances, and the disclosure does not specifically limit this.

[0161] For example, the following components can be connected to the I / O interface 305: input devices 306 including touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including magnetic tape, hard disks, etc.; and communication devices 309 including network interface cards such as LAN cards and modems. The communication devices 309 enable the electronic device 300 to exchange data with other devices via wireless or wired communication and to perform communication processing via a network such as the Internet. A driver 310 is also connected to the I / O interface 305 as needed. Removable media 311 such as magnetic disks, optical disks, magneto-optical disks, and semiconductor memory are installed in the driver 310 as needed, and computer programs read from them are installed in the storage device 308 as needed. Figure 11 shows an electronic device 300 including various devices, but it should be understood that it is not required to implement or include all the devices shown. Alternatively, more or fewer devices may be implemented or included.

[0162] For example, the electronic device 300 may further include an external interface (not shown in the figure), etc. The external interface can be of various types, such as a USB interface, a Lightning interface, etc. The communication device 309 can communicate with networks and other devices via wireless communication, and the network is a wireless network such as the Internet, an intranet and / or a mobile phone network, a wireless local area network (LAN) and / or a metropolitan area network (MAN). Wireless communication may use any one of several communication standards, protocols, and technologies, including, but not limited to, Global Mobile Communication System (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth®, Wi-Fi (e.g., based on IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n standards), Internet Protocol-based Voice Transmission (VoIP), Wi-MAX, protocols for email, instant messaging and / or Short Message Service (SMS), or any other suitable communication protocol.

[0163] For example, the electronic device can be any device such as a mobile phone, tablet computer, notebook computer, e-reader, game console, television, digital photo frame, or navigator. It can also be any combination of electronic device and hardware, and the embodiments of this disclosure are not limited thereto.

[0164] The following points need to be explained regarding this disclosure: (1) The accompanying drawings of the embodiments of this disclosure relate only to the structures relating to the embodiments of this disclosure, and other structures can be referenced from conventional designs. (2) Where there is no inconsistency, the embodiments and features of the embodiments of this disclosure can be combined with each other to obtain new embodiments.

[0165] The foregoing describes only specific methods for implementing the Disclosure, but the scope of protection of the Disclosure is not limited thereto, and the scope of protection of the Disclosure should be based on the scope of protection of the claims described above. [Explanation of Symbols]

[0166] 100: Data Processor 101: Tensor Unit 102_1,102_2,102_N: Computation Units 103: First data transfer channel 104: Second data transfer channel

Claims

1. A data processor comprising a tensor operation unit and N computation units, where N is a positive integer greater than 1. The tensor calculation unit is configured to perform a tensor calculation on input data to obtain a tensor calculation result, and the tensor calculation includes matrix multiplication or convolution operations. The N computation units are configured to perform at least one of the following: performing vector operations on the tensor computation results and generating the input data. A first data transfer channel is provided between the tensor calculation unit and at least some of the N calculation units, and the first data transfer channel is configured to directly supply the tensor calculation results to the corresponding calculation unit and to directly supply the input data from the calculation unit that generates the input data to the tensor calculation unit. Data processor.

2. Each computing unit includes a register, which is configured to store data related to the vector operation and data related to the generation of the input data. The tensor operation unit includes a cache block, and the cache block is configured to cache the input data. The first data transfer channel connects the registers in at least some of the computing units to the cache blocks. The data processor according to claim 1.

3. Each computing unit includes a shared cache and multiple computing cores, and the shared cache is configured to share data among the multiple computing cores included in the computing unit. A logical operation unit is provided in the shared cache. The logical operation unit is configured to perform logical operations between calculation units, and the logical operations between calculation units include performing arithmetic logic operations on data from different calculation units. The data processor according to claim 1.

4. The aforementioned arithmetic logic operation includes cumulative addition, and the logic operation unit includes an accumulator. The data processor according to claim 3.

5. Each computing unit includes a shared cache and multiple computing cores, and the shared cache is configured to share data among the multiple computing cores included in the computing unit. A second data transfer channel is provided between the shared caches of some or all of the N computing units, and the second data transfer channel is configured to transfer data between the shared caches of some or all of the computing units. The data processor according to claim 1.

6. The data processor includes a plurality of streaming processor clusters, each streaming processor cluster includes a plurality of computing units and one tensor operation unit. The second data transfer channel is further configured to transfer data between the shared caches of the compute units in the plurality of streaming processor clusters. The data processor according to claim 5.

7. The data processor is further configured to perform operator fusion, The fusion of operators includes performing a deep fusion of arbitrary operators, and in the operator fusion process, the tensor operation unit and the vector operation unit can both function as a task producer or a task consumer, the vector operation unit includes the N computation units, the task producer is configured to generate computation data, and the task consumer is configured to receive and process the computation data. A data processor according to any one of claims 1 to 6.

8. In the abstraction hierarchy of the data processor, the shared cache of some or all of the computing units is abstracted as a single intermediate-level cache layer. The data processor according to any one of claims 5 to 7.

9. A data processing method used in a data processor, wherein the data processor includes a tensor operation unit and a plurality of computing units, The tensor operation unit is configured to perform tensor calculations on input data to obtain tensor calculation results, and the tensor calculations include matrix multiplication or convolution operations. The plurality of computing units are configured to perform at least one of the following: perform a vector operation on the tensor calculation result and generate the input data. The aforementioned data processing method is: Using the first data transfer channel, the tensor calculation result is directly supplied to the corresponding computing unit, The input data is supplied directly from the calculation unit that generates the input data to the tensor calculation unit, including, Data processing method.

10. Each computing unit includes a register, which is configured to store data related to the vector operation and data related to the generation of the input data. The tensor operation unit includes a cache block, and the cache block is configured to cache the input data. The first data transfer channel connects registers in at least some of the plurality of computing units to the cache block. The data processing method according to claim 9.

11. Each computing unit includes a shared cache and multiple computing cores, and the shared cache is configured to share data among the multiple computing cores included in the computing unit. The aforementioned data processing method is: In the shared cache, logical operations are performed between computing units, and these logical operations between computing units involve performing arithmetic logic operations on data from different computing units. Further including, The data processing method according to claim 9.

12. The aforementioned arithmetic logic operation includes cumulative addition, and the logic operation unit includes an accumulator. The data processing method according to claim 11.

13. Each computing unit includes a shared cache and multiple computing cores, and the shared cache is configured to share data among the multiple computing cores included in the computing unit. The aforementioned data processing method is: To transfer data between the shared caches of at least some of the multiple computing units using the second data transfer channel. Further including, The data processing method according to claim 9.

14. The data processor includes a plurality of streaming processor clusters, each streaming processor cluster includes a plurality of computing units and one tensor operation unit. The aforementioned data processing method is: To transfer data between the shared caches of the compute units in the multiple streaming processor clusters using the second data transfer channel. Further including, The data processing method according to claim 13.

15. Abstracting the shared cache of at least some of the computing units as a whole as one intermediate-level cache layer in the abstraction hierarchy of the data processor. The data processing method according to claim 13 or 14, further comprising:

16. This further includes performing operator fusion, The fusion of operators includes performing a deep fusion of arbitrary operators, and in the process of the fusion of operators, the tensor operation unit and the vector operation unit can both function as a task producer or a task consumer, the vector operation unit includes the plurality of computation units, the task producer is configured to generate computation data, and the task consumer is configured to receive and process the computation data. The data processing method according to any one of claims 9 to 15.

17. An electronic device comprising a data processor according to any one of claims 1 to 8.

18. A non-temporary computer-readable storage medium in which instructions are stored, When the aforementioned instruction is executed by the processor, the data processing method described in any one of claims 9 to 16 is realized. Non-temporary computer-readable storage medium.

Images