Data processing method and device, chip, equipment and medium

Abstract

The invention provides a data processing method and device, a chip, equipment and a medium, and relates to the technical field of artificial intelligence, in particular to the technical fields of neural networks, deep learning, artificial intelligence chips and the like. The method comprises the steps that a computational graph is obtained, the computational graph describes a processing mode of target hardware on target data, and the target hardware comprises a plurality of clusters and at least one core of each cluster; an alternating operator is introduced into the computational graph, the alternating operator is configured to divide the target data into a plurality of continuous data segments and define the carrying relation between at least one core of each of the plurality of clusters and the plurality of data segments, and two adjacent data segments in the target data are carried by the cores from different clusters; compiling the computational graph to obtain an instruction code for the target hardware; and instructing the target hardware to process the target data by using the instruction code.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to technical fields such as neural networks, deep learning, and artificial intelligence chips, and specifically to a data processing method, a data processing device, an integrated circuit chip, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, training of neural network models, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include natural language processing technology, computer vision technology, speech recognition technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0003] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0004] The present disclosure provides a data processing method, a data processing device, an integrated circuit chip, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to another aspect of the present disclosure, a data processing method is provided, including: obtaining a computation graph, the computation graph describing a processing method of target data by target hardware, the target hardware including multiple clusters and at least one core of each of the multiple clusters; introducing an alternating operator into the computation graph, wherein the alternating operator is configured to divide the target data into multiple continuous data segments, and define a transport relationship between at least one core of each of the multiple clusters and the multiple data segments, wherein two adjacent data segments in the target data are transported by cores from different clusters; compiling the computation graph to obtain instruction codes for the target hardware; and instructing the target hardware to process the target data using the instruction codes.

[0006] According to one aspect of the present disclosure, a data processing device is provided, including: a computation graph acquisition unit, configured to acquire a computation graph, the computation graph describing a processing method of target data by target hardware, the target hardware including multiple clusters and at least one core of each of the multiple clusters; an alternating operator introduction unit, configured to introduce an alternating operator into the computation graph, wherein the alternating operator is configured to divide the target data into multiple continuous data segments, and define a transport relationship between at least one core of each of the multiple clusters and the multiple data segments, wherein two adjacent data segments in the target data are transported by cores from different clusters; a computation graph compilation unit, configured to compile the computation graph to obtain an instruction code for the target hardware; and a data processing unit, configured to instruct the target hardware to process the target data using the instruction code.

[0007] According to another aspect of the present disclosure, an integrated circuit chip is provided, comprising the above-mentioned data processing device.

[0008] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above method.

[0009] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the above method.

[0010] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0011] According to one or more embodiments of the present disclosure, the present disclosure designs a new alternation factor and introduces it into the computation graph, so that when multiple clusters use the instruction code obtained by compiling the computation graph to perform data processing, they can alternately transfer adjacent data segments, thereby making full use of bandwidth resources and making the load more balanced. In addition, this method does not need to modify the original semantics of the compiled programming language, and has better compatibility.

[0012] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0014] Figure 1 A flow chart showing a data processing method according to an exemplary embodiment of the present disclosure is shown;

[0015] Figure 2A-2D A schematic diagram showing a data transport method according to an exemplary embodiment of the present disclosure is shown;

[0016] Figure 3 A structural block diagram showing a data processing device according to an exemplary embodiment of the present disclosure; and

[0017] Figure 4 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0018] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0019] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0020] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0021] In the related art, during data transport and data distribution, adjacent data segments are handed over to adjacent cores in the same cluster for processing, which results in a waste of hardware resources.

[0022] To solve the above problems, the present disclosure designs a new alternation factor and introduces it into the computation graph, so that when multiple clusters use the instruction code obtained by compiling the computation graph to process data, they can alternately transfer adjacent data segments, thereby making full use of bandwidth resources and making the load more balanced. In addition, this method does not need to modify the original semantics of the compiled programming language, and has better compatibility.

[0023] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0024] According to one aspect of the present disclosure, a data processing method is provided. Figure 1 FIG. 1 is a flow chart showing a data processing method 100 according to an exemplary embodiment of the present disclosure. Figure 1 As shown, the data processing method includes: step S101, obtaining a calculation graph, the calculation graph describes the target hardware's processing method for the target data, the target hardware includes multiple clusters and at least one core of each of the multiple clusters; step S102, introducing an alternating operator into the calculation graph, wherein the alternating operator is configured to divide the target data into multiple continuous data segments, and define a transport relationship between at least one core of each of the multiple clusters and the multiple data segments, wherein two adjacent data segments in the target data are transported by cores from different clusters; step S103, compiling the calculation graph to obtain an instruction code for the target hardware; and step S104, instructing the target hardware to process the target data using the instruction code.

[0025] Therefore, the present disclosure designs a new alternation factor and introduces it into the computation graph, so that when multiple clusters use the instruction code obtained by compiling the computation graph to process data, they can alternately transfer adjacent data segments, thereby making full use of bandwidth resources and making the load more balanced. In addition, this method does not need to modify the original semantics of the compiled programming language, and has better compatibility.

[0026] The method, device and chip provided in the present disclosure have better adaptability to the Triton language and the domain-specific architecture (DSA) hardware in the artificial intelligence (AI) chip. The present disclosure will mainly use the Triton language and DSA hardware as examples for illustration. However, it should be noted that this does not mean to limit the scope of the present disclosure, and the technical solution of the present disclosure is also applicable to other programming languages ​​and other hardware.

[0027] Triton is a new type of AI compiler. The programming language it provides (Triton Language) is an open source programming language similar to Python. Its goal is to provide an open source environment to quickly write code with higher productivity than the Compute Unified Device Architecture (CUDA), and it has higher flexibility than other existing domain-specific language (DSL) compilers. Therefore, Triton's design is more friendly to the hardware of the Graphics Processing Unit (GPU), but it is not well adapted to the DSA architecture chips in AI chips.

[0028] Usually, the Block in the GPU corresponds to the Cluster in the DSA, and the Thread corresponds to the Core. When moving data (especially the Triton language), adjacent data segments can be handed over to adjacent cores for moving. The data segments moved within a cluster are continuous. Figure 2A As shown, assuming that the total number of data elements of the target data n_elements = 384, the total number of clusters cluster_num = 2, the number of cores in each cluster core_num = 4, and the number of data elements processed by each thread element_per_thread = 64, that is, a total of 384 data are implemented by 2 clusters Cluster, each cluster has 4 cores Core, and each core Core carries 64 numbers. Then for continuous data, subscripts 0-63 are carried by cluster 0core 0, 64-127 are carried by cluster 0core 1... 256-319 are carried by cluster 1core 0, 320-383 are carried by cluster 1core 1, and finally core 2 and core 3 of cluster 1 do not carry data.

[0029] The above method has the following problems:

[0030] 1. Each Cluster often corresponds to a channel of hardware to carry data, such as Figure 2B As shown in the figure, the left side is channel 0 corresponding to cluster cluster 0, and the right side is channel 1 corresponding to cluster cluster 1. When data is transferred in the above manner, only one channel can be used at the same time, resulting in a waste of hardware resources.

[0031] 2. When accessing data (for example, storing in memory), all clusters first access similar addresses and then move up together, which is advantageous for accessing GDDR. However, the current reading method is that 0 must wait until 0-63 is moved before 64-127 can be moved, resulting in a congestion.

[0032] 3. There is a problem of load imbalance for small-scale data. For example, if one cluster can complete the calculation of data, the other clusters will not participate in the calculation, and the hardware resources are not fully utilized. Figure 2A As shown in the figure, Cluster 0 is fully loaded, while Core 2 and Core 3 of Cluster 1 are actually idle. That is, the loads of the two Clusters are unbalanced. The overall calculation needs to wait until Cluster 0 is finished. This affects the performance of the operator to a certain extent.

[0033] In response to the above problems, the present disclosure proposes a data processing method, designs a new alternation factor, and introduces it into the computation graph, so that when multiple clusters use the instruction code obtained by compiling the computation graph for data processing, they can alternately transfer adjacent data segments, thereby making full use of bandwidth resources and making the load more balanced. In addition, this method does not need to modify the original semantics of the compiled programming language, and has better compatibility.

[0034] For ease of description, the data processed by the method of the present disclosure is referred to as target data, and the hardware used to process the target data is referred to as target hardware. In an embodiment of the present disclosure, the target hardware includes multiple clusters, each of which includes at least one core.

[0035] In step S201, a computation graph corresponding to the target data to be processed may be obtained. In the computation graph, each operation may be viewed as a node of the graph. By generating a graphical representation, specific hardware mapping and optimization may be performed in subsequent stages.

[0036] In step S202, an alternation operator may be introduced into the computation graph. The alternation operator is configured to divide the target data into a plurality of continuous data segments, and define a transport relationship between at least one core of each of the plurality of clusters and the plurality of data segments. Any two adjacent data segments in the target data are transported by cores from different clusters.

[0037] According to some embodiments, in step S202, the alternation operator may be configured to divide the target data into a plurality of continuous data segments according to the number of data elements processed by each thread of the target hardware. In this way, load balancing between multiple clusters and cores can be better achieved, and it is convenient to determine the data segments that each core needs to carry.

[0038] According to some embodiments, the alternation operator can be configured to: determine an intermediate result based on the core number of the current core, the cluster number of the current cluster and the total number of clusters, wherein the intermediate result indicates the nth data segment among multiple data segments that has a transport relationship with the current core; and determine the starting subscript offset of the data segment of the current core based on the intermediate result and the number of data elements processed per thread, wherein the starting subscript offset of the data segment of the current core is used to indicate the index of the data segment that has a transport relationship with the current core.

[0039] Through the above method, the data segment corresponding to each core in each cluster can be conveniently determined.

[0040] In order to make full use of the hardware bandwidth, adjacent data segments are preferentially allocated to the corresponding cores of different clusters, and data is transferred alternately between clusters. The above intermediate result can be expressed as tid = core number core_id() of the current core * total number of clusters cluster_num() + cluster number cluster_id() of the current cluster. The starting subscript offset of the data segment can be expressed as offset = intermediate result tid * number of data elements processed by each thread element_per_thread.

[0041] like Figure 2C As shown, in this example, the total number of clusters cluster_num is 2, and the number of data elements processed by each thread element_per_thread is 64. The alternating operator can divide the target data into six data segments of 0-63, 64-127...320-384 according to the number of data elements processed by each thread element_per_thread. According to the above calculation formula, for the kth core of the i-th cluster cluster, the intermediate result tid=k*2+i, and the data segment starting subscript offset offset=tid*64.

[0042] Specifically, the intermediate result corresponding to core 0 of cluster cluster 0 is tid=0, and the starting subscript offset of the data segment is offset=0, that is, the data segment with subscripts 0-63 is carried by core 0 of cluster 0; the intermediate result corresponding to cluster 1 core 0 is tid=1, and the starting subscript offset of the data segment is offset=64, that is, the data segment with subscripts 64-127 is carried by core 0 of cluster 1; the intermediate result corresponding to cluster 0 core 1 is tid=2, and the starting subscript offset of the data segment is offset=128, that is, the data segment with subscripts 128-191 is carried by core 1 of cluster 0, and so on.

[0043] This can effectively eliminate the two original disadvantages:

[0044] 1. Cluster 0 and cluster 1 are used to move data from 0 to 64 and from 64 to 127 respectively. The same is true for other data. No congestion occurs.

[0045] 2. Core 3 of Cluster 0 and Cluster 1 are both idle, and the loads of the two are more balanced. There is no longer a situation where Cluster 1 finishes calculation first and then waits for Cluster 0.

[0046] It should be noted that Figure 2C The example includes two clusters, each cluster includes 4 cores. The method disclosed in the present invention is also applicable to other numbers of clusters and other numbers of cores, which are not limited here.

[0047] It is understandable that the computation graph may include other operators besides the alternation operator, which is not limited here.

[0048] In step S203, the computation graph may be compiled into instruction codes that can be directly run on hardware, and then in step S204, the target hardware may be instructed to execute the instruction codes to complete the processing of the target data.

[0049] According to some embodiments, the target hardware may include multiple channels corresponding to the multiple clusters one by one. Step S204, instructing the target hardware to process the target data using the instruction code may include: instructing the multiple clusters to process the target data in parallel through the multiple channels.

[0050] Through the above method, data corresponding to multiple clusters can be processed in parallel, reducing conflicts between channels and making full use of bandwidth.

[0051] like Figure 2DAs shown in the figure, the left side is channel 0 corresponding to cluster 0, and the right side is channel 1 corresponding to cluster 1. When data is transferred in the above manner, both channels can be used at the same time, thus making full use of hardware resources.

[0052] According to some embodiments, the target data is stored in a graphics dedicated double data rate memory (GDDR).

[0053] The method proposed in the present disclosure has additional advantages for data stored in GDDR. When accessing dynamic random access memory (DRAM), all clusters initially access similar addresses, and then multiple clusters gradually and sequentially access adjacent memory address intervals, which is advantageous for accessing GDDR. Sequential access to adjacent memory addresses, especially when multiple clusters gradually "go up" in the same direction, will make access to GDDR video memory faster. Through this mode, the prefetch mechanism, row buffer, channel bandwidth, and cache unit are fully utilized, thereby reducing the delay in data transfer and improving the overall memory bandwidth utilization efficiency.

[0054] The data processing method proposed in the present disclosure has additional advantages for the Triton language and compiler, and is compatible with the CUDA Triton kernel without modifying the original semantics of Triton.

[0055] According to some embodiments, step S201, obtaining a computational graph may include: obtaining a data processing code based on the Triton language, the data processing code describing how the target hardware processes the target data; converting the data processing code into a Triton tensor intermediate representation, the Triton tensor intermediate representation including tensor operations and corresponding tensor dimensions; and converting the Triton tensor intermediate representation into a computational graph, the computational graph including a Triton tensor graph intermediate representation, the Triton tensor graph intermediate representation including nodes corresponding to the tensor operations.

[0056] Therefore, through the above method, the transition from Triton Language (TL) to Triton's intermediate representation (Triton Tensor Intermediate Representation (TTIR) and then to Triton Tensor Graph Intermediate Representation (TTGIR)) is realized. This process focuses on gradually converting Triton's high-level code (TL) into a representation suitable for the underlying hardware. Through these steps, the compiler is able to gradually reduce the level of abstraction and add specific optimizations (such as the alternation operator InterleaveOp) in the TTGIR stage to improve the efficiency of parallel computing. This process is fully compatible with the Triton compiler.

[0057] TL is Triton's high-level programming language, similar to Python, specifically designed for efficient parallel programming, especially for GPU computing. Developers can use TL to write data processing code for defined data processing tasks.

[0058] TTIR is the first intermediate representation stage of the Triton compilation process, which converts the Triton language code into an intermediate representation of tensor operations, maintaining the computational semantics of the TL code but making it more suitable for further analysis and optimization by the compiler. In the TTIR stage, the TL code is translated into a series of tensor operations that describe how to perform tasks on parallel hardware. This representation still contains higher-level semantic information, such as tensor operations and tensor dimensions. TTIR is mainly used to optimize the logical structure of tensor computations and lay the foundation for the next step of graphical representation (TTGIR).

[0059] According to some embodiments, step S202, introducing the alternation operator into the computation graph may include: replacing the combination of the program identification operator (ProgramIdOp) and the range generation operator (MakeRangeOp) in the preset pattern with the alternation operator.

[0060] During the Triton compilation process, ProgramIdOp and MakeRangeOp are common operations in the TTGIR stage, which are used to describe the allocation and index range of data in parallel computing. ProgramIdOp is used to obtain the unique identifier (ID) of the current thread in the entire parallel task. In multi-core or multi-threaded parallel computing, ProgramIdOp helps divide the workload. This ID can be used to determine the starting position of the data processed by the current thread or core, thereby dividing the data among different threads. MakeRangeOp is used to generate a range, usually a continuous sequence of indexes. This is similar to the range() function in Python, but here it is used for data indexing in parallel programming. MakeRangeOp helps define the location where each thread traverses the data within the range it is responsible for. After determining the thread ID through ProgramIdOp, MakeRangeOp can generate the data range of the thread, which is convenient for loading or processing data within the range.

[0061] In some embodiments, Triton's preset pattern includes a combination of ProgramIdOp and MakeRangeOp to achieve the transfer of adjacent data segments to adjacent Cores. However, as described above, such a transfer method will result in low transfer efficiency. By replacing the original combination with an alternating operator, the transfer method that satisfies "multiple clusters alternately transfer adjacent data segments" can be written into the low-level hardware instruction code during the Triton compilation process, thereby achieving more efficient data transfer and full utilization of hardware resources.

[0062] According to some embodiments, step S203, compiling the computation graph to obtain instruction code for the target hardware includes: converting the computation graph into a low-level virtual machine dialect, the low-level virtual machine dialect including the calculated starting subscript offset of the data segment of at least one core of each of the multiple clusters; and converting the low-level virtual machine dialect into a low-level intermediate representation, the low-level intermediate representation including instruction code directly used for execution by the target hardware.

[0063] Thus, through the above method, the Triton-based graphical intermediate representation is converted into low-level code so that it can run directly on the hardware. This process is fully compatible with the Triton compiler.

[0064] The Low-Level Vector Machine Dialect (LLVM Dialect) is a representation layer that is closer to the underlying hardware. It is based on the low-level representation of LLVM and specifically describes the detailed instructions for calculations and data handling. At this stage, the compiler gradually converts TTGIR into LLVM instructions so that the code can be mapped to specific hardware operations. In this stage, the compiler converts high-level graphics operations (such as the interleave operator InterleaveOp) into low-level instructions and completes the corresponding index calculations to obtain the starting subscript offset of the data segment for each core. After this stage is completed, the code has been optimized and mapped to a specific hardware instruction format, including detailed memory access, data handling, and calculation instructions.

[0065] The Low-Level Intermediate Representation (LLIR) is the final low-level intermediate representation, which contains an instruction set that can be directly used for hardware execution and is the closest representation to machine code. At this stage, the instructions of LLVM Dialect are further optimized and converted into instructions that the hardware can directly understand and execute. LLIR is ultimately used to instruct the hardware how to execute the program, including data loading, calculation, and transportation. This representation has been optimized to the instruction set of the target hardware, including complete memory addresses, data indexes, and operation steps to ensure that the hardware can run the code efficiently.

[0066] According to another aspect of the present disclosure, a data processing device is provided. Figure 3 The structural block diagram of a data processing device 300 according to an exemplary embodiment of the present disclosure is shown. The data processing device 300 includes: a computation graph acquisition unit 310, configured to acquire a computation graph, the computation graph describes the processing method of target data by target hardware, and the target hardware includes multiple clusters and at least one core of each of the multiple clusters; an alternating operator introduction unit 320, configured to introduce an alternating operator into the computation graph, wherein the alternating operator is configured to divide the target data into multiple continuous data segments, and define the transport relationship between at least one core of each of the multiple clusters and the multiple data segments, wherein two adjacent data segments in the target data are transported by cores from different clusters; a computation graph compilation unit 330, configured to compile the computation graph to obtain instruction codes for the target hardware; and a data processing unit 340, configured to instruct the target hardware to process the target data using the instruction codes.

[0067] It can be understood that the operations and effects of the units 310 to 340 in the data processing device 300 can refer to the above description of steps S101 to S104 in the data processing method 100, and will not be elaborated here.

[0068] According to some embodiments, the interleave operator may be configured to divide the target data into a plurality of continuous data segments according to the number of data elements processed per thread of the target hardware.

[0069] According to some embodiments, the alternation operator can be configured to: determine an intermediate result based on the core number of the current core, the cluster number of the current cluster and the total number of clusters, wherein the intermediate result indicates the nth data segment among multiple data segments that has a transport relationship with the current core; and determine the starting subscript offset of the data segment of the current core based on the intermediate result and the number of data elements processed per thread, wherein the starting subscript offset of the data segment of the current core is used to indicate the index of the data segment that has a transport relationship with the current core.

[0070] According to some embodiments, the target hardware may include a plurality of channels corresponding one-to-one to the plurality of clusters, wherein the data processing unit is configured to instruct the plurality of clusters to process the target data in parallel through the plurality of channels.

[0071] According to some embodiments, the target data may be stored in a graphics-dedicated double data rate memory.

[0072] According to some embodiments, the computational graph acquisition unit may include: an acquisition subunit, configured to acquire a data processing code based on the Triton language, the data processing code describing how the target hardware processes the target data; a first conversion unit, configured to convert the data processing code into a Triton tensor intermediate representation, the Triton tensor intermediate representation including tensor operations and corresponding tensor dimensions; and a second conversion unit, configured to convert the Triton tensor intermediate representation into a computational graph, the computational graph including a Triton tensor graph intermediate representation, the Triton tensor graph intermediate representation including nodes corresponding to the tensor operations.

[0073] According to some embodiments, the alternation operator introduction unit may be configured to replace the combination of the program identification operator (ProgramIdOp) and the range generation operator (MakeRangeOp) in the preset pattern with the alternation operator.

[0074] According to some embodiments, the computation graph compilation unit may include: a third conversion subunit, configured to convert the computation graph into a low-level virtual machine dialect, the low-level virtual machine dialect including the calculated data segment starting subscript offset of at least one core of each of the multiple clusters; and a fourth conversion subunit, configured to convert the low-level virtual machine dialect into a low-level intermediate representation, the low-level intermediate representation including instruction codes directly used for execution by the target hardware.

[0075] According to another aspect of the present disclosure, an integrated circuit chip is provided. The integrated circuit chip includes the above-mentioned data processing device 300 .

[0076] According to an embodiment of the present disclosure, an electronic device, a readable storage medium and a computer program product are also provided.

[0077] refer to Figure 4 , a block diagram of an electronic device 400 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0078] like Figure 4 As shown, the electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0079] Multiple components in the electronic device 400 are connected to the I / O interface 405, including: an input unit 406, an output unit 407, a storage unit 408, and a communication unit 409. The input unit 406 can be any type of device that can input information to the electronic device 400. The input unit 406 can receive input digital or character information and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 407 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 408 can include but is not limited to a disk, an optical disk. The communication unit 409 allows the electronic device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0080] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods, processes, and / or processes described above. For example, in some embodiments, these methods, processes, and / or processes may be implemented as computer software programs, which are tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the methods, processes, and / or processes described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform these methods, processes, and / or processes in any other appropriate manner (e.g., by means of firmware).

[0081] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0082] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0083] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0084] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0085] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0086] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0087] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0088] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A data processing method, comprising: Obtaining a computation graph, the computation graph describing a processing method of target data by target hardware, the target hardware comprising a plurality of clusters and at least one core of each of the plurality of clusters; Introducing an alternation operator into the computation graph, wherein the alternation operator is configured to divide the target data into a plurality of continuous data segments, and define a transport relationship between at least one core of each of the plurality of clusters and the plurality of data segments, wherein two adjacent data segments in the target data are transported by cores from different clusters; Compiling the computation graph to obtain instruction codes for the target hardware; and Instruct the target hardware to process the target data using the instruction code.

2. The method according to claim 1, wherein: The alternation operator is configured to divide the target data into a plurality of continuous data segments according to the number of data elements processed by each thread of the target hardware.

3. The method according to claim 2, wherein: The alternation operator is configured as: Determine an intermediate result based on the core number of the current core, the cluster number of the current cluster, and the total number of clusters, wherein the intermediate result indicates the data segment number of the multiple data segments that has a transport relationship with the current core; as well as Based on the intermediate result and the number of data elements processed by each thread, a data segment start subscript offset of the current core is determined, wherein the data segment start subscript offset of the current core is used to indicate an index of a data segment having a transport relationship with the current core.

4. The method according to claim 1, wherein: The target hardware includes a plurality of channels corresponding to the plurality of clusters one by one, wherein instructing the target hardware to process the target data using the instruction code includes: Instruct the multiple clusters to process the target data in parallel through the multiple channels.

5. The method according to claim 4, wherein: The target data is stored in a graphics-dedicated double data rate memory.

6. The method according to any one of claims 1 to 5, wherein: Obtaining the computational graph includes: Acquire a data processing code based on the Triton language, wherein the data processing code describes a processing method of the target hardware on the target data; Converting the data processing code into a Triton tensor intermediate representation, the Triton tensor intermediate representation including tensor operations and corresponding tensor dimensions; and The Triton tensor intermediate representation is converted into the computational graph, wherein the computational graph includes a Triton tensor graph intermediate representation, and the Triton tensor graph intermediate representation includes nodes corresponding to the tensor operation.

7. The method according to claim 6, wherein: Introducing the alternation operator into the computation graph includes: The alternation operator is used to replace the program identification operator and the range generation operator combination in the preset pattern.

8. The method according to claim 6, wherein: Compiling the computation graph to obtain instruction codes for the target hardware includes: Converting the computation graph into a low-level virtual machine dialect, wherein the low-level virtual machine dialect includes the calculated data segment start subscript offset of at least one core of each of the plurality of clusters; and The low-level virtual machine dialect is converted into a low-level intermediate representation, the low-level intermediate representation including the instruction code directly for execution by the target hardware.

9. A data processing device, comprising: A computation graph acquisition unit is configured to acquire a computation graph, wherein the computation graph describes a processing method of target data by target hardware, wherein the target hardware includes a plurality of clusters and at least one core of each of the plurality of clusters; an alternating operator introducing unit, configured to introduce an alternating operator into the computation graph, wherein the alternating operator is configured to divide the target data into a plurality of continuous data segments, and define a transport relationship between at least one core of each of the plurality of clusters and the plurality of data segments, wherein two adjacent data segments in the target data are transported by cores from different clusters; a computation graph compiling unit, configured to compile the computation graph to obtain instruction codes for the target hardware; and The data processing unit is configured to instruct the target hardware to process the target data using the instruction code.

10. The device according to claim 9, wherein: The alternation operator is configured to divide the target data into a plurality of continuous data segments according to the number of data elements processed by each thread of the target hardware.

11. The device according to claim 10, wherein: The alternation operator is configured as: Determine an intermediate result based on the core number of the current core, the cluster number of the current cluster, and the total number of clusters, wherein the intermediate result indicates the data segment number of the multiple data segments that has a transport relationship with the current core; as well as Based on the intermediate result and the number of data elements processed by each thread, a data segment start subscript offset of the current core is determined, wherein the data segment start subscript offset of the current core is used to indicate an index of a data segment having a transport relationship with the current core.

12. The device according to claim 9, wherein: The target hardware includes a plurality of channels corresponding one-to-one to a plurality of clusters, wherein the data processing unit is configured to instruct the plurality of clusters to process the target data in parallel through the plurality of channels.

13. The device according to claim 12, wherein: The target data is stored in a graphics-dedicated double data rate memory.

14. The device according to any one of claims 9 to 13, wherein: The calculation graph acquisition unit includes: an acquisition subunit, configured to acquire a data processing code based on the Triton language, wherein the data processing code describes a processing method of the target hardware on the target data; A first conversion unit is configured to convert the data processing code into a Triton tensor intermediate representation, wherein the Triton tensor intermediate representation includes a tensor operation and a corresponding tensor dimension; and The second conversion unit is configured to convert the Triton tensor intermediate representation into the computational graph, wherein the computational graph includes a Triton tensor graph intermediate representation, and the Triton tensor graph intermediate representation includes nodes corresponding to the tensor operation.

15. The device according to claim 14, wherein: The alternation operator introduction unit is configured to replace the program identification operator and the range generation operator combination in the preset pattern with the alternation operator.

16. The device according to claim 14, wherein: The computation graph compilation unit includes: A third conversion subunit is configured to convert the computation graph into a low-level virtual machine dialect, where the low-level virtual machine dialect includes the calculated data segment start subscript offset of at least one core of each of the plurality of clusters; and The fourth conversion subunit is configured to convert the low-level virtual machine dialect into a low-level intermediate representation, where the low-level intermediate representation includes the instruction code directly used for execution by the target hardware.

17. An integrated circuit chip, comprising: A device according to any one of claims 9 to 16.

18. An electronic device, comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

19. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

20. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images