Memory access system, method, first graphics processor, electronic device

Abstract

This disclosure relates to the field of chip technology, proposing a memory access system, method, first graphics processor, and electronic device. The system includes a copy engine and a cache with computational capabilities. The first graphics processor executes a first command from the central processing unit (CPU). When the execution conditions of the first command are met, the copy engine responds to the first command by copying a first operand and transferring the first operand to the cache in the graphics processor corresponding to a second address. The cache performs operations on the first operand and the second operand, and writes the result to the second address. In this embodiment, the memory access system is located outside the core of the first graphics processor and possesses computational capabilities. It can perform simple mathematical operations on behalf of the graphics processor core, thereby freeing up the computing resources of the graphics processor core. Furthermore, the path for moving data to the memory access system is shorter, which can improve command execution efficiency and reduce power consumption.

Description

Technical Field

[0001] This disclosure relates to the field of chips, and more particularly to a memory access system, method, first graphics processor, and electronic device. Prior Technology

[0002] The development of Large Language Models (LLMs) has made Natural Language Processing (NLP) a popular branch of general artificial intelligence. Due to the large scale of LLMs, distributed training is usually adopted, which uses multiple Graphics Processing Units (GPUs) to complete the training process of the large language models.

[0003] The all-reduce algorithm is a key technology in distributed training, applicable to both model parallelism and data parallelism. The algorithm primarily involves data movement and simple mathematical operations such as addition and subtraction. Current graphics processing units (GPUs) utilize GPU cores to perform data movement and computation when executing the all-reduce algorithm. However, GPU cores have highly complex pipelines and powerful computing capabilities; using them for these operations results in only a small fraction of their computational power being utilized, leading to wasted computing resources. Furthermore, the path for moving data to the GPU core is relatively long, reducing the efficiency of the all-reduce algorithm and increasing power consumption. Summary of the Invention

[0004] In view of this, this disclosure proposes a memory access system, method, first graphics processor, and electronic device. The memory access system of this disclosure is located within the first graphics processor, but outside the core of the first graphics processor. The memory access system has computing power and can perform simple mathematical operations on behalf of the graphics processor core, thereby freeing up the computing resources of the graphics processor core. Furthermore, the path for moving data to the memory access system is shorter, which can improve command execution efficiency and reduce power consumption.

[0005] According to one aspect of this disclosure, a memory access system is provided, the system being disposed in a first graphics processor, the system including a copy engine and a cache with computational capabilities, both the copy engine and the cache being disposed outside the core of the first graphics processor, wherein the first graphics processor executes a first command from a central processing unit, the first command including a first address and a second address, the first address serving as a source operand address in the first command, and the second address serving as both a source operand address and a destination operand address in the first command; when the execution conditions of the first command are met, the copy engine is used to respond to the first command by copying a first operand stored at the first address and transferring the first operand to the cache in the graphics processor corresponding to the second address; the cache is used to perform computation on the first operand and a second operand stored at the second address upon receiving the first operand, and write the computation result to the second address.

[0006] In one possible implementation, the first command is one of a plurality of commands generated by the central processing unit (CPU). The plurality of commands generated by the CPU are used to enable a plurality of graphics processors (GPUs) to jointly perform a ring-shaped global reduction operation. The first GPU is one of the plurality of GPUs. The execution condition of the first command is that the first command does not depend on other commands, or the copy engine detects that the first GPU has received a message indicating that the execution of the second command, which the first command depends on, has been completed. In this case, the first address included in the first command is the same as the second address included in the second command.

[0007] In one possible implementation, the cache is further configured to transmit a message indicating that the operation result has been written to the copy engine; the copy engine is further configured to, upon receiving the message indicating that the operation result has been written to the cache, transmit a message indicating that the first command has been executed to the graphics processor for executing a third command, wherein the third command depends on the first command and the first address included in the third command is the same as the second address included in the first command.

[0008] In one possible implementation, the cache is specifically used for: upon receiving the first operand, performing atomic operations on the first operand and the second operand; and writing the result of the operation to the second address via an atomic write operation. In another possible implementation, the first address and the second address correspond to different graphics processors.

[0009] In one possible implementation, the first graphics processor further includes an on-chip interconnect bus connecting the cache and the copy engine. The first graphics processor is connected to the second graphics processor via an inter-chip interconnect bus. When the first address corresponds to the second graphics processor and the second address corresponds to the first graphics processor, the copy engine reads the first operand through the on-chip interconnect bus and the inter-chip interconnect bus, and transmits the first operand through the on-chip interconnect bus; or when the first address corresponds to the first graphics processor and the second address corresponds to the second graphics processor, the copy engine reads the first operand through the on-chip interconnect bus and transmits the first operand through the on-chip interconnect bus and the inter-chip interconnect bus.

[0010] In one possible implementation, the first graphics processor executes a fourth command from the central processing unit, the fourth command including a third address and a fourth address, the third address serving as the source operand address and the fourth address serving as the destination operand address in the fourth command; when the execution conditions of the fourth command are met, the copy engine is further configured to respond to the fourth command by copying the third operand stored at the third address and transferring the third operand to the cache in the graphics processor corresponding to the fourth address; the cache is further configured to write the third operand to the fourth address upon receiving the third operand.

[0011] In one possible implementation, the cache supports any one of the following operation types: addition, subtraction, multiplication, division, finding the maximum value, and finding the minimum value.

[0012] In one possible implementation, the plurality of graphics processors (GPUs) each store a plurality of data to be processed. The result of the ring-wide reduction operation performed jointly by the plurality of GPUs is the result of the operation on the plurality of data to be processed. When the first command does not depend on other commands, the data stored at the first address of the first command is a part of the data to be processed stored on the GPU corresponding to the first address, and the data stored at the second address of the first command is a part of the data to be processed stored on the GPU corresponding to the second address. When the first command depends on other commands, the data stored at the first address of the first command is the result of the operation on a portion of the data to be processed, and the data stored at the second address of the first command is a part of the data to be processed stored on the GPU corresponding to the second address, or the result of the operation on a portion of the data to be processed.

[0013] According to another aspect of this disclosure, a memory access method is provided, applied to a memory access system, the system being disposed in a first graphics processor, the system including a copy engine and a cache with computational functions, the copy engine and the cache being disposed outside the core of the first graphics processor, the method comprising: the first graphics processor executing a first command from a central processing unit, the first command including a first address and a second address, the first address serving as a source operand address in the first command, and the second address serving as both a source operand address and a destination operand address in the first command; when the execution conditions of the first command are met, the copy engine responds to the first command by copying a first operand stored at the first address, and transferring the first operand to the cache in the graphics processor corresponding to the second address; when the cache receives the first operand, it performs a computation on the first operand and a second operand stored at the second address, and writes the computation result to the second address.

[0014] In one possible implementation, the first command is one of a plurality of commands generated by the central processing unit (CPU). The plurality of commands generated by the CPU are used to enable a plurality of graphics processors (GPUs) to jointly perform a ring-shaped global reduction operation. The first GPU is one of the plurality of GPUs. The execution condition of the first command is that the first command does not depend on other commands, or the copy engine detects that the first GPU has received a message indicating that the execution of the second command, which the first command depends on, has been completed. In this case, the first address included in the first command is the same as the second address included in the second command.

[0015] In one possible implementation, the method further includes: the cache transmitting a message indicating that the operation result has been written to the copy engine; when the copy engine receives the message indicating that the operation result has been written to the copy engine, it transmits a message indicating that the first command has been executed to the graphics processor for executing a third command, the third command depending on the first command, the first address included in the third command being the same as the second address included in the first command.

[0016] In one possible implementation, when the cache receives the first operand, it performs a calculation on the first operand and the second operand stored at the second address, and writes the calculation result to the second address. This includes: when the cache receives the first operand, it performs an atomic operation on the first operand and the second operand; and writes the calculation result to the second address through an atomic write operation.

[0017] In one possible implementation, the first address and the second address correspond to different graphics processors.

[0018] In one possible implementation, the first graphics processor further includes an on-chip interconnect bus connecting the cache and the copy engine. The first graphics processor is connected to the second graphics processor via an inter-chip interconnect bus. When the first address corresponds to the second graphics processor and the second address corresponds to the first graphics processor, the copy engine reads the first operand through the on-chip interconnect bus and the inter-chip interconnect bus, and transmits the first operand through the on-chip interconnect bus; or when the first address corresponds to the first graphics processor and the second address corresponds to the second graphics processor, the copy engine reads the first operand through the on-chip interconnect bus and transmits the first operand through the on-chip interconnect bus and the inter-chip interconnect bus.

[0019] In one possible implementation, the method further includes: the first graphics processor executing a fourth command from the central processing unit, the fourth command including a third address and a fourth address, the third address serving as a source operand address and the fourth address serving as a destination operand address in the fourth command; when the execution conditions of the fourth command are met, the copy engine responds to the fourth command by copying the third operand stored at the third address and transferring the third operand to the cache in the graphics processor corresponding to the fourth address; when the cache receives the third operand, it writes the third operand to the fourth address.

[0020] In one possible implementation, the cache supports any one of the following operation types: addition, subtraction, multiplication, division, finding the maximum value, and finding the minimum value.

[0021] In one possible implementation, the plurality of graphics processors (GPUs) each store a plurality of data to be processed. The result of the ring-wide reduction operation performed jointly by the plurality of GPUs is the result of the operation on the plurality of data to be processed. When the first command does not depend on other commands, the data stored at the first address of the first command is a part of the data to be processed stored on the GPU corresponding to the first address, and the data stored at the second address of the first command is a part of the data to be processed stored on the GPU corresponding to the second address. When the first command depends on other commands, the data stored at the first address of the first command is the result of the operation on a portion of the data to be processed, and the data stored at the second address of the first command is a part of the data to be processed stored on the GPU corresponding to the second address, or the result of the operation on a portion of the data to be processed.

[0022] According to another aspect of this disclosure, a first graphics processor is provided, including the memory access system described in any of the preceding claims.

[0023] According to another aspect of this disclosure, an electronic device is provided, including the first graphics processor described above.

[0024] According to an embodiment of the memory access system disclosed herein, the system is disposed in a first graphics processor (GPU). The system includes a copy engine and a cache with computational capabilities. The first GPU executes a first command from the central processing unit (CPU). When the execution conditions of the first command are met, the copy engine responds to the first command by copying a first operand stored at a first address, allowing the copy engine to replace the GPU core in copying the first operand. The copy engine then transmits the first operand to the cache in the GPU corresponding to a second address. Upon receiving the first operand, the cache performs computation on the first operand and a second operand stored at a second address, and writes the computation result to the second address. This allows the cache to replace the GPU core in performing computation on the first and second operands and writing the computation result. Both the copy engine and the cache are disposed outside the GPU core. In this case, the memory access system can replace the GPU core in performing copy, computation, and write operations, thereby freeing up the GPU core's computational resources. Furthermore, the paths for copy and write operations performed by the memory access system are shorter, which can improve command execution efficiency and reduce power consumption.

[0025] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Simple Explanation of the Diagram

[0026] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0027] Figure 1 shows a schematic diagram of the process of a prior art graphics processor executing a ring global reduction algorithm.

[0028] Figure 2 shows a schematic diagram of the process of a prior art graphics processor executing a ring global reduction algorithm.

[0029] Figure 3 shows a schematic diagram of the process of a prior art graphics processor executing a ring global reduction algorithm.

[0030] Figure 4a shows a schematic diagram of the process of a prior art graphics processor executing a ring global reduction algorithm.

[0031] Figure 4b shows a schematic diagram of the process of a prior art graphics processor executing a ring global reduction algorithm.

[0032] Figure 5a shows a schematic diagram of a single graphics processor in the prior art processing commands for a global reduction algorithm.

[0033] Figure 5b illustrates a schematic diagram of a prior art graphics processor processing commands for a global reduction algorithm.

[0034] Figure 6a illustrates an exemplary application scenario of a memory access system according to an embodiment of the present disclosure.

[0035] Figure 6b illustrates an exemplary application scenario of a memory access system according to an embodiment of the present disclosure.

[0036] Figure 7a shows a schematic diagram of the structure of a memory access system according to an embodiment of the present disclosure.

[0037] Figure 7b illustrates a method by which a central processing unit generates multiple commands and executes them by a graphics processor according to an embodiment of the present disclosure.

[0038] Figure 7c illustrates a method by which a central processing unit generates multiple commands and executes them by a graphics processor according to an embodiment of the present disclosure.

[0039] Figure 7d illustrates a method by which a central processing unit generates multiple commands and executes them by a graphics processor according to an embodiment of the present disclosure.

[0040] Figure 7e illustrates a method by which a central processing unit generates multiple commands and executes them by a graphics processor according to an embodiment of the present disclosure.

[0041] Figure 7f illustrates a method by which a central processing unit generates multiple commands and executes them by a graphics processor according to an embodiment of the present disclosure.

[0042] Figure 8 illustrates an example of accessing the first address and the second address according to an embodiment of the present disclosure.

[0043] Figure 9 illustrates an example of accessing the first address and the second address according to an embodiment of the present disclosure.

[0044] Figure 10 shows a schematic diagram of the flow of a memory access method according to an embodiment of the present disclosure. Implementation

[0045] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0046] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0047] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0048] The development of Large Language Models (LLMs) has made Natural Language Processing (NLP) a popular branch of general artificial intelligence. Due to the large scale of LLMs, distributed training is usually adopted, which uses multiple Graphics Processing Units (GPUs) to complete the training process of the large language models.

[0049] The all-reduce algorithm is a key technique in distributed training, applicable to both model parallelism and data parallelism. The operations involved in this algorithm primarily include data movement and simple mathematical operations such as addition and subtraction.

[0050] A classic type of global reduction algorithm is the ring all reduce algorithm. Figures 1-4b illustrate the flow of a prior art graphics processor executing the ring all reduce algorithm.

[0051] As shown in Figures 1-4b, assume that GPUs GPU0 through GPU3 are used to execute the ring global reduction algorithm. Data A is stored on GPU0, data B on GPU1, data C on GPU2, and data D on GPU3. The purpose of the ring global reduction algorithm is to ensure that data A+B+C+D are stored on GPUs GPU0 through GPU3 respectively (the data is stored in the corresponding memory or video memory of the GPU).

[0052] In step 1, as shown in Figure 1, a free storage space can be selected from the video memory or system memory (not shown) corresponding to each graphics processor as a shared buffer allocated to that graphics processor and the next graphics processor. The size of each buffer can be the same. Specifically, graphics processors GPU0 and GPU1 share buffer 0, graphics processors GPU1 and GPU2 share buffer 1, graphics processors GPU2 and GPU3 share buffer 2, and graphics processors GPU3 and GPU0 share buffer 3.

[0053] In step 2, as shown in Figure 2, the buffer can be divided into N equal parts (e.g., N=4) based on the number of graphics processors N. The offset between two adjacent data moves for each graphics processor is determined based on the data size of each part, and the data stored by each graphics processor is also divided into multiple parts according to the data size of each part. Each graphics processor moves its own data with the smallest offset (1 / 4) to a buffer shared with the next graphics processor. At this point, buffer 0 stores data A, buffer 1 stores data B, buffer 2 stores data C, and buffer 3 stores data D.

[0054] In step 3, as shown in Figure 3, each graphics processor reads a copy of data previously moved by the previous graphics processor from the buffer shared with the previous graphics processor, performs a calculation on this data and a copy of data stored at the same offset (addition is used as an example in Figure 2), and stores the result at the same offset in the buffer shared with the next graphics processor. At this time, buffer 0 stores data D+A, buffer 1 stores data A+B, buffer 2 stores data B+C, and buffer 3 stores data C+D.

[0055] In step 4, as shown in Figures 4a and 4b, step 3 is repeated until each graphics processor stores the same calculation result A+B+C+D.

[0056] Figures 5a and 5b illustrate schematic diagrams of how a single graphics processor in the prior art processes commands for a global reduction algorithm.

[0057] Existing graphics processing units (GPUs) use GPU cores to process commands for global reduction algorithms, performing data transfer and computation operations. However, GPU cores have very complex pipelines and powerful computing capabilities. When using GPU cores to perform these operations, only a very small portion of their computing power is utilized, resulting in a waste of computing resources.

[0058] The two pieces of data being processed typically lack temporal and spatial locality and are stored in the memory or video memory corresponding to different graphics processors (GPUs). They need to be read from the GPU cores separately, processed in the GPU cores to obtain the results, and then written to the memory or video memory corresponding to one of the GPUs. In the examples of Figures 5a and 5b, (1) and (2) represent the two pieces of data being read, and (3) represents the result of the processing. That is to say, for the GPU core, executing a command requires three memory accesses. Moreover, each memory access by the GPU core requires traversing the cache and bus, so each memory access path is relatively long. Due to the large number of memory accesses and the long memory access path, the memory access efficiency is relatively low and the power consumption is relatively high.

[0059] In view of this, this disclosure proposes a memory access system, method, first graphics processor, and electronic device. The memory access system of this disclosure is located within the first graphics processor, but outside the core of the first graphics processor. The memory access system has computing power and can perform simple mathematical operations on behalf of the graphics processor core, thereby freeing up the computing resources of the graphics processor core. Furthermore, the path for moving data to the memory access system is shorter, which can improve command execution efficiency and reduce power consumption.

[0060] Figures 6a and 6b illustrate exemplary application scenarios of a memory access system according to embodiments of the present disclosure.

[0061] As shown in Figures 6a and 6b, the memory access system of this embodiment can be located within the graphics processor (GPU) but outside the GPU core. The memory access system is also connected to the GPU's main memory or video memory.

[0062] The graphics processing unit (GPU) is also connected to the central processing unit (CPU). During the execution of the global reduction algorithm, the CPU generates multiple commands to be executed in a specific order. These commands can be executed by multiple GPUs (only the first and second GPUs are shown in Figures 6a and 6b), and each GPU can have the same architecture. Each command is executed by one GPU, and executing a command requires copying and writing operations, and may also require computational operations.

[0063] The central processing unit transmits multiple commands to each graphics processing unit (GPU) so that the memory access system can clearly define the execution conditions of commands to be executed by its own GPU. Examples of commands and execution conditions can be found in the following descriptions and Figures 7a-7f.

[0064] The commands executed by the first graphics processor may include the first command described below. Executing the first command requires copying, computation, and writing operations. When the execution conditions of the first command are met, the memory access system responds to the first command, copies data A and data B, performs a computation on data A and data B, and writes the computation result A+B (taking addition as an example) to the address of one of the two data sets being operated on. In the examples of Figures 6a and 6b, the write address of the computation result A+B and the storage address of data A can be the same. In the example of Figure 6a, the storage address of data A can be an address in the memory or video memory of the first graphics processor, and the storage address of data B can be an address in the memory or video memory of the second graphics processor. In the example of Figure 6b, the storage address of data A can be an address in the memory or video memory of the second graphics processor, and the storage address of data B can be an address in the memory or video memory of the first graphics processor. The execution of the first command does not require the participation of the core of the first graphics processor.

[0065] Multiple commands may also include commands that only involve copying and writing, without any computation (such as the fourth command described below). In this case, the command is still executed by the memory access system, and the execution effect may be to copy the data stored in the memory, video memory, or cache (not shown) of one graphics processor and write it to the memory, video memory, or cache (not shown) of another graphics processor.

[0066] The effect of executing multiple commands in sequence can be seen in the relevant descriptions in Figures 7b-7f below.

[0067] Those skilled in the art should understand that the memory access system can also be applied to other types of global reduction scenarios besides the ring global reduction scenario, or to reduction scenarios, or to any scenario that requires the graphics processor to perform copy operations, write operations (optionally, also including arithmetic operations), and the embodiments disclosed herein do not limit this.

[0068] Figure 7a shows a schematic diagram of the structure of a memory access system according to an embodiment of the present disclosure.

[0069] As shown in Figure 7a, in one possible implementation, this disclosure proposes a memory access system, which is located in a first graphics processor. The system includes a copy engine and a cache with computational capabilities. Both the copy engine and the cache are located outside the core of the first graphics processor.

[0070] The first graphics processor executes a first instruction from the central processing unit. The first instruction includes a first address and a second address. The first address serves as the source operand address in the first instruction, and the second address serves as both the source operand address and the destination operand address in the first instruction.

[0071] When the execution conditions of the first command are met, the copy engine responds to the first command by copying the first operand stored at the first address and transferring the first operand to the cache in the graphics processor corresponding to the second address;

[0072] The cache is used to perform operations on the first operand and the second operand stored at the second address when the first operand is received, and write the result of the operation to the second address.

[0073] For example, a memory access system may include a copy engine and a cache with computational capabilities. The copy engine may have Direct Memory Access (DMA) functionality, allowing it to access the cache on the graphics processing unit (GPU) and the GPU's corresponding main memory or video memory. The cache can be any cache supporting various types of mathematical operations, such as the Last-Level Cache (LLC). The copy engine and cache can be located outside the GPU core.

[0074] In one possible implementation, the cache supports any one of the following operation types: addition, subtraction, multiplication, division, finding the maximum value, and finding the minimum value. Those skilled in the art will understand that the operation types can also include more types, such as logical AND and logical OR, etc. This disclosure does not limit the types of operations that the cache can support.

[0075] Furthermore, a single graphics processor may include at least one copy engine and at least one cache with computing capabilities. This disclosure does not limit the specific number of copy engines and caches with computing capabilities.

[0076] The first graphics processor can execute a first instruction from the central processing unit. This first instruction may include a first address and a second address. The first address serves as the source operand address in the first instruction, and the second address serves as both the source and destination operand address. Because two addresses serve as source operand addresses, the first instruction is an instruction that includes an arithmetic operation. For example, if the operation type of the first instruction is addition, the first address is the address of data B, and the second address is the address of data A, an example of the first instruction could be B += A.

[0077] When the execution conditions of the first command are met, the copy engine can respond to the first command and copy the first operand stored at the first address, such as data B. The copy engine can then transfer the first operand to the cache of the graphics processor corresponding to the second address. In the example of Figure 7a, assuming the second address is in the cache of the first graphics processor, or in the memory or video memory connected to the first graphics processor, the second address corresponds to the first graphics processor, and therefore the first operand can be transferred to the cache with computational capabilities on the first graphics processor.

[0078] If the second address is in the cache of the second graphics processor, or in the memory or video memory connected to the second graphics processor, then the second address corresponds to the second graphics processor, and therefore the first operand can be transferred to the cache with computing function on the second graphics processor.

[0079] Upon receiving the first operand, the cache with arithmetic capabilities can begin retrieving the second operand, such as A, stored at the second address. If the second address is the address of the cache with arithmetic capabilities, there is no need to copy the second operand; if the second address is not the address of the cache with arithmetic capabilities, the second operand can be copied from the second address to the cache. Then, operations can be performed on the first and second operands, such as B += A, and the result, such as A + B, is written to the second address. After the result is written, the first command is executed. Details regarding the cache's operations on the first and second operands, and the details of writing the result to the second address, can be found in the following descriptions.

[0080] According to an embodiment of the memory access system disclosed herein, the system is disposed in a first graphics processor (GPU). The system includes a copy engine and a cache with computational capabilities. The first GPU executes a first command from the central processing unit (CPU). When the execution conditions of the first command are met, the copy engine responds to the first command by copying a first operand stored at a first address, allowing the copy engine to replace the GPU core in copying the first operand. The copy engine then transmits the first operand to the cache in the GPU corresponding to a second address. Upon receiving the first operand, the cache performs computation on the first operand and a second operand stored at a second address, and writes the computation result to the second address. This allows the cache to replace the GPU core in performing computation on the first and second operands and writing the computation result. Both the copy engine and the cache are disposed outside the GPU core. In this case, the memory access system can replace the GPU core in performing copy, computation, and write operations, thereby freeing up the GPU core's computational resources. Furthermore, the paths for copy and write operations performed by the memory access system are shorter, which can improve command execution efficiency and reduce power consumption.

[0081] In this embodiment, the memory access system only needs to execute one command to complete the three operations of copying, computation, and writing. However, for a graphics processing unit (GPU) core, typically one command is used to complete only one operation, making it difficult to perform multiple operations with a single command. Therefore, this embodiment also simplifies the command execution.

[0082] A copy engine with DMA functionality and a cache with computational capabilities can be implemented based on existing technologies, and their structures will not be elaborated here.

[0083] In one possible implementation, the first command is one of multiple commands generated by the central processing unit (CPU). These CPU-generated commands enable multiple graphics processing units (GPUs) to jointly perform a ring-shaped global reduction operation. The first GPU is one of these GPUs.

[0084] The execution conditions for the first command are: the first command does not depend on other commands, or the copy engine detects that the first graphics processor has received a message that the second command, which the first command depends on, has been executed. In this case, the first address included in the first command is the same as the second address included in the second command.

[0085] For example, a first command might depend on other commands, such as using the execution result of a dependent second command for computation. In this case, the first address of the first command and the second address of the second command are the same. A first command might also be depended on by other commands, such as a third command that depends on the first command and uses the execution result of the first command for computation. In this case, the first address of the third command and the second address of the first command are the same. A first command might also have no dependencies on other commands. If a first command does not depend on other commands, it can be executed directly. If a first command depends on a second command, it can be executed after the second command. The execution condition for the first command can be that it does not depend on other commands, or that the copy engine detects that the first graphics processor has received a message indicating that the second command, which the first command depends on, has completed execution.

[0086] Multiple graphics processors in this embodiment can communicate in a ring. In this case, the multiple graphics processors can implement the ring global reduction algorithm.

[0087] The following section uses the command for implementing the ring global reduction algorithm as an example to introduce the characteristics of commands generated by the central processing unit and the way the graphics processing unit executes commands.

[0088] Figures 7b-7f illustrate a method by which a central processing unit generates multiple commands and executes them by a graphics processor according to an embodiment of the present disclosure.

[0089] The central processing unit (CPU) first determines which data stored on the graphics processing units (GPUs) needs to be processed using the circular global reduction algorithm. Let's assume that data stored on GPUs 0 through 2 needs to be processed using this algorithm. Specifically, GPU 0 stores data A, GPU 1 stores data B, and GPU 2 stores data C. The circular global reduction algorithm ensures that all GPUs 0 through 2 store data A+B+C (data stored in the corresponding memory or video memory of each GPU).

[0090] In step 1, the data stored on each graphics processor can be divided into 3 parts according to the number of graphics processors N (N=3). As shown in Figure 7b, when graphics processors GPU0-GPU2 are not executing commands, the data A stored on graphics processor GPU0 is divided into 3 parts A1, A2, and A3; the data B stored on graphics processor GPU1 is divided into 3 parts B1, B2, and B3; and the data C stored on graphics processor GPU2 is divided into 3 parts C1, C2, and C3.

[0091] The central processing unit (CPU) can determine that the commands to be generated are divided into two types: operation type and transfer type. Operation type commands are generated and executed before transfer type commands. Executing operation type commands requires completing copy, operation, and write operations, while executing transfer type commands requires completing copy and write operations. The total number of operation type commands can be N(N-1) = 6. The total number of transfer type commands can be N(N-1) = 6.

[0092] The following describes the commands of operation types generated by the central processing unit. Each operation type command can belong to N-1=2 rounds of computation. In each round, each graphics processor executes one operation type command. The commands executed in each round are independent of each other and can be executed in parallel.

[0093] The first command of the first round of computation can be executed by the graphics processing unit (GPU0). This command copies the first third of data B1 from GPU1 and performs computation on the first third of data A1 stored in GPU0, and writes the result to the storage location of data A1. Therefore, the first address of the first command of the first round of computation can be the address of data B1, and the second address can be the address of data A1.

[0094] The second command in the first round of computation can be executed by the graphics processing unit (GPU1). This command copies the second / third copy of data C2 from GPU2 and performs computation on the second / third copy of data B2 stored within GPU1, and writes the result to the storage location of data B2. Therefore, the first address in the second command of the first round of computation can be the address of data C2, and the second address can be the address of data B2.

[0095] The third command in the first round of computation can be executed by the graphics processing unit (GPU2). This command copies the third / third copy of data A3 from GPU0 and performs computation on the third / third copy of data C3 stored within GPU2, and writes the result to the storage location of data C3. Therefore, the first address in the third command of the first round of computation can be the address of data A3, and the second address can be the address of data C3.

[0096] As shown in Figure 7c, after the execution of the three commands in the first round of computation, the three sets of data stored on GPU0 are A1+B1, A2, and A3, the three sets of data stored on GPU1 are B1, B2+C2, and B3, and the three sets of data stored on GPU2 are C1, C2, and C3+A3. None of the three commands in the first round of computation depended on any other commands.

[0097] The first command of the second round of computation can be executed by the graphics processing unit (GPU0). This command copies the second / third copy of data B2+C2 from GPU1 and performs a computation on the third / third copy of data A2 stored within GPU0, then writes the result to the storage location of data A2. Therefore, the first address of the first command of the second round of computation can be the address of data B2+C2, and the second address can be the address of data A2. Since data B2+C2 is the result of the second command of the first round of computation, the first command of the second round of computation depends on the second command of the first round of computation.

[0098] The second instruction in the second round of computation can be executed by the graphics processing unit (GPU1). This instruction copies the third / third copy of data C3+A3 from GPU2 and performs computation on the third / third copy of data B3 stored within GPU1, then writes the result to the storage location of data B3. Therefore, the first address of the second instruction in the second round of computation can be the address of data C3+A3, and the second address can be the address of data B3. Since data C3+A3 is the result of the third instruction in the first round of computation, the second instruction in the second round of computation depends on the third instruction in the first round of computation.

[0099] The third command in the second round of computation can be executed by the graphics processing unit (GPU2). This command copies the first third of the data A1+B1 from GPU0 and performs a computation on the first third of the data C1 stored within GPU2, then writes the result to the storage location of data C1. Therefore, the first address of the third command in the second round of computation can be the address of data A1+B1, and the second address can be the address of data C1. Since data A1+B1 is the result of the first command in the first round of computation, the third command in the second round of computation depends on the first command in the first round of computation.

[0100] As shown in Figure 7d, after the execution of the three commands in the second round of computation, the three sets of data stored on GPU0 are A1+B1, A2+B2+C2, and A3, the three sets of data stored on GPU1 are B1, B2+C2, and B3+C3+A3, and the three sets of data stored on GPU2 are C1+A1+B1, C2, and C3+A3.

[0101] The following describes the move-type commands generated by the CPU. Each move-type command can belong to N-1=2 rounds of move operations. In each round, each GPU executes one command. The commands executed in each round are independent of each other and can be executed in parallel.

[0102] The first command of the first round of data transfer can be executed by GPU0. This command writes the second / third copy of data A2+B2+C2 stored in GPU0 to the storage location of the second / third copy of data in GPU1 (i.e., the storage location of data B2+C2). Therefore, the first address of the first command of the first round of data transfer can be the address of data A2+B2+C2 in GPU0, and the second address can be the address of data B2+C2. Since data A2+B2+C2 in GPU0 is the result of the execution of the first command of the second round of computation, the first command of the first round of data transfer depends on the first command of the second round of computation.

[0103] The second command in the first round of data transfer can be executed by GPU1. This command writes the third / third copy of data B3+C3+A3 stored in GPU1 to the storage location of the third / third copy of data in GPU2 (i.e., the storage location of data C3+A3). Therefore, the first address of the second command in the first round of data transfer can be the address of data B3+C3+A3 in GPU1, and the second address can be the address of data C3+A3. Since data B3+C3+A3 in GPU1 is the result of the second command in the second round of computation, the second command in the first round of data transfer depends on the second command in the second round of computation.

[0104] The third command in the first round of data transfer can be executed by GPU2. This command writes the first third copy of data C1+A1+B1 stored in GPU2 to the storage location of the first third copy of data in GPU0 (i.e., the storage location of data A1+B1). Therefore, the first address of the third command in the first round of data transfer can be the address of data C1+A1+B1 in GPU2, and the second address can be the address of data A1+B1. Since data C1+A1+B1 in GPU2 is the result of the third command in the second round of computation, the third command in the first round of data transfer depends on the third command in the second round of computation.

[0105] As shown in Figure 7e, after the execution of the three commands in the first round of data transfer, the three sets of data stored on GPU0 are C1+A1+B1, A2+B2+C2, and A3, the three sets of data stored on GPU1 are B1, A2+B2+C2, and B3+C3+A3, and the three sets of data stored on GPU2 are C1+A1+B1, C2, and B3+C3+A3.

[0106] The first command of the second round of data transfer can be executed by GPU0. This command writes the first third of the data C1+A1+B1 stored in GPU0 to the storage location of the first third of the data in GPU1 (i.e., the storage location of data B1). Therefore, the first address of the first command of the second round of data transfer can be the address of data C1+A1+B1 in GPU0, and the second address can be the address of data B1. Since data C1+A1+B1 in GPU0 is the result of the execution of the third command of the first round of data transfer, the first command of the second round of data transfer depends on the third command of the first round of data transfer.

[0107] The second command in the second round of data transfer can be executed by GPU1. This command writes the second / third copy of data A2+B2+C2 stored in GPU1 to the storage location of the second / third copy of data in GPU2 (i.e., the storage location of data C2). Therefore, the first address in the second command of the second round of data transfer can be the address of data A2+B2+C2 in GPU1, and the second address can be the address of data C2. Since data A2+B2+C2 in GPU1 is the result of the execution of the first command in the first round of data transfer, the second command of the second round of data transfer depends on the first command of the first round of data transfer.

[0108] The third command in the second round of data transfer can be executed by GPU2. This command writes the third / third copy of data B3+C3+A3 stored in GPU2 to the storage location of the third / third copy of data in GPU0 (i.e., the storage location of data A3). Therefore, the first address in the third command of the second round of data transfer can be the address of data B3+C3+A3 in GPU2, and the second address can be the address of data A3. Since data B3+C3+A3 in GPU2 is the result of the execution of the second command in the first round of data transfer, the third command of the second round of data transfer depends on the second command in the first round of data transfer.

[0109] As shown in Figure 7f, after the execution of the three commands in the second round of data transfer, the three sets of data stored on GPU0, GPU1, and GPU2 are all C1+A1+B1, A2+B2+C2, and B3+C3+A3. Each GPU then performs calculations on its own three sets of data to obtain A+B+C.

[0110] The central processing unit (CPU) can mark dependencies for the generated commands and transmit all generated commands to the respective graphics processing units (GPUs). Each GPU can then execute a command that meets the execution conditions to achieve the corresponding effect.

[0111] In the examples of Figures 7b-7f, each graphics processor can be used as a first graphics processor. When an operation-type command is used as the first command, the included second address corresponds to the first graphics processor, and the included first address corresponds to the next graphics processor after the first graphics processor. When a move-type command is used as the first command, the included first address corresponds to the first graphics processor, and the included second address corresponds to the next graphics processor after the first graphics processor. Those skilled in the art should understand that in practical applications, there is no restriction on which graphics processor the second and first addresses of the first command specifically correspond to, as long as the ring-wide reduction operation can be implemented.

[0112] In one possible implementation, multiple graphics processors (GPUs) each store multiple data sets to be processed, and the result of the ring-shaped global reduction operation performed by the multiple GPUs is the result of the operation on the multiple data sets to be processed.

[0113] When the first command does not depend on other commands, the data stored at the first address of the first command is a part of the data to be processed stored on the graphics processor corresponding to the first address, and the data stored at the second address of the first command is a part of the data to be processed stored on the graphics processor corresponding to the second address.

[0114] When the first command depends on other commands, the data stored at the first address of the first command is the result of a partial operation on the data to be operated on, and the data stored at the second address of the first command is a part of the data to be operated on stored on the graphics processor corresponding to the second address, or the result of a partial operation on the data to be operated on.

[0115] For example, in a ring-shaped global reduction operation scenario, multiple graphics processors store multiple data to be operated on, such as data A, B, and C in the relevant description of Figure 7b. At this time, the result of the operation on the multiple data to be operated on can be A+B+C.

[0116] If the first command does not depend on other commands, as seen in the three commands in the first round of computation described above, it can be seen that the computation object of each command is a part of the data stored on the graphics processor. That is, when the first command does not depend on other commands, the data stored at the first address included in the first command is a part of the data to be computed stored on the graphics processor corresponding to the first address, and the data stored at the second address included in the first command is a part of the data to be computed stored on the graphics processor corresponding to the second address.

[0117] If the first command depends on other commands, such as the three commands in the second round of calculation, the three commands in the first round of transfer, and the three commands in the second round of transfer mentioned above, it can be seen that the data stored at the first address must be the execution result of other commands, that is, the result of a partial calculation of the data to be calculated. The data stored at the second address may be the execution result of other commands, or it may be a part of the data to be calculated stored on the graphics processor corresponding to the second address. In other words, when the first command depends on other commands, the data stored at the first address included in the first command is the result of a partial calculation of the data to be calculated, and the data stored at the second address included in the first command is a part of the data to be calculated stored on the graphics processor corresponding to the second address, or the result of a partial calculation of the data to be calculated.

[0118] In one possible implementation, the cache is also used to transmit a message indicating that the computation result has been written to the replication engine;

[0119] The copy engine is also used to transmit a message indicating that the first command has been executed to the graphics processor that executes the third command when it receives a message indicating that the operation result has been written. The third command depends on the first command, and the first address included in the third command is the same as the second address included in the first command.

[0120] For example, when the calculation result is written, it means that the first command has been executed. At this time, the third command, which depends on the first command, can begin execution. Therefore, the cache can transmit the message that the calculation result has been written to the copy engine to notify the copy engine that the first command has been executed; the copy engine can transmit the message that the first command has been executed to the graphics processor executing the third command to notify the graphics processor executing the third command that it can begin executing the third command.

[0121] In this way, the execution of commands can be synchronized across multiple graphics processors, and multiple commands can be executed automatically in a preset order. The memory access system does not need to control the execution order of commands, thus reducing the design complexity of the memory access system.

[0122] In one possible implementation, the cache is specifically used for:

[0123] Upon receiving the first operand, perform atomic operations on the first and second operands;

[0124] The result of the operation is written to the second address through an atomic write operation.

[0125] For example, in certain situations, the second address might be accessible to two or more graphics processors. During the execution of a single command, data consistency needs to be maintained, and atomic operations are uninterrupted; therefore, the cache can be set to perform atomic operations by default. In this case, the cache can perform atomic copy, atomic operation, or atomic write operation in response to the first operand, ensuring the accuracy of the calculation result.

[0126] In other words, when the cache receives the first operand, it performs operations on the first and second operands, which can be done atomically. The cache writes the result to the second address, which can be done atomically. When the cache needs to copy the first operand, it can do so atomically.

[0127] In one scenario, the copy engine on the first graphics processor can transmit the commands generated by the central processing unit and to be executed by the first graphics processor to the cache in sequence. The cache can record the receiving order of the received operands. When the i-th operand is received, it can be directly determined that the i-th command to be executed is the received command, and the second operand stored at the second address in the i-th command can be found.

[0128] In another scenario, when the copy engine responds to a command, it can also directly convert the command into an atomic operation type. For example, if the execution conditions of the first command are met, the copy engine can also convert the first command into a command of atomic operation type, and the converted command also includes the first address and the second address. The copy engine can then transfer the converted command and the first operand together to the cache of the graphics processor corresponding to the second address (in the example of Figure 7a, the first graphics processor). The first operand is used as a parameter when executing the converted command.

[0129] This cache with computational capabilities can, upon receiving the converted command and the first operand, retrieve the second operand based on the second address in the converted command, perform computation on the first and second operands, and write the result to the second address. Once the result is written, the converted command has been executed, and the first command can be considered to have also been executed.

[0130] For clarity, the following text uses the example of a replication engine only transmitting the first operand to the cache. However, those skilled in the art will understand that if the replication engine needs to transmit the transformed command, the transmission method of the transformed command can be the same as the transmission method of the first operand.

[0131] In one possible implementation, the first address and the second address correspond to different graphics processors.

[0132] For example, the graphics processors corresponding to the first address and the second address can be different; for instance, the first address and the second address can correspond to the first graphics processor and the second graphics processor, respectively. When the first address and the second address correspond to different graphics processors, the methods for accessing the first address and the second address are also different. Figures 8 and 9 respectively illustrate examples of accessing the first address and the second address according to embodiments of this disclosure.

[0133] In one possible implementation, the first graphics processor further includes an on-chip interconnect bus connecting the cache and copy engine, and the first graphics processor is connected to the second graphics processor via an inter-chip interconnect bus.

[0134] When the first address corresponds to the second graphics processor and the second address corresponds to the first graphics processor, the copy engine reads the first operand through the on-chip interconnect bus and the inter-chip interconnect bus, and transmits the first operand through the on-chip interconnect bus; or

[0135] When the first address corresponds to the first graphics processor and the second address corresponds to the second graphics processor, the copy engine reads the first operand through the on-chip interconnect bus and transmits the first operand through the on-chip interconnect bus and the inter-chip interconnect bus.

[0136] For example, as shown in Figures 8 and 9, a first graphics processor may include a copy engine, a cache with computing capabilities, a graphics processor core, and an on-chip interconnect bus connecting the cache and the copy engine. A second graphics processor also includes a copy engine, a cache with computing capabilities, a graphics processor core, and an on-chip interconnect bus connecting the cache and the copy engine. The on-chip interconnect bus also connects to the graphics processor core. The first and second graphics processors are connected via an inter-chip interconnect bus.

[0137] In the example of Figure 8, the first address corresponds to the second graphics processor and is used to store data B. Since the first operand is read from the first address, the path for the copy engine to copy the first operand can include the on-chip interconnect bus, inter-chip interconnect bus, on-chip interconnect bus of the first graphics processor, cache of the second graphics processor, or optional memory or video memory of the second graphics processor. If the first address is an address on the cache of the second graphics processor, then the path may not include the memory or video memory of the second graphics processor. If the first address is an address on the memory or video memory of the second graphics processor, then the path may include the memory or video memory of the second graphics processor.

[0138] The second address corresponds to the first graphics processor and is used to store data A. The copy engine transfers the first operand to the cache in the graphics processor corresponding to the second address; therefore, the path for transferring the first operand may include the on-chip interconnect bus and the cache on the first graphics processor.

[0139] If the second address is an address in the cache of the first graphics processor, then the cache of the first graphics processor does not need to access memory for the calculation result (A+B). If the second address is an address in the main memory or video memory of the first graphics processor, then the cache of the first graphics processor can write the calculation result to the main memory or video memory of the first graphics processor.

[0140] In the example of Figure 9, the first address corresponds to the first graphics processor and is used to store the first operand (data B). Since the first operand is read from the first address, the path for the copy engine to copy the first operand can include the on-chip interconnect bus, the cache on the first graphics processor, the memory of the first graphics processor, or video memory (optionally). If the first address is an address on the cache of the first graphics processor, then the path may not include the memory or video memory of the first graphics processor. If the first address is an address on the memory or video memory of the first graphics processor, then the path may include the memory or video memory of the first graphics processor.

[0141] The second address corresponds to the second graphics processor and is used to store the second operand (data A). The copy engine transfers the first operand to the cache in the graphics processor corresponding to the second address. Therefore, the path for transferring the first operand may include the on-chip interconnect bus, the inter-chip interconnect bus on the first graphics processor, the on-chip interconnect bus on the second graphics processor, the cache on the second graphics processor, the memory or video memory of the second graphics processor (optional).

[0142] If the second address is an address in the cache of the second graphics processor, then the cache of the second graphics processor does not need to access memory for the calculation result (A+B). If the second address is an address in the main memory or video memory of the second graphics processor, then the cache of the second graphics processor can write the calculation result to the main memory or video memory of the second graphics processor.

[0143] This approach makes the memory access method of the replication engine more flexible.

[0144] In one possible implementation, the first graphics processor executes a fourth instruction from the central processing unit, the fourth instruction including a third address and a fourth address, wherein the third address serves as the source operand address and the fourth address serves as the destination operand address in the fourth instruction;

[0145] When the execution conditions for the fourth command are met, the copy engine is also used to execute the fourth command to read the third operand stored at the third address and transfer the third operand to the cache in the graphics processor corresponding to the fourth address;

[0146] The cache is also used to write the third operand to the fourth address when the third operand is received.

[0147] For example, the first graphics processor can also be used to execute a fourth command that does not involve arithmetic operations (i.e., a move-type command, examples of which can be found in the relevant descriptions of Figures 7e-7f). In this case, the fourth command may include a third address and a fourth address, where the third address serves as the source operand address and the fourth address serves as the destination operand address.

[0148] When the execution conditions of the fourth command are met, the copy engine can respond to the fourth command to copy the third operand stored at the third address. The third operand is then transferred to the cache in the graphics processor corresponding to the fourth address.

[0149] For example, when the fourth address is in the cache of the first graphics processor, or in the memory or video memory connected to the first graphics processor, the fourth address corresponds to the first graphics processor, so the third operand can be transferred to the cache with computing function on the first graphics processor.

[0150] If the fourth address is in the cache of the second graphics processor, or in the memory or video memory connected to the second graphics processor, the fourth address corresponds to the second graphics processor, so the third operand can be transferred to the cache with computing capabilities on the second graphics processor.

[0151] This cache with computational capabilities is used to write the third operand to the fourth address via an atomic write operation when the third operand is received.

[0152] For example, the replication engine can also convert the fourth command into an atomic operation type command, which also includes the third and fourth addresses. The replication engine can then transfer the converted command and the third operand together to the cache. Upon receiving the converted command and the third operand, the cache writes the third operand to the fourth address based on the fourth address in the converted command. After the third operand is written, the converted command and the fourth command are executed.

[0153] In this way, the graphics processor can support the processing of more types of commands.

[0154] In one possible implementation, the fourth command is executed when it does not depend on any other command, or when the copy engine detects that the first graphics processor has received a message indicating that the sixth command, which the fourth command depends on, has completed execution.

[0155] Similar to the execution conditions of the first command, the fourth command may not depend on other commands, or it may depend on other commands. In this case, the execution condition of the fourth command could be that it does not depend on other commands, or that the copy engine detects that the first graphics processor has received a message indicating that the sixth command, which the fourth command depends on, has completed execution. This message can come from the graphics processor used to execute the sixth command. The first address included in the fourth command is the same as the second address included in the sixth command.

[0156] In one possible implementation, the cache is also used to transmit a message indicating that the third operand has been written to the replication engine;

[0157] The copy engine is also used to transmit a message indicating that the fourth command has been executed to the graphics processor that executes the eighth command when it receives a message indicating that the operation result has been written. The eighth command depends on the fourth command, and the first address included in the eighth command is the same as the second address included in the fourth command.

[0158] For example, when the third operand is written, it means that the fourth command has also been executed. At this point, the eighth command, which depends on the fourth command, can begin execution. Therefore, the cache can transmit the message that the third operand has been written to the copy engine to notify the copy engine that the fourth command has been executed; the copy engine can then transmit the message that the fourth command has been executed to the graphics processor used to execute the eighth command to notify the graphics processor that it can begin executing the eighth command.

[0159] In one possible implementation, the third and fourth addresses correspond to different graphics processors. When the third and fourth addresses correspond to different graphics processors, the methods of accessing the third and fourth addresses also differ. Examples can be found in the descriptions of Figures 8 and 9, showing how the first and second addresses are accessed when they correspond to different processors. Simply replace the first address with the third address and the second address with the fourth address; the methods of accessing the third and fourth addresses will not be elaborated further here.

[0160] This disclosure also provides a memory access method. FIG10 shows a schematic diagram of the flow of the memory access method according to an embodiment of this disclosure.

[0161] As shown in Figure 10, in one possible implementation, the method is applied to a memory access system, which is located in a first graphics processor. The system includes a copy engine and a cache with computational capabilities. Both the copy engine and the cache are located outside the core of the first graphics processor. The method includes:

[0162] The first graphics processor executes a first instruction from the central processing unit. The first instruction includes a first address and a second address. The first address serves as the source operand address in the first instruction, and the second address serves as both the source operand address and the destination operand address in the first instruction.

[0163] Step S11: When the execution conditions of the first command are met, the copy engine responds to the first command by copying the first operand stored at the first address and transferring the first operand to the cache in the graphics processor corresponding to the second address;

[0164] Step S12: When the cache receives the first operand, it performs a calculation on the first operand and the second operand stored at the second address, and writes the calculation result to the second address.

[0165] In one possible implementation, the first command is one of a plurality of commands generated by the central processing unit (CPU). The plurality of commands generated by the CPU are used to enable a plurality of graphics processors (GPUs) to jointly perform a ring-shaped global reduction operation. The first GPU is one of the plurality of GPUs. The execution condition of the first command is that the first command does not depend on other commands, or the copy engine detects that the first GPU has received a message indicating that the execution of the second command, which the first command depends on, has been completed. In this case, the first address included in the first command is the same as the second address included in the second command.

[0166] In one possible implementation, the method further includes:

[0167] The cache transmits a message indicating that the calculation result has been written to the replication engine;

[0168] When the copy engine receives a message indicating that the calculation result has been written, it transmits a message indicating that the first command has been executed to the graphics processor for executing the third command. The third command depends on the first command, and the first address included in the third command is the same as the second address included in the first command.

[0169] In one possible implementation, when the cache receives the first operand, it performs a calculation on the first operand and the second operand stored at the second address, and writes the calculation result to the second address, including:

[0170] When the cache receives the first operand, it performs atomic operations on the first operand and the second operand; and writes the result of the operation to the second address through an atomic write operation.

[0171] In one possible implementation, the first address and the second address correspond to different graphics processors.

[0172] In one possible implementation, the first graphics processor further includes an on-chip interconnect bus connecting the cache and the copy engine, and the first graphics processor is connected to the second graphics processor via an inter-chip interconnect bus.

[0173] When the first address corresponds to the second graphics processor and the second address corresponds to the first graphics processor, the copy engine reads the first operand through the intra-chip interconnect bus and the inter-chip interconnect bus, and transmits the first operand through the intra-chip interconnect bus; or

[0174] When the first address corresponds to the first graphics processor and the second address corresponds to the second graphics processor, the copy engine reads the first operand through the on-chip interconnect bus and transmits the first operand through the on-chip interconnect bus and the inter-chip interconnect bus.

[0175] In one possible implementation, the method further includes:

[0176] The first graphics processor executes a fourth instruction from the central processing unit, the fourth instruction including a third address and a fourth address, wherein the third address serves as the source operand address and the fourth address serves as the destination operand address in the fourth instruction;

[0177] When the execution conditions of the fourth command are met, the copy engine responds to the fourth command by copying the third operand stored at the third address and transferring the third operand to the cache in the graphics processor corresponding to the fourth address;

[0178] When the cache receives the third operand, it writes the third operand to the fourth address.

[0179] In one possible implementation, the cache supports any one of the following operation types: addition, subtraction, multiplication, division, finding the maximum value, and finding the minimum value.

[0180] In one possible implementation, the plurality of graphics processors each store a plurality of data to be processed, and the result of the ring global reduction operation performed jointly by the plurality of graphics processors is the result of the operation on the plurality of data to be processed.

[0181] When the first command does not depend on other commands, the data stored at the first address included in the first command is a part of the data to be processed stored on the graphics processor corresponding to the first address, and the data stored at the second address included in the first command is a part of the data to be processed stored on the graphics processor corresponding to the second address;

[0182] When the first command depends on other commands, the data stored at the first address included in the first command is the result of a partial operation on the data to be operated on, and the data stored at the second address included in the first command is a part of the data to be operated on stored on the graphics processor corresponding to the second address, or the result of a partial operation on the data to be operated on.

[0183] This disclosure also proposes a first graphics processor, including the memory access system described above. A schematic diagram of the structure of the first graphics processor can be seen in Figures 8-9.

[0184] This disclosure also proposes an electronic device including the first graphics processor described above. The electronic device may be a server or a terminal device; the embodiments of this disclosure do not limit the specific type of electronic device.

[0185] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0186] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technological improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0187] A,A1,A2,A3,B,B1,B2,B3,C,C1,C2,C3: Data GPU0, GPU1, GPU2, GPU3: Graphics Processing Units S11, S12: Steps

Claims

1. A memory access system, characterized in that the system is disposed in a first graphics processor, the system comprising a copy engine and a cache with computing functions, wherein the copy engine and the cache are both disposed outside the core of the first graphics processor, wherein, The first graphics processor executes a first command from the central processing unit, the first command including a first address and a second address, the first address serving as the source operand address in the first command, and the second address serving as both the source operand address and the destination operand address in the first command; when the execution conditions of the first command are met, the copy engine is used to respond to the first command by copying the first operand stored at the first address and transferring the first operand to the cache in the graphics processor corresponding to the second address; The cache is used to perform a calculation on the first operand and the second operand stored at the second address when the first operand is received, and write the calculation result to the second address.

2. The system as described in claim 1, characterized in that the first command is one of a plurality of commands generated by the central processing unit, the plurality of commands generated by the central processing unit being used to enable a plurality of graphics processors to jointly perform a ring-shaped global reduction operation, the first graphics processor being one of the plurality of graphics processors, and the execution condition of the first command being that the first command does not depend on other commands, or the copy engine detecting that the first graphics processor has received a message indicating that the execution of a second command, on which the first command depends, has been completed, wherein, The first address included in the first command is the same as the second address included in the second command.

3. The system as described in claim 1 or 2, characterized in that the cache is further configured to transmit a message indicating that the operation result has been written to the copy engine; the copy engine is further configured to, upon receiving the message indicating that the operation result has been written, transmit a message indicating that the first command has been executed to a graphics processor for executing a third command, the third command depending on the first command, wherein the first address included in the third command is the same as the second address included in the first command.

4. The system as described in claim 1, wherein the cache is specifically used for: upon receiving the first operand, performing atomic operations on the first operand and the second operand; and writing the operation result to the second address via an atomic write operation.

5. The system as claimed in claim 1, wherein the graphics processors corresponding to the first address and the second address are different.

6. The system as claimed in claim 5, characterized in that the first graphics processor further includes an on-chip interconnect bus connecting the cache and the copy engine, the first graphics processor is connected to the second graphics processor via an inter-chip interconnect bus, and when the first address corresponds to the second graphics processor and the second address corresponds to the first graphics processor, the copy engine reads the first operand via the on-chip interconnect bus and the inter-chip interconnect bus, and transmits the first operand via the on-chip interconnect bus; or when the first address corresponds to the first graphics processor and the second address corresponds to the second graphics processor, the copy engine reads the first operand via the on-chip interconnect bus, and transmits the first operand via the on-chip interconnect bus and the inter-chip interconnect bus.

7. The system of claim 1, characterized in that the first graphics processor executes a fourth command from the central processing unit, the fourth command including a third address and a fourth address, the third address serving as a source operand address and the fourth address serving as a destination operand address in the fourth command; when the execution conditions of the fourth command are met, the copy engine is further configured to respond to the fourth command by copying a third operand stored at the third address and transferring the third operand to a cache in the graphics processor corresponding to the fourth address; the cache is further configured to write the third operand to the fourth address upon receiving the third operand.

8. The system as described in claim 1, wherein the cache supports any one of the following operation types: addition, subtraction, multiplication, division, finding the maximum value, and finding the minimum value.

9. The system as described in claim 2, characterized in that the plurality of graphics processors respectively store a plurality of data to be processed, and the result of the ring global reduction operation jointly performed by the plurality of graphics processors is the result of the operation on the plurality of data to be processed; when the first command does not depend on other commands, the data stored at the first address included in the first command is a part of the data to be processed stored on the graphics processor corresponding to the first address, and the data stored at the second address included in the first command is a part of the data to be processed stored on the graphics processor corresponding to the second address; when the first command depends on other commands, the data stored at the first address included in the first command is the result of the operation on a portion of the data to be processed, and the data stored at the second address included in the first command is a part of the data to be processed stored on the graphics processor corresponding to the second address, or the result of the operation on a portion of the data to be processed.

10. A memory access method, characterized in that it is applied to a memory access system, the system being disposed in a first graphics processor, the system including a copy engine and a cache with computing functions, wherein the copy engine and the cache are both disposed outside the core of the first graphics processor, the method comprising: The first graphics processor executes a first command from the central processing unit. The first command includes a first address and a second address. The first address serves as the source operand address in the first command, and the second address serves as both the source operand address and the destination operand address in the first command. When the execution conditions of the first command are met, the copy engine responds to the first command by copying the first operand stored at the first address and transferring the first operand to the cache in the graphics processor corresponding to the second address. When the cache receives the first operand, it performs a calculation on the first operand and the second operand stored at the second address, and writes the calculation result to the second address.

11. A first graphics processor, characterized in that it includes a memory access system as described in any one of claims 1-9.

12. An electronic device, characterized in that it includes the first graphics processor as described in claim 11.

Images