Memory access system and method, first graphics processing unit and electronic device

Abstract

The present disclosure relates to the field of chips, and provides a memory access system and method, a first graphics processing unit and an electronic device. The system comprises a replication engine and a cache having a computing function. The first graphics processing unit executes a first command from a central processing unit. When an execution condition of the first command is met, the replication engine copies a first operand in response to the first command and transmits the first operand to a cache in a graphics processing unit corresponding to a second address; and the cache computes the first operand and a second operand and writes a computing result into the second address. According to embodiments of the present disclosure, the memory access system is arranged outside a core of the first graphics processing unit, and the memory access system has computing capability and can replace a graphics processing unit core to complete simple mathematical operation, so as to release computing resources of the graphics processing unit core; and a path of migrating data to the memory access system is relatively short, so that the command execution efficiency can be improved, and power consumption is reduced.

Description

Memory access system, method, first graphics processor, and electronic device

[0001] This application claims priority to Chinese patent application No. 202410389700.8, filed on April 1, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of chips, and in particular to a memory access system, method, first graphics processor, and electronic device. Background Art

[0003] The development of large language models (LLMs) has made natural language processing (NLP) a hot topic in the field of general artificial intelligence. Due to their large size, large language models are often trained using a distributed approach, using multiple graphics processing units (GPUs) to jointly complete their training.

[0004] The global reduce algorithm is a key technology for distributed training and can be used for model parallelism and data parallelism. The operations involved in this algorithm mainly include data movement and simple mathematical operations such as addition and subtraction. When executing the global reduce algorithm, the graphics processor of the prior art uses the graphics processor core to perform data movement and calculation operations. However, the pipeline of the graphics processor core is very complex and has powerful computing power. When using the graphics processor core to perform these operations, its computing power can only be exerted to a very small extent, resulting in a waste of computing resources. In addition, the path for moving data to the graphics processor core is relatively long, which reduces the efficiency of executing the global reduce algorithm and increases power consumption. Summary of the Invention

[0005] In view of this, the present disclosure provides a memory access system, method, first graphics processor, and electronic device. The memory access system of the disclosed embodiment is disposed within the first graphics processor, but outside the core of the first graphics processor. The memory access system has computing power and can replace the graphics processor core to perform simple mathematical operations, thereby freeing up the computing resources of the graphics processor core. Furthermore, the path for moving data to the memory access system is shorter, which can improve command execution efficiency and reduce power consumption.

[0006] According to one aspect of the present disclosure, a memory access system is provided, which is arranged in a first graphics processor, and includes a copy engine and a cache with a computing function. The copy engine and the cache are both arranged outside the core of the first graphics processor, wherein the first graphics processor executes a first command from a central processing unit, the first command including a first address and a second address, the first address serving as a source operand address in the first command, and the second address serving as both a source operand address and a destination operand address in the first command; when an execution condition of the first command is met, the copy engine is configured to respond to the first command by copying a first operand stored at the first address and transferring the first operand to a cache in the graphics processor corresponding to the second address; and the cache is configured to, upon receiving the first operand, perform a calculation on the first operand and a second operand stored at the second address, and write the calculation result to the second address.

[0007] In one possible implementation, the first command is one of multiple commands generated by the central processing unit, and the multiple commands generated by the central processing unit are used to enable multiple graphics processors to jointly implement a ring global reduction operation. The first graphics processor is one of the multiple graphics processors. The execution condition of the first command is that the first command does not depend on other commands, or the replication engine detects that the first graphics processor has received information that a second command, on which the first command depends, has completed execution. The first address included in the first command is the same as the second address included in the second command.

[0008] In one possible implementation, the cache is further configured to transmit information indicating completion of writing the calculation result to the copy engine; upon receiving the information indicating completion of writing the calculation result, the copy engine is further configured to transmit information indicating completion of execution of the first command to a graphics processor configured to execute a third command, wherein the third command is dependent on the first command, and a first address included in the third command is the same as a second address included in the first command.

[0009] In one possible implementation, the cache is specifically configured to: upon receiving the first operand, perform an operation on the first operand and the second operand in an atomic operation; and write the operation result to the second address through an atomic write operation. In one possible implementation, the first address and the second address correspond to different graphics processors.

[0010] In one possible implementation, the first graphics processor further includes an intra-chip interconnect bus connecting the cache and the copy engine. The first graphics processor is connected to a second graphics processor via an inter-chip interconnect bus. When the first address corresponds to the second graphics processor and the second address corresponds to the first graphics processor, the copy engine reads the first operand via the intra-chip interconnect bus and the inter-chip interconnect bus and transmits the first operand via the intra-chip interconnect bus; or when the first address corresponds to the first graphics processor and the second address corresponds to the second graphics processor, the copy engine reads the first operand via the intra-chip interconnect bus and transmits the first operand via the intra-chip interconnect bus and the inter-chip interconnect bus.

[0011] In one possible implementation, the first graphics processor executes a fourth command from the central processing unit, where the fourth command includes a third address and a fourth address, the third address serving as a source operand address in the fourth command, and the fourth address serving as a destination operand address in the fourth command; when an execution condition of the fourth command is met, the copy engine is further configured to respond to the fourth command to copy a third operand stored at the third address and transfer the third operand to a cache in the graphics processor corresponding to the fourth address; and the cache is further configured to write the third operand to the fourth address upon receiving the third operand.

[0012] In a possible implementation, the operation type supported by the cache includes any one of addition, subtraction, multiplication, division, maximum value, and minimum value.

[0013] In one possible implementation, the multiple graphics processors respectively store multiple data to be calculated, and the calculation result of the ring global reduction operation implemented by the multiple graphics processors together is the calculation result of the multiple data to be calculated. When the first command does not depend on other commands, the data stored at the first address included in the first command is a part of the data to be calculated stored on the graphics processor corresponding to the first address, and the data stored at the second address included in the first command is a part of the data to be calculated stored on the graphics processor corresponding to the second address; when the first command depends on other commands, the data stored at the first address included in the first command is the calculation result of part of the data to be calculated, and the data stored at the second address included in the first command is a part of the data to be calculated stored on the graphics processor corresponding to the second address, or the calculation result of part of the data to be calculated.

[0014] According to another aspect of the present disclosure, a memory access method is provided, which is applied to a memory access system, wherein the system is arranged in a first graphics processor, the system includes a copy engine and a cache with a computing function, and the copy engine and the cache are both arranged outside the core of the first graphics processor. The method includes: the first graphics processor executes a first command from a central processing unit, the first command includes a first address and a second address, the first address serves as a source operand address in the first command, and the second address serves as both a source operand address and a destination operand address in the first command; when an execution condition of the first command is met, the copy engine responds to the first command to copy a first operand stored at the first address, and transfers the first operand to a cache in the graphics processor corresponding to the second address; when the cache receives the first operand, it operates on the first operand and the second operand stored at the second address, and writes the operation result to the second address.

[0015] In one possible implementation, the first command is one of multiple commands generated by the central processing unit, and the multiple commands generated by the central processing unit are used to enable multiple graphics processors to jointly implement a ring global reduction operation. The first graphics processor is one of the multiple graphics processors. The execution condition of the first command is that the first command does not depend on other commands, or the replication engine detects that the first graphics processor has received information that a second command, on which the first command depends, has completed execution. The first address included in the first command is the same as the second address included in the second command.

[0016] In one possible implementation, the method further includes: the cache transmitting information indicating completion of writing the calculation result to the copy engine; upon receiving the information indicating completion of writing the calculation result, the copy engine transmitting information indicating completion of executing the first command to a graphics processor for executing a third command, wherein the third command is dependent on the first command, and a first address included in the third command is the same as a second address included in the first command.

[0017] In one possible implementation, when the cache receives the first operand, it operates on the first operand and the second operand stored at the second address, and writes the operation result to the second address, including: when the cache receives the first operand, it operates on the first operand and the second operand in an atomic operation manner; and writes the operation result to the second address through an atomic write operation.

[0018] In a possible implementation, the first address and the second address correspond to different graphics processors.

[0019] In one possible implementation, the first graphics processor further includes an intra-chip interconnect bus connecting the cache and the copy engine. The first graphics processor is connected to a second graphics processor via an inter-chip interconnect bus. When the first address corresponds to the second graphics processor and the second address corresponds to the first graphics processor, the copy engine reads the first operand via the intra-chip interconnect bus and the inter-chip interconnect bus and transmits the first operand via the intra-chip interconnect bus; or when the first address corresponds to the first graphics processor and the second address corresponds to the second graphics processor, the copy engine reads the first operand via the intra-chip interconnect bus and transmits the first operand via the intra-chip interconnect bus and the inter-chip interconnect bus.

[0020] In one possible implementation, the method further includes: the first graphics processor executing a fourth command from the central processing unit, the fourth command including a third address and a fourth address, the third address serving as a source operand address in the fourth command, and the fourth address serving as a destination operand address in the fourth command; when an execution condition of the fourth command is met, the copy engine responding to the fourth command to copy a third operand stored at the third address, and transferring the third operand to a cache in the graphics processor corresponding to the fourth address; and upon receiving the third operand, the cache writing the third operand to the fourth address.

[0021] In a possible implementation, the operation type supported by the cache includes any one of addition, subtraction, multiplication, division, maximum value, and minimum value.

[0022] In one possible implementation, the multiple graphics processors respectively store multiple data to be calculated, and the calculation result of the ring global reduction operation implemented by the multiple graphics processors together is the calculation result of the multiple data to be calculated. When the first command does not depend on other commands, the data stored at the first address included in the first command is a part of the data to be calculated stored on the graphics processor corresponding to the first address, and the data stored at the second address included in the first command is a part of the data to be calculated stored on the graphics processor corresponding to the second address; when the first command depends on other commands, the data stored at the first address included in the first command is the calculation result of part of the data to be calculated, and the data stored at the second address included in the first command is a part of the data to be calculated stored on the graphics processor corresponding to the second address, or the calculation result of part of the data to be calculated.

[0023] According to another aspect of the present disclosure, a first graphics processor is provided, comprising any one of the above memory access systems.

[0024] According to another aspect of the present disclosure, an electronic device is provided, including the first graphics processor described above.

[0025] According to an embodiment of the present disclosure, a memory access system is provided in a first graphics processor and includes a copy engine and a cache with computational capabilities. The first graphics processor executes a first command from a central processing unit (CPU). When the execution conditions of the first command are met, the copy engine responds to the first command by copying a first operand stored at a first address, allowing the copy engine to complete the copying of the first operand on behalf of the graphics processor core. The copy engine then transmits the first operand to a cache in the graphics processor corresponding to a second address. Upon receiving the first operand, the cache performs a computation on the first operand and a second operand stored at a second address, and writes the computation result to the second address, allowing the cache to complete the computation on the first operand and the second operand, as well as write the computation result, on behalf of the graphics processor core. The copy engine and the cache are both provided outside the first graphics processor core. In this case, the memory access system can complete the copy, computation, and write operations on behalf of the graphics processor core, thereby freeing up computing resources of the graphics processor core. Furthermore, the paths for the copy and write operations performed by the memory access system are relatively short, thereby improving command execution efficiency and reducing power consumption.

[0026] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0028] FIG1 is a schematic diagram showing a flow of a graphics processor in the prior art executing a ring global reduction algorithm.

[0029] FIG. 2 is a schematic diagram showing a flow of a graphics processor in the prior art executing a ring global reduction algorithm.

[0030] FIG. 3 is a schematic diagram showing a flow of a graphics processor in the prior art executing a ring global reduction algorithm.

[0031] FIG. 4 a is a schematic diagram showing a flow of a graphics processor in the prior art executing a ring global reduction algorithm.

[0032] FIG. 4 b is a schematic diagram showing a flow of a graphics processor in the prior art executing a ring global reduction algorithm.

[0033] FIG. 5 a is a schematic diagram showing a single graphics processor in the prior art processing commands of a global reduction algorithm.

[0034] FIG. 5 b is a schematic diagram showing a single graphics processor in the prior art processing commands of a global reduction algorithm.

[0035] FIG6 a shows an exemplary application scenario of a memory access system according to an embodiment of the present disclosure.

[0036] FIG6 b shows an exemplary application scenario of the memory access system according to an embodiment of the present disclosure.

[0037] FIG7 a shows a schematic diagram of the structure of a memory access system according to an embodiment of the present disclosure.

[0038] FIG7 b is a schematic diagram showing a method in which a central processing unit generates multiple commands and a graphics processor executes the commands according to an embodiment of the present disclosure.

[0039] FIG. 7 c is a schematic diagram illustrating a method in which a central processing unit generates multiple commands and a graphics processing unit executes the commands according to an embodiment of the present disclosure.

[0040] FIG7 d is a schematic diagram showing a method in which a central processing unit generates multiple commands and a graphics processing unit executes the commands according to an embodiment of the present disclosure.

[0041] FIG. 7 e is a schematic diagram illustrating a method in which a central processing unit generates multiple commands and a graphics processing unit executes the commands according to an embodiment of the present disclosure.

[0042] FIG. 7 f is a schematic diagram illustrating a method in which a central processing unit generates multiple commands and a graphics processing unit executes the commands according to an embodiment of the present disclosure.

[0043] FIG. 8 illustrates an example of accessing a first address and a second address according to an embodiment of the present disclosure.

[0044] FIG. 9 illustrates an example of accessing a first address and a second address according to an embodiment of the present disclosure.

[0045] FIG10 is a schematic diagram showing the flow of a memory access method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0046] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0047] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0048] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0049] The development of large language models (LLMs) has made natural language processing (NLP) a hot topic in the field of general artificial intelligence. Due to their large size, large language models are often trained using a distributed approach, using multiple graphics processing units (GPUs) to jointly complete their training.

[0050] The all-reduce algorithm is a key technology for distributed training and can be used for both model and data parallelism. The operations involved in this algorithm primarily include data movement and simple mathematical operations such as addition and subtraction.

[0051] A classic type of global reduction algorithm is the ring all reduce algorithm. Figures 1 to 4b are schematic diagrams illustrating the process of executing the ring all reduce algorithm in a graphics processor in the prior art.

[0052] As shown in Figures 1-4b, assume that GPU0 through GPU3 are used to execute a ring-shaped global reduction algorithm. Data A is stored on GPU0, data B is stored on GPU1, data C is stored on GPU2, and data D is stored on GPU3. The goal of the ring-shaped global reduction algorithm is to store data A+B+C+D on GPU0 through GPU3, respectively (the data is stored in the GPU's corresponding memory or video memory).

[0053] In step 1, as shown in FIG1 , a free storage space is selected from the video memory or internal memory (not shown) corresponding to each graphics processor to be used as a shared buffer allocated to the graphics processor and the next graphics processor. Each buffer can be the same size. GPUs GPU0 and GPU1 share buffer 0, GPU1 and GPU2 share buffer 1, GPU2 and GPU3 share buffer 2, and GPU3 and GPU0 share buffer 3.

[0054] In step 2, as shown in Figure 2, the buffer is divided into N equal parts (e.g., N = 4) based on the number of GPUs N. The offset between two adjacent data moves is determined based on the amount of data in each part, and the GPU's own stored data is also divided into multiple parts based on the amount of data in each part. Each GPU moves the part of its stored data with the smallest offset (1 / 4) to the buffer shared with the next GPU. At this point, buffer 0 stores data A, buffer 1 stores data B, buffer 2 stores data C, and buffer 3 stores data D.

[0055] In step 3, as shown in Figure 3, each GPU reads the data previously moved by the previous GPU from the shared buffer. It then performs an operation (using addition as an example in Figure 2) on this data and its own stored data at the same offset. The result is stored at the same offset in the buffer shared with the next GPU. At this point, buffer 0 stores data D+A, buffer 1 stores data A+B, buffer 2 stores data B+C, and buffer 3 stores data C+D.

[0056] In step 4, as shown in FIG. 4 a and FIG. 4 b , step 3 is repeated until each GPU stores the same operation result A+B+C+D.

[0057] FIG. 5 a and FIG. 5 b are schematic diagrams showing a single graphics processor processing commands of a global reduction algorithm in the prior art.

[0058] Existing graphics processors use GPU cores to process commands from global reduction algorithms, completing data movement and computational operations. However, the GPU core's pipeline is highly complex and possesses powerful computing capabilities. When using the GPU core to execute these operations, only a minimal amount of its computing power is utilized, resulting in a waste of computing resources.

[0059] The two copies of data being calculated usually have no temporal and spatial locality and are stored in the memory or video memory corresponding to different graphics processors. They need to be read separately to the graphics processor core, calculated in the graphics processor core to obtain the calculation results, and then written to the memory or video memory corresponding to one of the graphics processors. In the examples of Figures 5a and 5b, (1) and (2) represent the two copies of data being read, and (3) represents the calculation results. In other words, for the graphics processor core, executing a command requires three memory accesses. And each time the graphics processor core accesses memory, it needs to pass through the cache and bus, so the path for each memory access is relatively long. Due to the large number of memory accesses and the long memory access path, the memory access efficiency is relatively low and the power consumption is relatively high.

[0060] In view of this, the present disclosure provides a memory access system, method, first graphics processor, and electronic device. The memory access system of the disclosed embodiment is disposed within the first graphics processor, but outside the core of the first graphics processor. The memory access system has computing power and can replace the graphics processor core to perform simple mathematical operations, thereby freeing up the computing resources of the graphics processor core. Furthermore, the path for moving data to the memory access system is shorter, which can improve command execution efficiency and reduce power consumption.

[0061] 6a and 6b illustrate exemplary application scenarios of a memory access system according to an embodiment of the present disclosure.

[0062] As shown in Figures 6a and 6b, the memory access system of the embodiment of the present disclosure can be set in the graphics processor and located outside the core of the graphics processor. The memory access system is also connected to the memory or video memory of the graphics processor.

[0063] The GPU is also connected to the CPU. When executing the global reduction algorithm, the CPU generates multiple commands that are executed in a specific order. These commands can be executed by multiple GPUs (only the first and second GPUs are shown in Figures 6a and 6b), and each GPU can have the same architecture. Each command is executed by a GPU, and executing the command requires copy and write operations, and may also require computation operations.

[0064] The CPU transmits multiple commands to each GPU so that the memory access system can clearly understand the execution conditions of the commands executed by its GPU. Examples of commands and execution conditions can be found below and in the descriptions of Figures 7a-7f.

[0065] The commands executed by the first graphics processor may include the first command described below. Execution of the first command requires completion of copy, calculation, and write operations. When the execution conditions of the first command are met, the memory access system responds to the first command, copies data A and data B, performs a calculation on data A and data B, and writes the calculation result A+B (using addition as an example) to the address of one of the two copies of the data being calculated. In the examples of Figures 6a and 6b, the write address of the calculation result A+B and the storage address of data A can be the same. In the example of Figure 6a, the storage address of data A can be an address in the memory or video memory of the first graphics processor, and the storage address of data B can be an address in the memory or video memory of the second graphics processor. In the example of Figure 6b, the storage address of data A can be an address in the memory or video memory of the second graphics processor, and the storage address of data B can be an address in the memory or video memory of the first graphics processor. Execution of the first command does not require participation by the core of the first graphics processor.

[0066] The multiple commands may also include commands that only involve copying and writing, and do not involve calculations (such as the fourth command described below). In this case, the command is still executed by the memory access system, and the execution effect may be to copy data stored in the memory, video memory, or cache (not shown) of one graphics processor and write it to the memory, video memory, or cache (not shown) of another graphics processor.

[0067] The effects of executing multiple commands in sequence can be seen in the relevant descriptions of Figures 7b to 7f below.

[0068] Those skilled in the art should understand that the memory access system can also be applied to other types of global reduction scenarios besides the ring global reduction scenario, or to reduction scenarios, or to any scenario that requires a graphics processor to perform copy operations, write operations (optionally, also including calculation operations), and the embodiments of the present disclosure are not limited to this.

[0069] FIG7 a shows a schematic diagram of the structure of a memory access system according to an embodiment of the present disclosure.

[0070] As shown in FIG7a , in a possible implementation, the present disclosure proposes a memory access system, which is provided in a first graphics processor. The system includes a copy engine and a cache with a computing function. The copy engine and the cache are both provided outside the core of the first graphics processor, wherein:

[0071] The first graphics processor executes a first command from the central processing unit, the first command including a first address and a second address, the first address serving as a source operand address in the first command, and the second address serving as both a source operand address and a destination operand address in the first command;

[0072] When an execution condition of the first command is met, the copy engine is configured to copy the first operand stored at the first address in response to the first command, and transfer the first operand to a cache in the graphics processor corresponding to the second address;

[0073] The cache is used to, when receiving the first operand, perform an operation on the first operand and the second operand stored at the second address, and write the operation result into the second address.

[0074] For example, the memory access system may include a copy engine and a cache with computational capabilities. The copy engine may include direct memory access (DMA) capabilities, enabling it to access cache on the graphics processor and the GPU's corresponding memory or video memory. The cache may be any cache that supports various types of mathematical operations, such as a last-level cache (LLC). The copy engine and cache may be located outside the graphics processor core.

[0075] In one possible implementation, the cache supports operations including addition, subtraction, multiplication, division, maximum, and minimum. Those skilled in the art will appreciate that the cache supports more types of operations, such as logical AND and logical OR. The present disclosure does not limit the supported cache operations.

[0076] Furthermore, a single graphics processor may include at least one replication engine and at least one cache with computing capabilities. The embodiments of the present disclosure do not limit the specific number of replication engines and caches with computing capabilities.

[0077] The first graphics processor may execute a first command from the central processing unit. The first command may include a first address and a second address. The first address serves as a source operand address in the first command, and the second address serves as both a source operand address and a destination operand address in the first command. Since both addresses serve as source operand addresses, the first command includes an arithmetic operation. For example, if the arithmetic operation type of the first command is addition, the first address is the address of data B, and the second address is the address of data A, an example of the first command may be B+=A.

[0078] When the execution conditions of the first command are met, the copy engine can respond to the first command by copying the first operand stored at the first address, such as data B. The copy engine can then transfer the first operand to the cache in the graphics processor corresponding to the second address. In the example of Figure 7a, assuming that the second address is in the cache of the first graphics processor, or in the memory or video memory connected to the first graphics processor, the second address corresponds to the first graphics processor, and therefore the first operand can be transferred to the cache of the first graphics processor with computing capabilities.

[0079] If the second address is in the cache of the second graphics processor, or in the memory or video memory connected to the second graphics processor, the second address corresponds to the second graphics processor, so the first operand can be transferred to the cache with calculation function on the second graphics processor.

[0080] In response to receiving the first operand, the cache with the operation function can start to obtain the second operand stored at the second address, such as A. When the second address is the address of the cache with the operation function, there is no need to copy the second operand. When the second address is not the address of the cache with the operation function, the second operand can be copied from the second address to the cache. Then the first operand and the second operand can be operated, for example, B+=A is operated, and the operation result such as A+B is written to the second address. After the operation result is written, the execution of the first command is completed. For details of the cache's operation on the first operand and the second operand, and details of writing the operation result to the second address, please refer to the relevant description below.

[0081] According to an embodiment of the present disclosure, a memory access system is provided in a first graphics processor and includes a copy engine and a cache with computational capabilities. The first graphics processor executes a first command from a central processing unit (CPU). When the execution conditions of the first command are met, the copy engine responds to the first command by copying a first operand stored at a first address, allowing the copy engine to complete the copying of the first operand on behalf of the graphics processor core. The copy engine then transmits the first operand to a cache in the graphics processor corresponding to a second address. Upon receiving the first operand, the cache performs a computation on the first operand and a second operand stored at a second address, and writes the computation result to the second address, allowing the cache to complete the computation on the first operand and the second operand, as well as write the computation result, on behalf of the graphics processor core. The copy engine and the cache are both provided outside the first graphics processor core. In this case, the memory access system can complete the copy, computation, and write operations on behalf of the graphics processor core, thereby freeing up computing resources of the graphics processor core. Furthermore, the paths for the copy and write operations performed by the memory access system are relatively short, thereby improving command execution efficiency and reducing power consumption.

[0082] In the disclosed embodiment, the memory access system only needs to execute a single command to complete the three operations of copying, computing, and writing. However, for a graphics processor core, a single command is typically used to complete only one operation, making it difficult to use a single command to perform multiple operations. Therefore, the disclosed embodiment also simplifies commands.

[0083] The copy engine with DMA function and the cache with computing function can be implemented based on existing technologies, and their structures will not be described in detail here.

[0084] In one possible implementation, the first command is one of multiple commands generated by a central processing unit, the multiple commands generated by the central processing unit are used to enable multiple graphics processors to jointly implement a ring global reduction operation, and the first graphics processor is one of the multiple graphics processors.

[0085] The execution condition of the first command is that the first command does not depend on other commands, or the copy engine detects that the first graphics processor has received information that the second command, on which the first command depends, has completed execution, wherein the first address included in the first command is the same as the second address included in the second command.

[0086] For example, the first command may be dependent on other commands, such as the first command uses the execution result of the second command on which it depends to perform an operation. In this case, the first address included in the first command is the same as the second address included in the second command. The first command may also be dependent on other commands, such as the third command that depends on the first command uses the execution result of the first command to perform an operation. In this case, the first address included in the third command is the same as the second address included in the first command. The first command may also have no dependency on other commands. If the first command does not depend on other commands, the first command can be executed directly. If the first command also depends on the second command, the first command can be executed after the second command. In this regard, the execution condition of the first command may be that the first command does not depend on other commands, or the copy engine detects that the first graphics processor has received information that the execution of the second command on which the first command depends has been completed.

[0087] The multiple graphics processors of the embodiment of the present disclosure can communicate in a ring, and in this case, the multiple graphics processors can implement a ring global reduction algorithm.

[0088] The following uses the commands that implement the Ring Global Reduction Algorithm as an example to describe the characteristics of commands generated by the CPU and how the GPU executes them.

[0089] 7b-7f are schematic diagrams illustrating a method in which a central processing unit generates multiple commands and a graphics processor executes the commands according to an embodiment of the present disclosure.

[0090] The CPU can first determine which GPUs store data that requires computation using the ring global reduction algorithm. Suppose it determines that the data stored on GPUs GPU0 and GPU2 requires computation using the ring global reduction algorithm. GPU0 stores data A, GPU1 stores data B, and GPU2 stores data C. The ring global reduction algorithm ensures that data A+B+C (data stored in the corresponding GPU memory or video memory) is stored on all GPUs GPU0 and GPU2.

[0091] In step 1, the data stored on each GPU can be divided into three parts based on the number of GPUs N (N=3). As shown in Figure 7b, when GPUs GPU0-GPU2 are not executing commands, data A stored on GPU0 is divided into three parts: data A1, A2, and A3; data B stored on GPU1 is divided into three parts: data B1, B2, and B3; and data C stored on GPU2 is divided into three parts: data C1, C2, and C3.

[0092] The CPU can then determine that the commands to be generated are of two types: calculation type and transfer type. Calculation type commands are generated and executed before transfer type commands. Executing calculation type commands requires completing copy, calculation, and write operations, while executing transfer type commands requires completing copy and write operations. The total number of calculation type commands can be N(N-1)=6. The total number of transfer type commands can be N(N-1)=6.

[0093] The following describes the computational commands generated by the CPU. These commands can be assigned to N-1 = 2 computation rounds. In each round, each GPU executes one command of the same computational type. The commands executed in each round are independent of each other and can be executed in parallel.

[0094] The first command of the first round of operations can be executed by graphics processor GPU0. This command is used to copy the 1 / 3 portion of data B1 from graphics processor GPU1, perform an operation on the 1 / 3 portion of data A1 stored in GPU0, and write the operation result to the storage location of data A1. Therefore, the first address included in the first command of the first round of operations can be the address of data B1, and the second address can be the address of data A1.

[0095] The second command of the first round of operations can be executed by GPU1. This command is used to copy the second-third portion of data C2 from GPU2 and perform an operation on the second-third portion of data B2 stored in GPU1, and write the operation result to the storage location of data B2. Therefore, the first address included in the second command of the first round of operations can be the address of data C2, and the second address can be the address of data B2.

[0096] The third command of the first round of operations can be executed by graphics processor GPU2. This command is used to copy the 3 / 3 portion of data A3 from graphics processor GPU0, perform an operation on the 3 / 3 portion of data C3 stored in GPU2, and write the operation result to the storage location of data C3. Therefore, the first address included in the third command of the first round of operations can be the address of data A3, and the second address can be the address of data C3.

[0097] As shown in Figure 7c, after the three commands in the first round of operations are executed, the three copies of data stored on GPU0 are A1+B1, A2, and A3; the three copies of data stored on GPU1 are B1, B2+C2, and B3; and the three copies of data stored on GPU2 are C1, C2, and C3+A3. None of the three commands in the first round of operations depend on any other command.

[0098] The first command of the second round of operations can be executed by graphics processor GPU0. This command is used to copy the 2 / 3 portion of data B2+C2 from graphics processor GPU1 and perform an operation on the 3 / 3 portion of data A2 stored in GPU0, and write the operation result to the storage location of data A2. Therefore, the first address included in the first command of the second round of operations can be the address of data B2+C2, and the second address can be the address of data A2. Because data B2+C2 is the result of executing the second command of the first round of operations, the first command of the second round of operations is dependent on the second command of the first round of operations.

[0099] The second command of the second round of operations can be executed by graphics processor GPU1. This command is used to copy the 3 / 3 portion of data C3+A3 from graphics processor GPU2, perform an operation on the 3 / 3 portion of data B3 stored in GPU1, and write the operation result to the storage location of data B3. Therefore, the first address included in the second command of the second round of operations can be the address of data C3+A3, and the second address can be the address of data B3. Because data C3+A3 is the result of executing the third command of the first round of operations, the second command of the second round of operations is dependent on the third command of the first round of operations.

[0100] The third command of the second round of operations can be executed by graphics processor GPU2. This command is used to copy the 1 / 3 portion of data A1+B1 from graphics processor GPU0 and perform an operation on the 1 / 3 portion of data C1 stored in its own memory, and then write the operation result to the storage location of data C1. Therefore, the first address included in the third command of the second round of operations can be the address of data A1+B1, and the second address can be the address of data C1. Because data A1+B1 is the result of executing the first command of the first round of operations, the third command of the second round of operations is dependent on the first command of the first round of operations.

[0101] As shown in Figure 7d, after the three commands of the second round of calculation are executed, the three copies of data stored on the graphics processor GPU0 are A1+B1, A2+B2+C2, and A3, the three copies of data stored on the graphics processor GPU1 are B1, B2+C2, and B3+C3+A3, and the three copies of data stored on the graphics processor GPU2 are C1+A1+B1, C2, and C3+A3.

[0102] The following describes the move-type commands generated by the CPU. These move-type commands can be assigned to N-1 = 2 move rounds. In each move round, each GPU executes one command. The commands executed in each move round have no dependencies and can be executed in parallel.

[0103] The first command of the first round of migration can be executed by graphics processor GPU0. This command is used to write the 2 / 3 portion of data A2+B2+C2 stored in itself to the storage location of the 2 / 3 portion of data in graphics processor GPU1 (that is, the storage location of data B2+C2). Therefore, the first address included in the first command of the first round of migration can be the address of data A2+B2+C2 in graphics processor GPU0, and the second address can be the address of data B2+C2. Since data A2+B2+C2 in graphics processor GPU0 is the execution result of the first command of the second round of calculation, the first command of the first round of migration depends on the first command of the second round of calculation.

[0104] The second command of the first round of migration can be executed by graphics processor GPU1. This command is used to write the 3 / 3rd portion of data B3+C3+A3 stored in itself to the storage location of the 3 / 3rd portion of data in graphics processor GPU2 (that is, the storage location of data C3+A3). Therefore, the first address included in the second command of the first round of migration can be the address of data B3+C3+A3 in graphics processor GPU1, and the second address can be the address of data C3+A3. Since data B3+C3+A3 in graphics processor GPU1 is the execution result of the second command of the second round of calculation, the second command of the first round of migration depends on the second command of the second round of calculation.

[0105] The third command of the first round of migration can be executed by graphics processor GPU2. This command is used to write the 1 / 3 portion of data C1+A1+B1 stored in itself to the storage location of the 1 / 3 portion of data in graphics processor GPU0 (that is, the storage location of data A1+B1). Therefore, the first address included in the third command of the first round of migration can be the address of data C1+A1+B1 in graphics processor GPU2, and the second address can be the address of data A1+B1. Since data C1+A1+B1 in graphics processor GPU2 is the execution result of the third command of the second round of calculation, the third command of the first round of migration depends on the third command of the second round of calculation.

[0106] As shown in Figure 7e, after the execution of the three commands in the first round of migration, the three copies of data stored on the graphics processor GPU0 are C1+A1+B1, A2+B2+C2, and A3; the three copies of data stored on the graphics processor GPU1 are B1, A2+B2+C2, and B3+C3+A3; and the three copies of data stored on the graphics processor GPU2 are C1+A1+B1, C2, and B3+C3+A3.

[0107] The first command of the second round of migration can be executed by graphics processor GPU0. This command is used to write the 1 / 3 portion of data C1+A1+B1 stored in itself to the storage location of the 1 / 3 portion of data in graphics processor GPU1 (that is, the storage location of data B1). Therefore, the first address included in the first command of the second round of migration can be the address of data C1+A1+B1 in graphics processor GPU0, and the second address can be the address of data B1. Since data C1+A1+B1 in graphics processor GPU0 is the result of executing the third command of the first round of migration, the first command of the second round of migration is dependent on the third command of the first round of migration.

[0108] The second command of the second round of migration can be executed by graphics processor GPU1. This command is used to write the 2 / 3rd portion of data A2+B2+C2 stored in itself to the storage location of the 2 / 3rd portion of data in graphics processor GPU2 (that is, the storage location of data C2). Therefore, the first address included in the second command of the second round of migration can be the address of data A2+B2+C2 in graphics processor GPU1, and the second address can be the address of data C2. Since data A2+B2+C2 in graphics processor GPU1 is the result of executing the first command of the first round of migration, the second command of the second round of migration is dependent on the first command of the first round of migration.

[0109] The third command of the second round of migration can be executed by graphics processor GPU2. This command is used to write the 3 / 3rd portion of data B3+C3+A3 stored in itself to the storage location of the 3 / 3rd portion of data in graphics processor GPU0 (i.e., the storage location of data A3). Therefore, the first address included in the third command of the second round of migration can be the address of data B3+C3+A3 in graphics processor GPU2, and the second address can be the address of data A3. Since data B3+C3+A3 in graphics processor GPU2 is the result of executing the second command of the first round of migration, the third command of the second round of migration is dependent on the second command of the first round of migration.

[0110] As shown in Figure 7f, after the three commands in the second round of migration are executed, the three copies of data stored on GPUs GPU0, GPU1, and GPU2 are C1+A1+B1, A2+B2+C2, and B3+C3+A3. Each GPU then calculates the three copies of data it has stored to obtain A+B+C.

[0111] The CPU can mark the generated commands with dependencies and transmit all generated commands to each GPU separately. When each GPU determines that a received command meets the execution conditions, it can execute the command to achieve the corresponding effect.

[0112] In the examples of Figures 7b-7f, each GPU can function as the first GPU. When a calculation-type command is used as the first command, the included second address corresponds to the first GPU, and the included first address corresponds to the GPU immediately following the first GPU. When a move-type command is used as the first command, the included first address corresponds to the first GPU, and the included second address corresponds to the GPU immediately following the first GPU. Those skilled in the art will appreciate that, in practical applications, there is no restriction on which GPUs the second and first addresses included in the first command specifically correspond to, as long as a ring-type global reduction operation can be implemented.

[0113] In a possible implementation, multiple graphics processors respectively store multiple data to be calculated, and the calculation result of the ring global reduction operation performed by the multiple graphics processors together is the calculation result of the multiple data to be calculated.

[0114] When the first command does not depend on other commands, the data stored at the first address included in the first command is part of the data to be calculated stored on the graphics processor corresponding to the first address, and the data stored at the second address included in the first command is part of the data to be calculated stored on the graphics processor corresponding to the second address;

[0115] When a first command depends on other commands, the data stored at a first address included in the first command is a calculation result of part of the data to be calculated, and the data stored at a second address included in the first command is a part of the data to be calculated stored on the graphics processor corresponding to the second address, or is a calculation result of part of the data to be calculated.

[0116] For example, in a ring global reduction computing scenario, multiple graphics processors respectively store multiple data to be computed, such as data A, B, and C in the relevant description of Figure 7b. At this time, the computing result of the multiple data to be computed can be A+B+C.

[0117] If the first command does not depend on other commands, such as the three commands in the first round of operations described above, it can be seen that the operation object of each command is a portion of the data stored on the GPU. In other words, when the first command does not depend on other commands, the data stored at the first address included in the first command is a portion of the data to be operated on stored on the GPU corresponding to the first address, and the data stored at the second address included in the first command is a portion of the data to be operated on stored on the GPU corresponding to the second address.

[0118] If the first command depends on other commands, such as the three commands in the second round of calculations, the three commands in the first round of transfers, and the three commands in the second round of transfers described above, it can be seen that the data stored at the first address must be the execution result of the other command, that is, the calculation result of part of the data to be calculated. The data stored at the second address may be the execution result of the other command, or it may be a portion of the data to be calculated stored on the graphics processor corresponding to the second address. That is, when the first command depends on other commands, the data stored at the first address included in the first command is the calculation result of part of the data to be calculated, and the data stored at the second address included in the first command is a portion of the data to be calculated stored on the graphics processor corresponding to the second address, or the calculation result of part of the data to be calculated.

[0119] In one possible implementation, the cache is also used to transmit information indicating that the calculation result has been written to the replication engine;

[0120] The copy engine is further configured to transmit, upon receiving information indicating completion of writing the calculation result, information indicating completion of executing the first command to a graphics processor configured to execute a third command, wherein the third command depends on the first command, and a first address included in the third command is the same as a second address included in the first command.

[0121] For example, when the calculation result is written, it means that the first command has completed execution. At this point, the third command that depends on the first command can begin execution. Therefore, the cache can transmit the calculation result writing completion information to the replication engine to notify the replication engine of the completion of the first command. The replication engine can then transmit the first command execution completion information to the graphics processor executing the third command to notify the GPU executing the third command that it can begin execution of the third command.

[0122] In this way, the execution of commands can be synchronized between multiple graphics processors, and multiple commands can be automatically executed in a preset order. The memory access system does not need to additionally control the execution order of commands, reducing the design complexity of the memory access system.

[0123] In one possible implementation, the cache is specifically used to:

[0124] Upon receiving the first operand, performing an operation on the first operand and the second operand in an atomic operation;

[0125] The operation result is written to the second address through an atomic write operation.

[0126] For example, in certain circumstances, the second address may be accessible to two or more graphics processors. During the execution of a single command, data consistency must be maintained, and atomic operations are uninterrupted, so the cache can be configured to perform atomic operations by default. In this case, the cache can perform atomic copy, atomic operation, and atomic write operations in response to the first operand, ensuring the accuracy of the result.

[0127] That is, upon receiving the first operand, the cache performs an operation on the first operand and the second operand, which may be performed through an atomic operation. The cache writes the operation result to the second address, which may be performed through an atomic write operation. If the cache needs to copy the first operand, it may copy the first operand through an atomic copy operation.

[0128] In one scenario, the copy engine on the first graphics processor can transmit the commands generated by the central processor and to be executed by the first graphics processor to the cache in sequence. The cache can record the order in which the received operands are received. When the i-th operand is received, it can directly determine that the i-th command to be executed is received by finding the second operand stored at the second address in the i-th command.

[0129] In another scenario, when the copy engine responds to a command, it can also directly convert the command into an atomic operation type. For example, when the execution condition of the first command is met, the copy engine can also convert the first command into an atomic operation type command, and the converted command also includes a first address and a second address. The copy engine can transmit the converted command and the first operand together to the cache in the graphics processor corresponding to the second address (in the example of Figure 7a, it is the first graphics processor). The first operand is used as a parameter when executing the converted command.

[0130] The cache having the operation function may receive the converted command and the first operand, obtain the second operand according to the second address in the converted command, perform the operation on the first operand and the second operand, and write the operation result to the second address. After the operation result is written, the execution of the converted command is completed, and the execution of the first command can be considered to be completed.

[0131] For clarity of description, the following text uses the copy engine transmitting only the first operand to the cache as an example. However, those skilled in the art will appreciate that if the copy engine needs to transmit the converted command, the converted command and the first operand can be transmitted in the same manner.

[0132] In a possible implementation, the first address and the second address correspond to different graphics processors.

[0133] For example, the first address and the second address may correspond to different graphics processors. For example, the first address and the second address may correspond to a first graphics processor and a second graphics processor, respectively. When the first address and the second address correspond to different graphics processors, the methods for accessing the first address and the second address are also different. Figures 8 and 9 respectively illustrate examples of accessing the first address and the second address according to embodiments of the present disclosure.

[0134] In one possible implementation, the first graphics processor further includes an intra-chip interconnect bus connecting the cache and the copy engine, and the first graphics processor is connected to the second graphics processor via an inter-chip interconnect bus.

[0135] When the first address corresponds to the second graphics processor and the second address corresponds to the first graphics processor, the copy engine reads the first operand through the intra-chip interconnect bus and the inter-chip interconnect bus, and transmits the first operand through the intra-chip interconnect bus; or

[0136] When the first address corresponds to the first graphics processor and the second address corresponds to the second graphics processor, the copy engine reads the first operand through the intra-chip interconnect bus and transmits the first operand through the intra-chip interconnect bus and the inter-chip interconnect bus.

[0137] For example, as shown in Figures 8 and 9, a first graphics processor may include a replication engine, a cache with computing capabilities, a graphics processor core, and an intra-chip interconnect bus connecting the cache and replication engine. A second graphics processor also includes a replication engine, a cache with computing capabilities, a graphics processor core, and an intra-chip interconnect bus connecting the cache and replication engine. The intra-chip interconnect bus further connects the graphics processor core. The first and second graphics processors are connected via an inter-chip interconnect bus.

[0138] In the example of FIG8 , the first address corresponds to the second graphics processor and is used to store data B. Because the first operand is read from the first address, the path taken by the copy engine to copy the first operand may include an intra-chip interconnect bus on the first graphics processor, an inter-chip interconnect bus, an intra-chip interconnect bus on the second graphics processor, a cache on the second graphics processor, and optionally, the memory or video memory of the second graphics processor. If the first address is an address in the cache of the second graphics processor, the path may not include the memory or video memory of the second graphics processor. If the first address is an address in the memory or video memory of the second graphics processor, the path may include the memory or video memory of the second graphics processor.

[0139] The second address corresponds to the first graphics processor and is used to store data A. The copy engine transfers the first operand to the cache in the graphics processor corresponding to the second address, so the path for transferring the first operand may include the intra-chip interconnect bus and the cache on the first graphics processor.

[0140] If the second address is an address in the cache of the first GPU, the cache of the first GPU no longer needs to access the memory for the calculation result (A+B). If the second address is an address in the memory or video memory of the first GPU, the cache of the first GPU can still write the calculation result to the memory or video memory of the first GPU.

[0141] In the example of FIG9 , the first address corresponds to the first graphics processor and is used to store the first operand (data B). Since the first operand is read from the first address, the path taken by the copy engine to copy the first operand may include the on-chip interconnect bus, the cache on the first graphics processor, and the memory or video memory of the first graphics processor (optional). If the first address is an address in the cache on the first graphics processor, the path may not include the memory or video memory of the first graphics processor. If the first address is an address in the memory or video memory of the first graphics processor, the path may include the memory or video memory of the first graphics processor.

[0142] The second address corresponds to the second graphics processor and is used to store the second operand (data A). The copy engine transfers the first operand to a cache in the graphics processor corresponding to the second address. Therefore, a path for transferring the first operand may include an intra-chip interconnect bus on the first graphics processor, an inter-chip interconnect bus, an intra-chip interconnect bus on the second graphics processor, a cache on the second graphics processor, and optionally, a memory or video memory of the second graphics processor.

[0143] If the second address is an address in the cache of the second GPU, the cache of the second GPU does not need to access the memory for the calculation result (A+B). If the second address is an address in the memory or video memory of the second GPU, the cache of the second GPU can still write the calculation result to the memory or video memory of the second GPU.

[0144] In this way, the memory access method of the replication engine is made more flexible.

[0145] In one possible implementation, the first graphics processor executes a fourth command from the central processing unit, where the fourth command includes a third address and a fourth address, the third address serving as a source operand address in the fourth command, and the fourth address serving as a destination operand address in the fourth command;

[0146] When an execution condition of the fourth command is met, the copy engine is further configured to execute the fourth command to read a third operand stored at the third address, and transfer the third operand to a cache in the graphics processor corresponding to the fourth address;

[0147] The cache is further configured to write the third operand into the fourth address upon receiving the third operand.

[0148] For example, the first graphics processor can also be used to execute a fourth command that does not include an operation (i.e., a move-type command, see the description of Figures 7e and 7f for examples). In this case, the fourth command may include a third address and a fourth address, with the third address serving as a source operand address in the fourth command and the fourth address serving as a destination operand address in the fourth command.

[0149] When the execution condition of the fourth command is met, the copy engine may respond to the fourth command to copy the third operand stored at the third address, and then transfer the third operand to the cache in the graphics processor corresponding to the fourth address.

[0150] For example, when the fourth address is in the cache of the first graphics processor, or in the memory or video memory connected to the first graphics processor, the fourth address corresponds to the first graphics processor, so the third operand can be transferred to the cache with computing function on the first graphics processor.

[0151] If the fourth address is in the cache of the second graphics processor, or in the memory or video memory connected to the second graphics processor, the fourth address corresponds to the second graphics processor, so the third operand can be transferred to the cache with calculation function on the second graphics processor.

[0152] The cache with the operation function is used to write the third operand into the fourth address through an atomic write operation when receiving the third operand.

[0153] Exemplarily, the copy engine may also convert the fourth command into a command of an atomic operation type, where the converted command also includes a third address and a fourth address. The copy engine may transmit the converted command and the third operand together to the cache. After receiving the converted command and the third operand, the cache writes the third operand to the fourth address according to the fourth address in the converted command. After the third operand is written, the converted command is executed and the fourth command is also executed.

[0154] In this way, the graphics processor can support and process more types of commands.

[0155] In a possible implementation, the execution condition of the fourth command is that the fourth command does not depend on other commands, or the execution condition of the fourth command is that the copy engine detects that the first graphics processor has received information that the sixth command, on which the fourth command depends, has been executed.

[0156] Similar to the execution conditions of the first command, the fourth command may not depend on other commands, or it may have dependencies on other commands. In this regard, the execution condition for the fourth command may be that the fourth command does not depend on other commands, or that the execution condition for the fourth command is that the replication engine detects that the first graphics processor has received information that the sixth command, on which the fourth command depends, has completed execution. This information may come from the graphics processor executing the sixth command. The first address included in the fourth command is the same as the second address included in the sixth command.

[0157] In a possible implementation, the cache is further configured to transmit information indicating completion of writing the third operand to the copy engine;

[0158] The copy engine is further configured to transmit, upon receiving information indicating completion of writing the calculation result, information indicating completion of executing the fourth command to a graphics processor configured to execute an eighth command, wherein the eighth command depends on the fourth command, and a first address included in the eighth command is the same as a second address included in the fourth command.

[0159] For example, when the third operand is written, the fourth command is also executed. At this point, the eighth command, which depends on the fourth command, can begin execution. Therefore, the cache can transmit the completion information of the third operand writing to the copy engine to notify the copy engine of the completion of the fourth command. The copy engine can then transmit the completion information of the fourth command to the graphics processor executing the eighth command, notifying the graphics processor executing the eighth command that it can begin executing the eighth command.

[0160] In one possible implementation, the third address and the fourth address correspond to different graphics processors. When the third address and the fourth address correspond to different graphics processors, the methods for accessing the third address and the fourth address are also different. For an example, see the description of Figures 8 and 9 , which illustrate how to access the first address and the second address when they correspond to different processors. Simply replace the first address with the third address and the second address with the fourth address. The methods for accessing the third address and the fourth address are not further described here.

[0161] The present disclosure further provides a memory access method. FIG10 is a schematic diagram showing a flow of the memory access method according to an embodiment of the present disclosure.

[0162] As shown in FIG10 , in one possible implementation, the method is applied to a memory access system, the system being disposed in a first graphics processor, the system including a copy engine and a cache having a computing function, the copy engine and the cache both being disposed outside a core of the first graphics processor, and the method including:

[0163] The first graphics processor executes a first command from the central processing unit, the first command including a first address and a second address, the first address serving as a source operand address in the first command, and the second address serving as both a source operand address and a destination operand address in the first command;

[0164] Step S11, when an execution condition of the first command is met, the copy engine responds to the first command to copy the first operand stored at the first address, and transfers the first operand to a cache in the graphics processor corresponding to the second address;

[0165] Step S12: When the cache receives the first operand, it operates on the first operand and the second operand stored at the second address, and writes the operation result into the second address.

[0166] In one possible implementation, the first command is one of multiple commands generated by the central processing unit, and the multiple commands generated by the central processing unit are used to enable multiple graphics processors to jointly implement a ring global reduction operation. The first graphics processor is one of the multiple graphics processors. The execution condition of the first command is that the first command does not depend on other commands, or the replication engine detects that the first graphics processor has received information that a second command, on which the first command depends, has completed execution. The first address included in the first command is the same as the second address included in the second command.

[0167] In a possible implementation, the method further includes:

[0168] The cache transmits information that the operation result is written to the replication engine;

[0169] Upon receiving the information indicating that the operation result has been written, the copy engine transmits information indicating that the first command has been executed to a graphics processor configured to execute a third command, wherein the third command is dependent on the first command, and a first address included in the third command is the same as a second address included in the first command.

[0170] In a possible implementation, upon receiving the first operand, the cache performs an operation on the first operand and a second operand stored at the second address, and writes the operation result to the second address, including:

[0171] When receiving the first operand, the cache performs an operation on the first operand and the second operand in an atomic operation manner; and writes the operation result to the second address through an atomic write operation.

[0172] In a possible implementation, the first address and the second address correspond to different graphics processors.

[0173] In a possible implementation, the first graphics processor further includes an intra-chip interconnect bus connecting the cache and the copy engine, and the first graphics processor is connected to the second graphics processor via an inter-chip interconnect bus.

[0174] When the first address corresponds to the second graphics processor and the second address corresponds to the first graphics processor, the copy engine reads the first operand through the intra-chip interconnect bus and the inter-chip interconnect bus, and transmits the first operand through the intra-chip interconnect bus; or

[0175] When the first address corresponds to the first graphics processor and the second address corresponds to the second graphics processor, the copy engine reads the first operand through the intra-chip interconnect bus and transmits the first operand through the intra-chip interconnect bus and the inter-chip interconnect bus.

[0176] In a possible implementation, the method further includes:

[0177] The first graphics processor executes a fourth command from the central processing unit, the fourth command including a third address and a fourth address, the third address serving as a source operand address in the fourth command, and the fourth address serving as a destination operand address in the fourth command;

[0178] When an execution condition of the fourth command is met, the copy engine responds to the fourth command to copy the third operand stored at the third address, and transfers the third operand to a cache in the graphics processor corresponding to the fourth address;

[0179] Upon receiving the third operand, the cache writes the third operand into the fourth address.

[0180] In a possible implementation, the operation type supported by the cache includes any one of addition, subtraction, multiplication, division, maximum value, and minimum value.

[0181] In a possible implementation, the multiple graphics processors respectively store multiple data to be calculated, and the calculation result of the ring global reduction calculation performed by the multiple graphics processors together is the calculation result of the multiple data to be calculated.

[0182] When the first command does not depend on other commands, the data stored at the first address included in the first command is part of the data to be calculated stored on the graphics processor corresponding to the first address, and the data stored at the second address included in the first command is part of the data to be calculated stored on the graphics processor corresponding to the second address;

[0183] When the first command depends on other commands, the data stored at the first address included in the first command is a calculation result of part of the data to be calculated, and the data stored at the second address included in the first command is a part of the data to be calculated stored on the graphics processor corresponding to the second address, or is a calculation result of part of the data to be calculated.

[0184] The present disclosure further provides a first graphics processor, comprising the memory access system described above. A schematic diagram of the structure of the first graphics processor can be seen in Figures 8 and 9 .

[0185] The present disclosure also provides an electronic device, including the first graphics processor described above. The electronic device may be a server or a terminal device, and the embodiments of the present disclosure do not limit the specific type of the electronic device.

[0186] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0187] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A memory access system, characterized in that: The system is set in a first graphics processor, and the system includes a copy engine and a cache with a computing function, and the copy engine and the cache are both set outside the core of the first graphics processor, wherein, The first graphics processor executes a first command from the central processing unit, the first command including a first address and a second address, the first address serving as a source operand address in the first command, and the second address serving as both a source operand address and a destination operand address in the first command; When an execution condition of the first command is met, the copy engine is configured to respond to the first command by copying a first operand stored at the first address, and transferring the first operand to a cache in the graphics processor corresponding to the second address; The cache is used to, upon receiving the first operand, perform an operation on the first operand and a second operand stored at the second address, and write the operation result into the second address.

2. The system according to claim 1, wherein: The first command is one of a plurality of commands generated by the central processing unit, the plurality of commands generated by the central processing unit are used to enable a plurality of graphics processors to jointly implement a ring global reduction operation, the first graphics processor being one of the plurality of graphics processors, The execution condition of the first command is that the first command does not depend on other commands, or the copy engine detects that the first graphics processor has received information that a second command, on which the first command depends, has been executed, wherein a first address included in the first command is the same as a second address included in the second command.

3. The system according to claim 1 or 2, characterized in that The cache is further configured to transmit information indicating that the operation result has been written to the replication engine; The copy engine is further configured to transmit, upon receiving information indicating completion of writing the calculation result, information indicating completion of executing the first command to a graphics processor configured to execute a third command, wherein the third command is dependent on the first command, and a first address included in the third command is the same as a second address included in the first command.

4. The system according to claim 1, wherein: The cache is specifically used for: Upon receiving the first operand, performing an operation on the first operand and the second operand in an atomic operation; The operation result is written into the second address through an atomic write operation.

5. The system according to claim 1, wherein: The first address and the second address correspond to different graphics processors.

6. The system according to claim 5, characterized in that The first graphics processor further includes an intra-chip interconnect bus connecting the cache and the copy engine, and the first graphics processor is connected to the second graphics processor via an inter-chip interconnect bus. When the first address corresponds to the second graphics processor and the second address corresponds to the first graphics processor, the copy engine reads the first operand through the intra-chip interconnect bus and the inter-chip interconnect bus, and transmits the first operand through the intra-chip interconnect bus; or When the first address corresponds to the first graphics processor and the second address corresponds to the second graphics processor, the copy engine reads the first operand through the intra-chip interconnect bus and transmits the first operand through the intra-chip interconnect bus and the inter-chip interconnect bus.

7. The system according to claim 1, wherein: The first graphics processor executes a fourth command from the central processing unit, the fourth command including a third address and a fourth address, the third address serving as a source operand address in the fourth command, and the fourth address serving as a destination operand address in the fourth command; When an execution condition of the fourth command is met, the copy engine is further configured to respond to the fourth command to copy the third operand stored at the third address, and transfer the third operand to a cache in the graphics processor corresponding to the fourth address; The cache is further configured to write the third operand into the fourth address upon receiving the third operand.

8. The system according to claim 1, wherein: The operation types supported by the cache include any one of addition, subtraction, multiplication, division, maximum value, and minimum value.

9. The system according to claim 2, wherein: The multiple graphics processors respectively store multiple data to be calculated, and the calculation result of the ring global reduction calculation performed by the multiple graphics processors together is the calculation result of the multiple data to be calculated. When the first command does not depend on other commands, the data stored at the first address included in the first command is part of the data to be calculated stored on the graphics processor corresponding to the first address, and the data stored at the second address included in the first command is part of the data to be calculated stored on the graphics processor corresponding to the second address; When the first command depends on other commands, the data stored at the first address included in the first command is a calculation result of part of the data to be calculated, and the data stored at the second address included in the first command is a part of the data to be calculated stored on the graphics processor corresponding to the second address, or is a calculation result of part of the data to be calculated.

10. A memory access method, characterized in that: Applied to a memory access system, the system is provided in a first graphics processor, the system includes a copy engine and a cache with computing functions, the copy engine and the cache are both provided outside the core of the first graphics processor, the method includes: The first graphics processor executes a first command from the central processing unit, the first command including a first address and a second address, the first address serving as a source operand address in the first command, and the second address serving as both a source operand address and a destination operand address in the first command; When an execution condition of the first command is met, the copy engine responds to the first command to copy the first operand stored at the first address, and transmits the first operand to a cache in the graphics processor corresponding to the second address; When receiving the first operand, the cache performs an operation on the first operand and the second operand stored at the second address, and writes the operation result into the second address.

11. A first graphics processor, characterized in that: A memory access system comprising any one of claims 1-9.

12. An electronic device, characterized in that: The graphics processor according to claim 11 is included.

Images