Chip system, data processing method, electronic device, and storage medium

Abstract

The disclosure provides a chip system, a data processing method, an electronic device and a computer readable storage medium. The chip system comprises a cluster shared memory, a crossbar network and a plurality of computing units. The cluster shared memory comprises a plurality of memory banks. The crossbar network is coupled between the plurality of computing units and the plurality of memory banks. A data merging unit is arranged in or coupled to the computing unit. The data merging unit is configured to receive a memory access request initiated by the corresponding computing unit, and perform data merging processing on the memory access request to obtain a memory access request data set. The size of the memory access request data set is an integer multiple of the data bus width of the crossbar network. Through the above technical solution, the silicon area occupied by the crossbar network can be saved, thereby the manufacturing cost of the chip can be saved.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a chip system, a data processing method, an electronic device, and a storage medium. Background Technology

[0002] In a graphics processing unit (GPU), a compute unit (CU) consists of a copy engine (CE), a tensor processing engine (TPE), and multiple execution units (EUs). CE, TPE, and EUs can all access cluster shared memory. During the process of CE, TPE, and EUs accessing cluster shared memory, they can rely on a fine-grained cross-connect network. However, cluster shared memory is composed of multiple static random access memory (SRAM) banks, and the data width of the fine-grained cross-connect network is the same as the port width of the SRAM banks. Therefore, the fine-grained cross-connect network requires a large silicon area to meet the requirements of CU accessing any one of the SRAM banks, leading to an increase in chip manufacturing costs. Summary of the Invention

[0003] This disclosure provides a chip system, data processing method, electronic device, and storage medium that can save silicon wafer area occupied by cross-switch networks, thereby significantly reducing chip manufacturing costs.

[0004] According to one aspect of this disclosure, a chip system is provided, comprising:

[0005] Cluster shared memory, which includes multiple storage units;

[0006] Multiple computing units;

[0007] A cross-connect network is coupled between the plurality of said computing units and the plurality of said memory banks;

[0008] A data merging unit, disposed within or coupled to the computing unit, is used to receive memory access requests initiated by the computing unit and to perform data merging processing on the memory access requests to obtain a memory access request data set.

[0009] The size of the memory access request data set is an integer multiple of the data bus width of the crossbar network, and the memory access request data set is transmitted between the computing unit and the cluster shared memory through the crossbar network.

[0010] Optionally, the data in the cluster's shared memory is organized and addressed in blocks; the size of each block is an integer multiple of the data bus width of the crossbar switch network, and all data within the same block is accessed simultaneously.

[0011] Optionally, multiple storage units form a storage unit group, each storage unit group stores a data block, the size of each data block is an integer multiple of the data bus width of the crossbar switch network, and all data within the same data block is accessed simultaneously.

[0012] Optionally, the data bus width of the crossbar network is greater than the port width of a single memory bank, and the number of crossbars in the crossbar network is less than the product of the number of computing units and the number of memory banks.

[0013] Optionally, each computing unit includes multiple execution units.

[0014] The data merging unit includes a first merging buffer, which is used to receive memory access requests initiated by multiple execution units and to perform data merging processing on the memory access requests.

[0015] Optionally, each of the computing units includes a tensor processing engine, which includes multiple tensor processing sub-modules, wherein the tensor processing sub-modules are tensor processing systolic arrays or addition trees;

[0016] The data merging unit includes a first merging register, which is used to merge the memory access requests sent by multiple tensor processing submodules.

[0017] Optionally, each of the computing units includes a replication engine, which includes multiple replication submodules.

[0018] The data merging unit includes a second merging buffer, which is used to perform data merging processing on the memory access requests sent by multiple copying submodules.

[0019] According to one aspect of this disclosure, a data processing method is provided, applied to a chip system, the chip system including clustered shared memory, a crossbar network, a data merging unit, and multiple computing units, the clustered shared memory including multiple storage banks, the crossbar network coupled between the multiple computing units and the multiple storage banks, and the data merging unit disposed within or coupled to the computing units, the data processing method comprising:

[0020] The data merging unit receives the memory access request initiated by the corresponding computing unit;

[0021] The memory access request is merged based on the data merging unit to obtain a memory access request data set, wherein the size of the memory access request data set is an integer multiple of the data bus width of the crossbar switch network.

[0022] The memory access request data set is transmitted from the corresponding computing unit to the cluster shared memory via the crossbar network for data reading from the cluster shared memory.

[0023] Optionally, each computing unit includes multiple execution units, the data merging unit is a first merging cache, and receiving the memory access request initiated by the corresponding computing unit through the data merging unit includes:

[0024] The first merging buffer receives the memory access request initiated by any of the execution units in the corresponding computing units.

[0025] Optionally, the step of performing data merging processing on the memory access requests based on the data merging unit to obtain a memory access request data set includes:

[0026] After each data reception cycle, the memory access requests sent by the execution unit are processed by the first merging buffer according to a preset first byte throughput to obtain the memory access request data set.

[0027] Optionally, each of the computation units includes a tensor processing engine, the data merging unit is a first merging register, and the step of receiving the memory access request initiated by the corresponding computation unit through the data merging unit includes:

[0028] The first merging register receives the memory access request initiated by the tensor processing engine in the corresponding computing unit.

[0029] Optionally, the step of performing data merging processing on the memory access requests based on the data merging unit to obtain a memory access request data set includes:

[0030] The memory access request sent by the tensor processing submodule is processed by merging data based on the first merging register to obtain the memory access request data set. The tensor processing engine includes multiple tensor processing submodules, which are tensor processing systolic arrays or addition trees.

[0031] Optionally, each computing unit includes a replication engine, the data merging unit is a second merge cache, and receiving the memory access request initiated by the corresponding computing unit through the data merging unit includes:

[0032] The second merging cache receives the memory access request initiated by the replication engine in the corresponding computing unit.

[0033] Optionally, the step of performing data merging processing on the memory access requests based on the data merging unit to obtain a memory access request data set includes:

[0034] After each data reception cycle, the memory access requests sent by the replication submodule are processed by the second merging buffer according to a preset second byte throughput to obtain the memory access request data set. The replication engine includes multiple replication submodules.

[0035] Optionally, the step of transferring the memory access request data set from the corresponding computing unit to the cluster shared memory via the crossbar switch network for data reading from the cluster shared memory includes:

[0036] The address resolution unit in the chip system parses the memory access request data set to obtain data read address information.

[0037] Based on the data read address information, the corresponding data is read from the cluster shared memory and into the corresponding computing unit through the crossbar switch network.

[0038] Optionally, the data processing method further includes:

[0039] After a preset first number of data reception cycles, if the total data size of all memory access requests received through the first merging register does not reach the data size of the memory access request data set, then based on all the received memory access requests, the corresponding data is read from the cluster shared memory and into the corresponding computing unit through the cross-connect network.

[0040] Optionally, the data processing method further includes:

[0041] After a preset second data reception cycle, if the total data size of all memory access requests received through the second merging register does not reach the data size of the memory access request data set, then based on all the received memory access requests, the corresponding data is read from the cluster shared memory and into the corresponding computing unit through the cross-connect network.

[0042] According to one aspect of this disclosure, an electronic device is proposed, the electronic device including a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for implementing connection communication between the processor and the memory, wherein the program is executed by the processor to implement the data processing method as described above.

[0043] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores one or more programs, which can be executed by one or more processors to implement the data processing method described above.

[0044] The chip system, data processing method, electronic device, and storage medium disclosed herein include: a chip system comprising: a clustered shared memory, a crossbar network, a data merging unit, and multiple computing units; the clustered shared memory includes multiple storage banks, the crossbar network is coupled between the multiple computing units and the multiple storage banks, and the data merging unit is disposed within or coupled to the computing units; wherein, the data merging unit can receive memory access requests initiated by corresponding computing units, and then perform data merging processing on the received memory access requests to obtain a memory access request data set, the size of which is an integer multiple of the data bus width of the crossbar network; the memory access request data set can be transmitted from the corresponding computing unit to the clustered shared memory through the crossbar network for data reading from the clustered shared memory. The above technical solution utilizes a data merging unit to merge memory access requests, ensuring that the size of the memory access request data set is an integer multiple of the data bus width of the crossbar network. Finally, the memory access request data set is transmitted from the corresponding computing unit to the cluster shared memory via the crossbar network to achieve data reading. Unlike the previous method of using fine-grained data lines to read data from the cluster shared memory based on memory access requests, this solution reads data based on the memory access request data set while meeting data reading requirements. This significantly saves silicon area occupied by the crossbar network, thereby reducing chip manufacturing costs and simplifying the chip manufacturing process.

[0045] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0046] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0047] Figure 1 This is a system architecture diagram of the data processing method applied in the embodiments of this disclosure;

[0048] Figure 2 This is a schematic diagram of a structure in the prior art that connects computing units and cluster shared memory via a crossbar network;

[0049] Figure 3 This is a schematic diagram of the specific structure of a chip system according to an embodiment of the present disclosure;

[0050] Figure 4 This is a main flowchart of a data processing method according to an embodiment of the present disclosure;

[0051] Figure 5 yes Figure 4 The first sub-flowchart of step S201;

[0052] Figure 6 yes Figure 4 The first sub-flowchart of step S202;

[0053] Figure 7 This is a schematic diagram of an execution unit sending a memory access request to a first merging cache according to an embodiment of the present disclosure;

[0054] Figure 8 yes Figure 4 The second sub-flowchart of step S201;

[0055] Figure 9 yes Figure 4 The second sub-flowchart of step S202;

[0056] Figure 10 This is a schematic diagram of a tensor processing systolic array or adder tree sending a memory access request to a merge register according to an embodiment of this disclosure;

[0057] Figure 11 yes Figure 4 The third sub-flowchart of step S201;

[0058] Figure 12 yes Figure 4 The third sub-flowchart of step S202;

[0059] Figure 13 This is a schematic diagram of a replication engine sending a memory access request to a second merging cache according to an embodiment of this disclosure;

[0060] Figure 14 yes Figure 4 A sub-flowchart of step S203;

[0061] Figure 15 This is a first sub-flowchart of an embodiment of the present disclosure when the data size of a received memory access request does not meet the standard;

[0062] Figure 16 This is a second sub-flowchart illustrating a scenario where the data size of a received memory access request is insufficient, according to one embodiment of this disclosure.

[0063] Figure 17 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0065] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:

[0066] Graphics Processing Unit (GPU): A microprocessor specifically designed for processing graphics and image computations. Originally designed to accelerate computer graphics rendering, GPUs have expanded far beyond this scope with technological advancements. Initially designed to accelerate 2D and 3D graphics rendering, improving the performance of games and professional graphics software, GPUs possess thousands of cores capable of processing massive amounts of data simultaneously, making them ideal for parallel computing tasks. With technological advancements, GPUs are no longer limited to graphics processing; they are now used for a wide range of general-purpose computing tasks. GPUs play a crucial role in deep learning, machine learning, and artificial intelligence because they can rapidly process large amounts of data, accelerating the training and inference of neural networks. In scientific computing and data analysis, GPUs are used to accelerate complex numerical simulations and data analysis tasks. In cloud computing and data centers, GPUs provide high-performance computing resources to support various computationally intensive applications. In professional applications such as video editing, 3D modeling, and scientific visualization, GPUs provide real-time, high-performance rendering.

[0067] Compute Unit (CU): In GPU architecture, the compute unit is the basic unit for executing computational tasks. A workgroup running on a GPU actually executes within a compute unit. The compute unit is the fundamental unit on the GPU used to execute parallel computing tasks. It contains a number of processing elements that can execute multiple threads in parallel. The compute unit works by first scheduling the next thread bundle (a set of 32 threads) command from the input queue. The thread bundle scheduler then dispatches the thread bundle command to the appropriate functional unit. The functional unit executes the thread bundle command and generates output data, which is then written back to the thread bundle scheduler. The thread bundle scheduler checks if the thread bundle has completed execution; if so, it sends the thread bundle end data to the output queue. Furthermore, the compute unit typically contains its own resources, such as register files and shared memory, which can be shared by threads within the same compute unit. A compute unit can handle multiple workgroups simultaneously. Each workgroup contains a number of work items that execute in parallel within the compute unit. Properly designing the size and configuration of the workgroups can optimize the resource utilization of the compute unit and improve program performance and efficiency.

[0068] Cluster shared memory: Cluster shared memory refers to a high-speed, programmable cache consisting of static random access memory within each streaming multiprocessor. Its core mission is to provide an ultra-low latency data sharing and communication platform for all threads within a block of threads that are executing collaboratively on that processor. This block of memory has a very small physical size, is located on the chip, and has access speeds close to those of registers, but its capacity is limited.

[0069] Static Random Access Memory (SRAM): SRAM is the core microarchitectural unit designed to achieve high-concurrency access. It physically divides the address space of the cluster's shared memory into multiple (usually 32, matching the thread bundle size) storage modules that can be operated independently and in parallel. Each SRAM has its own dedicated data read and write channel. Ideally, when 32 threads in a thread bundle initiate access requests simultaneously, if the data of these requests is evenly distributed in the 32 different SRAMs, all requests can be satisfied simultaneously within one clock cycle, thereby achieving the theoretical peak bandwidth.

[0070] Crossbar network: A crossbar network is a high-performance hardware interconnect architecture. Its core is a topology in which N input ports and N output ports are directly connected through a switch matrix. It can establish a non-blocking, low-latency dedicated communication channel between any input and output ports, thereby realizing the parallel transmission of multiple sets of data. However, because the complexity of physical wiring increases quadratically with the number of ports, it is usually used inside chips.

[0071] Execution unit (EU): The execution unit is the smallest programmable execution core in the GPU. It is a multi-threaded single instruction multiple data processor responsible for executing general-purpose computational instructions such as integer, floating-point, and logic. It is a complete processor core that includes instruction pipeline, register file and multi-threaded context.

[0072] Copy Engine (CE): The main purpose of a copy engine is to improve the efficiency of moving data between different storage tiers, especially in scenarios where data needs to be frequently copied between global memory and local memory. A copy engine is an engine that optimizes data copying operations at the workgroup level. It improves data transfer efficiency and overall computing performance through asynchronous data transfer and intelligent data movement strategies.

[0073] Tensor Processing Engine (TPE): A Tensor Processing Engine is a collective term for a class of technologies designed specifically for efficient tensor operations. It can be a dedicated processor that achieves disruptive performance improvements through customized hardware architecture; or a compiler and runtime system that enables high-performance machine learning inference on general-purpose hardware through software stack optimization. Both share the common goal of addressing the ever-increasing demand for massive tensor operations in artificial intelligence computing, representing important practices of the "domain-specific computing" concept at different levels.

[0074] In existing technologies, fine-grained cross-connect networks can be relied upon for CE, TPE, and EU to access cluster shared memory. However, cluster shared memory consists of multiple static random access memory (SRAM) banks, and the data width of the fine-grained cross-connect network is the same as the port width of the SRAM bank. As a result, the fine-grained cross-connect network requires a large silicon area to meet the CU's requirement to access any SRAM bank, which leads to an increase in chip manufacturing costs.

[0075] Based on this, this disclosure proposes a chip system, data processing method, electronic device, and computer-readable storage medium that can save the silicon area occupied by cross-connect network, thereby significantly reducing chip manufacturing costs.

[0076] System architecture description used in the embodiments of this disclosure:

[0077] Figure 1This is an architecture diagram of a chip system according to an embodiment of the present disclosure. The chip system includes a clustered shared memory 100, a crossbar network 200, and multiple computing units 300. The chip system may also include a data merging unit. The clustered shared memory 100 includes multiple storage banks 110, and the crossbar network 200 is coupled between the multiple computing units 300 and the multiple storage banks 110. The data merging unit may be located inside or coupled to the computing units 300. The data merging unit may receive memory access requests initiated by the computing units 300 and perform data merging processing on the received memory access requests to obtain a memory access request data set. The size of the memory access request data set is an integer multiple of the data bus width of the crossbar network 200, and the memory access request data set can be transmitted between the computing units 300 and the clustered shared memory 100 through the crossbar network 200.

[0078] It is worth noting that each computing unit 300 communicates with each storage bank 110 in the cluster shared memory 100 through the crossbar network 200; Figure 1 This is merely an example of disclosing a chip system including four computing units 300, but it does not mean that a chip system can only include four computing units 300. The number of computing units 300 can be five, six, eight, etc., and the number of computing units 300 is not limited here. Furthermore, Figure 1 This document merely discloses, by way of example, that the cluster shared memory 100 includes two storage units 110, but does not mean that a cluster shared memory 100 includes only two storage units 110. The number of storage units 110 can be three, five, seven, etc., and the number of storage units 110 is not limited here.

[0079] In this embodiment, the chip system includes a data merging unit disposed within or coupled to the computing unit 300. The data merging unit can receive and process memory access requests initiated by the computing unit 300, and can merge the received memory access requests to obtain a memory access request data set. The size of the merged memory access request data set is an integer multiple of the data bus width of the crossbar switch network 200. Therefore, the crossbar switch network 200 can be used to transmit the merged memory access request data set to the cluster shared memory 100, providing a basis for subsequent data reading. Because the data merging unit can merge the received memory access requests, and the size of the merged memory access request data set is an integer multiple of the data bus width of the crossbar switch network 200, the memory access request data set can be transmitted as a whole, eliminating the need to transmit individual memory access requests as in the past. This significantly saves silicon area occupied by the crossbar array and reduces chip manufacturing costs.

[0080] It is worth noting that the storage unit 110 in this embodiment can be a static random access memory (SRAM); wherein, the static random access memory is the core microarchitecture unit designed to achieve high-concurrency access, which physically divides the address space of the cluster shared memory 100 into multiple storage modules that can be operated independently in parallel.

[0081] In some embodiments of this disclosure, data in the cluster shared memory 100 is organized and addressed in blocks, and the size of each block is an integer multiple of the data bus width of the crossbar switch network 200. All data within the same block can be accessed simultaneously. Data in the cluster shared memory 100 is read in blocks, and the size of each block is an integer multiple of the data bus width of the crossbar switch network 200. This allows the crossbar switch network 200 to read and transfer entire data blocks in the cluster shared memory 100, unlike previous methods that only read and transfer data from a single storage unit 110. This ensures that all data within the same data block can be accessed simultaneously.

[0082] It is worth noting that the size of the data block in the cluster shared memory 100 is an integer multiple of the data bus width of the crossbar switch network 200, so that the entire data block in the cluster shared memory 100 can be transmitted to the corresponding computing unit 300 using the crossbar switch network 200, so that the data in the entire data block can be accessed simultaneously.

[0083] In some embodiments of this disclosure, the cluster shared memory 100 includes multiple storage banks 110, and these multiple storage banks 110 can form storage bank groups. Each storage bank group can store and process a data block, and the size of a data block is an integer multiple of the data bus width of the crossbar switch network 200. All data within the same data block can be accessed simultaneously. Therefore, reading the entire data block is equivalent to reading and processing all data in the storage bank group of the cluster shared memory 100. Then, the crossbar switch network 200 is used to transmit the data blocks stored in the storage bank group. The cluster shared memory 100 includes multiple storage banks 110, and a certain number of storage banks 110 can be configured to form a storage bank group. For example, four storage banks 110 can be configured to form a storage bank group, or eight storage banks 110 can be configured to form a storage bank group; this is not limited here.

[0084] In some embodiments of this disclosure, the data bus width of the crossbar network 200 is greater than the port width of a single memory bank 110, so that all data stored in a single memory bank 110 can be transmitted and processed through the data bus of the crossbar network 200, thus ensuring the stability and reliability of data transmission. Furthermore, the number of crossbar points in the crossbar network 200 is less than the product of the number of computing units 300 and the number of memory banks 110. Since the crossbar network 200 reads data from the cluster shared memory 100, it reads and transmits all data from multiple memory banks 110 simultaneously, reading the data in the cluster shared memory 100 in data blocks, with each data block size being an integer multiple of the data bus width of the crossbar network 200. This eliminates the need for precise connections and data transmission between individual memory banks 110 and individual computing units 300, thus reducing the number of crossbar points in the crossbar network 200 to less than the product of the number of computing units 300 and the number of memory banks 110. This effectively saves silicon area occupied by the crossbar array and significantly reduces chip manufacturing costs.

[0085] In some embodiments of this disclosure, each computing unit 300 includes multiple execution units, and the data merging unit includes a first merging buffer. The first merging buffer can be used to receive memory access requests initiated by multiple execution units and perform data merging processing on the received memory access requests. When an execution unit in computing unit 300 needs to read data from the cluster shared memory 100, the execution unit can initiate a memory access request. Then, the first merging buffer can perform data merging processing on the memory access requests initiated by the execution unit, so that the size of the merged memory access request data set is an integer multiple of the data bus width of the crossbar network 200, so that the crossbar network 200 can be used for subsequent transmission processing of the entire memory access request data set.

[0086] In some embodiments of this disclosure, each computing unit 300 includes a tensor processing engine, which in turn includes multiple tensor processing submodules. These submodules can be tensor processing systolic arrays or addition trees. The data merging unit includes a first merging register, which can be used to merge memory access requests sent by multiple tensor processing submodules. When a tensor processing submodule in computing unit 300 needs to read data from the cluster shared memory 100, it can initiate a memory access request. The first merging register then merges the memory access request data, ensuring that the size of the merged memory access request data set is an integer multiple of the data bus width of the crossbar network 200. This allows the crossbar network 200 to be used for subsequent transmission and processing of the entire memory access request data set.

[0087] In some embodiments of this disclosure, each computing unit 300 includes a replication engine, which in turn includes multiple replication submodules; and the data merging unit includes a second merging buffer, which can be used to merge memory access requests sent by multiple replication submodules. When a replication submodule in computing unit 300 needs to read data from the cluster shared memory 100, the replication submodule can initiate a memory access request. Then, the second merging buffer can merge the memory access request initiated by the replication submodule so that the size of the merged memory access request data set is an integer multiple of the data bus width of the crossbar network 200, so that the crossbar network 200 can be used for subsequent transmission processing of the entire memory access request data set.

[0088] Reference Figure 2 , Figure 2 An exemplary schematic diagram of the prior art is disclosed, showing a structure in which a computing unit 300 and a cluster shared memory 100 are connected via a crossbar network 200. Figure 2 The crossbar network 200 has 32 ports connected to the computing unit 300, and the cluster shared memory 100 has 32 storage banks 110. The number of ports of the crossbar network 200 connected to the storage banks 110 in the cluster shared memory 100 is also 32. In order to enable any computing unit 300 to read data from any storage bank 110 in the cluster shared memory 100, the crossbar network 200 needs to perform fine-grained data transmission operations. Since the data width of each data line of the crossbar network 200 and the port width of the storage bank 110 are both 512 bits, the silicon area occupied by the crossbar network 200 can be measured by 32*32*512 bits.

[0089] Reference Figure 3 , Figure 3 This invention provides an exemplary schematic diagram of a structure that connects the computing unit 300 and the cluster shared memory 100 via the cross switch network 200, thereby solving the problem of the large silicon wafer area occupied by the cross switch network 200 using the technical solution of the present invention. Figure 3 In this configuration, four memory banks 110 are grouped into a memory bank group, resulting in eight memory bank groups within the cluster shared memory 100. The data size of each memory bank group is equal to four times the bus width of the crossbar network 200. Since the width of each data bus of the crossbar network 200 and the port width of the memory bank 110 are both 512 bits, the silicon area occupied by the crossbar network 200 can be measured using 8*8*2048 bits. This effectively saves silicon area occupied by the crossbar network 200, significantly reduces chip manufacturing costs, and simplifies the manufacturing process.

[0090] It is worth noting that, Figure 3 The above is merely an example of arranging four memory banks 110 into a memory bank group. It does not mean that only four memory banks 110 can be arranged into a memory bank group. Two, eight, or sixteen memory banks 110 can also be arranged into a memory bank group. This is not limited here.

[0091] It is worth noting that the data size of each memory bank group can be an integer multiple of the data bus width of the crossbar switch network 200; for example, when the memory bank group consists of 8 memory banks 110, the data size of the memory bank group is 8 times the data bus width of the crossbar switch network 200.

[0092] It is worth noting that the bus based on the crossbar switch network 200 can read the corresponding storage bank group from the cluster shared memory 100. Using the storage bank group as the unit of data reading, that is, all data in multiple storage banks 110 is read and processed in parallel, which improves the efficiency of data reading.

[0093] It is worth noting that the chip system in this embodiment can be a GPU; wherein, a GPU is a microprocessor specifically designed for processing graphics and image calculations. It was originally designed to accelerate the rendering of computer graphics, but with the development of technology, the application of GPUs has far exceeded this scope.

[0094] Overall implementation of the data processing method of this disclosure:

[0095] This disclosure provides a data processing method applied to the chip system described in the above embodiments. (Refer to...) Figure 4 Data processing methods include:

[0096] Step S201: Receive memory access requests initiated by the corresponding computing unit through the data merging unit;

[0097] Step S202: The memory access request is processed by data merging unit to obtain memory access request data set, wherein the size of memory access request data set is an integer multiple of the data bus width of crossbar switch network;

[0098] Step S203: The set of memory access request data is transmitted from the corresponding computing unit to the cluster shared memory through the cross-connect network in order to read data from the cluster shared memory.

[0099] Specifically, in this disclosure, the data processing method proposed in the embodiments of this disclosure is applied to a chip system. The chip system includes a clustered shared memory, a crossbar network, and multiple computing units. The chip system may also include a data merging unit. The clustered shared memory includes multiple storage banks, and the crossbar network is coupled between the multiple computing units and the multiple storage banks. The data merging unit may be located inside the computing units or coupled to the computing units. The data merging unit can receive memory access requests initiated by the computing units and perform data merging processing on the received memory access requests to obtain a memory access request data set. The size of the memory access request data set is an integer multiple of the data bus width of the crossbar network, and the memory access request data set can be transmitted between the computing units and the clustered shared memory through the crossbar network.

[0100] In step S201, each submodule in the computing unit generates a memory access request when it needs to access the cluster shared memory. After a memory access request is generated in the computing unit, the data merging unit in the computing unit can perform receive and buffer processing on the memory access request generated by the computing unit to prepare for subsequent data reading. For example, a chip system includes four computing units and cluster shared memory. Each computing unit is internally equipped with a data merging unit. The four computing units are a first computing unit, a second computing unit, a third computing unit, and a fourth computing unit. During a data processing process, the first computing unit needs to access the cluster shared memory, which generates a memory access request. The data merging unit in the first computing unit also needs to perform receive and buffer processing on the memory access request generated by the first computing unit to prepare for subsequent data access.

[0101] It is worth noting that the computing unit includes a replication engine, a tensor processing engine, and multiple execution units. When the replication engine, tensor processing engine, and execution units need to access the cluster's shared memory, they will generate corresponding memory access requests. For example, when the tensor processing engine of a computing unit needs to access the cluster's shared memory, the tensor processing engine will generate a corresponding memory access request. At this time, the data merging unit in the computing unit can receive the received memory access request and perform data merging processing to prepare for subsequent data reading. Alternatively, when the replication engine of a computing unit needs to access the cluster's shared memory, the replication engine will generate a corresponding memory access request. At this time, the data merging unit in the computing unit can also receive the received memory access request and perform data merging processing to prepare for subsequent data reading.

[0102] In step S202, after the data merging unit receives and buffers the memory access requests generated by the computing unit due to accessing the cluster's shared memory, the different received memory access requests can be merged to obtain a memory access request data set. The size of the merged memory access request data set is an integer multiple of the data bus width of the crossbar network, so that the crossbar network can be used for subsequent data transmission processing. For example, a chip system includes two computing units, namely a first computing unit and a second computing unit. The first computing unit is equipped with a first execution unit and a second execution unit. During a data access process, both the first execution unit and the second execution unit need to read data from the cluster shared memory. Since the width of the data bus of the crossbar network is 256 bytes, the first execution unit generates a first memory access request because it needs to access the cluster shared memory, and the data size of the first memory access request is 128 bytes. At this time, the first memory access request is first cached in the data merging unit. Then, the second execution unit generates a second memory access request because it needs to access the cluster shared memory, and the data size of the second memory access request is also 128 bytes. After receiving the second memory access request, the data merging unit, which caches the first memory access request, can obtain a 256-byte memory access request data set. Then, based on the memory access request data set obtained by merging the first memory access request and the second memory access request, the crossbar network can be used to perform corresponding data reading processing. Alternatively, the second computing unit may contain a third and a fourth execution unit. During a data access process, both the third and fourth execution units need to read data from the cluster's shared memory. The third execution unit generates a third memory access request because it needs to access the cluster's shared memory, and the size of the third memory access request is 280 bytes. At this time, the third memory access request is first cached in the data merging unit. Then, the fourth execution unit generates a fourth memory access request because it needs to access the cluster's shared memory, and the size of the fourth memory access request is also 232 bytes. After receiving the fourth memory access request, the data merging unit, which has cached the third memory access request, can obtain a 512-byte set of memory access request data. Since the size of the 512-byte set of memory access request data is twice the width of the crossbar network's data bus, the crossbar network can also be used to perform corresponding data reading and processing based on the 512-byte set of memory access request data.

[0103] It is worth noting that the size of the memory access request data set obtained by caching and merging in the data merging unit needs to be an integer multiple of the width of the crossbar network's data bus. This allows for better utilization of the crossbar network for data transmission processing. For example, if the size of the memory access request data set obtained by caching and merging in the data merging unit is 256 bytes, and the width of the crossbar network's data bus is also 256 bytes, the memory access request data set generated on the computing unit side can be directly transmitted using the crossbar network's bus, and the crossbar network can well meet the data transmission requirements. Alternatively, if the size of the memory access request data set obtained by caching and merging in the data merging unit is 512 bytes, and the width of the crossbar network's data bus is 256 bytes, then the memory access request data set can be transmitted twice using the crossbar network's data bus, which also meets the data transmission requirements.

[0104] It is worth noting that each data bus in the crossbar network has the same width. One side of the crossbar network is connected to multiple computing units through multiple data buses, and the other side of the crossbar network is connected to multiple storage banks in the cluster shared memory through multiple data buses. Each data bus corresponds to one storage bank in the cluster shared memory. Thus, the crossbar network can receive the memory access request data set merged in the data merging unit through the data bus, and then perform data reading operations from the corresponding storage bank group according to the memory access request data set.

[0105] In step S203, after the memory access request data set is merged in the data merging unit, it can be transmitted from the corresponding computing unit to the cluster shared memory via a crossbar network. Then, based on the memory access request data set, the corresponding data can be read from the cluster shared memory via the data bus of the crossbar network, and the read data can be transmitted to the computing unit that initiated the memory access via the crossbar network, preparing for subsequent data operation processing. Specifically, the address resolution unit in the chip system parses the memory access request data set to obtain the corresponding data read address information, which allows the corresponding data to be read from the cluster shared memory.

[0106] For example, the cluster shared memory includes four memory banks: a first memory bank, a second memory bank, a third memory bank, and a fourth memory bank. The address of the first memory bank is marked as 001, the address of the second memory bank is marked as 002, the address of the third memory bank is marked as 003, and the address of the fourth memory bank is marked as 004. During a data operation, the address resolution unit of the chip system parses the memory access request data set obtained by caching and merging in the data merging unit, and the data read address information 003 can be obtained. Subsequently, the crossbar switch network can be used to read data from the third memory bank in the cluster shared memory, preparing for subsequent data operations.

[0107] It is worth noting that after parsing the data read address information from the memory access request data set, the crossbar network can connect the relevant internal data transmission channels according to the data read address information. This allows the data of the corresponding memory bank group to be directly transmitted to the corresponding computing unit through the transmission channels connected within the crossbar network, making the data transmission process simpler, faster, and more reliable.

[0108] In steps S201 to S203, a memory access request is generated when the computing unit needs to access the cluster shared memory. Each submodule within the computing unit generates a corresponding memory access request when it needs to access the cluster shared memory. After a memory access request is generated in the computing unit, the data merging unit within the computing unit can receive and cache the memory access request. After the data merging unit receives and caches the memory access requests generated by the computing unit due to accessing the cluster shared memory, the different received memory access requests can be merged to obtain a memory access request data set. The size of the merged memory access request data set is an integer multiple of the width of the crossbar network's data bus. After caching the merged memory access request data set in the data merging unit, the memory access request data set can be transmitted from the corresponding computing unit to the cluster shared memory via the crossbar network. Then, based on the memory access request data set, the corresponding data can be read from the cluster shared memory via the crossbar network's data bus, and the read data can be transmitted to the computing unit that initiated the memory access via the crossbar network, preparing for subsequent data operation processing.

[0109] For example, a chip system includes four computing units: a first computing unit, a second computing unit, a third computing unit, and a fourth computing unit. Each computing unit has an internal replication engine, and each replication engine includes multiple replication sub-modules. During a data access process, the replication engine in the second computing unit needs to read data from the cluster shared memory. Since the data bus width of the crossbar network is 256 bytes, the first replication sub-module in the replication engine of the second computing unit needs to generate a first memory access request because it needs to access the cluster shared memory. The data size of the first memory access request is 212 bytes. At this time, the first memory access request is first cached in the data merging unit. Then, the second replication sub-module in the replication engine of the second computing unit generates a second memory access request because it needs to access the cluster shared memory. The data size of the second memory access request is also 300 bytes. After receiving the second memory access request, the data merging unit, which caches the first memory access request, can obtain a 512-byte set of memory access request data. The data bus width of the crossbar network is 256 bytes. Since the size of the 512-byte memory access request data set is twice the width of the crossbar network's data bus, it can be sent to the cluster shared memory via the crossbar network. The address resolution unit in the chip system parses the memory access request data set to obtain data read address information 003 and address information 004. The cluster shared memory includes four memory banks: the first, second, third, and fourth memory banks. The first bank's address is marked as 001, the second as 002, the third as 003, and the fourth as 004. Therefore, based on the data read address information 003 and address information 004, the crossbar network can be used to read data from the third and fourth memory banks respectively. The read third and fourth memory banks are then transmitted to the replication engine of the second computing unit, where the replication engine can perform subsequent data operations on them.

[0110] In one embodiment, refer to Figure 5 Each computing unit includes multiple execution units, and the data merging unit is a first merging buffer. Step S201 may also include:

[0111] Step S301: Receive memory access requests initiated by any execution unit in the corresponding computing unit through the first merging buffer.

[0112] In step S301, each computing unit of the chip system is provided with multiple execution units, and each computing unit is also provided with a first merging cache. When the execution unit of the computing unit needs to access the cluster shared memory, it will initiate a memory access request, and the first merging cache can receive and merge the memory access request generated by the execution unit. For example, a chip system includes two computing units and a cluster shared memory. The two computing units are a first computing unit and a second computing unit. Each of the first and second computing units has a first merging cache. The first computing unit has a first execution unit and a second execution unit. The second computing unit has a third execution unit and a fourth execution unit. During a data processing operation, the third and fourth execution units within the second computing unit need to access the cluster shared memory. When the third execution unit needs to access the cluster shared memory, it generates a first memory access request, which is first cached in the first merging cache. When the fourth execution unit needs to access the cluster shared memory, it generates a second memory access request, which is also cached in the first merging cache. Then, the size of the memory access request merged in the first merging cache is compared with the width of the data bus of the crossbar switch network. If the size of the memory access request merged in the first merging cache meets the requirements, data can be read from the cluster shared memory according to the memory access request merged in the first merging cache.

[0113] It is worth noting that each computing unit is equipped with a first merge cache, so that memory access requests generated by all execution units within the computing unit that initiate cluster shared memory access can be cached in the first merge cache, in preparation for subsequent data transmission.

[0114] It is worth noting that the first merging cache in this embodiment is a type of memory access optimization hardware in a chip system, mainly used to merge multiple small, scattered memory access requests into a memory access request data set, thereby improving the overall efficiency of the chip system. The first merging cache and the second merging cache in this embodiment have the same attributes and properties. The distinction between "first" and "second" is merely for better illustration of the technical solution of the embodiment and does not imply that they have different attributes and properties.

[0115] In one embodiment, refer to Figure 6 Step S202 may also include:

[0116] In step S401, after each data receiving cycle, the memory access requests sent by the execution unit are processed by merging data based on the first merging buffer according to the preset first byte throughput to obtain a memory access request data set.

[0117] In step S401, during the process of caching and merging memory access requests generated by the execution unit initiating cluster shared memory access using the first merging cache, it is necessary to use the first merging cache to perform data merging processing on the memory access requests sent by the execution unit according to the preset first byte throughput after each data receiving cycle, so as to obtain the memory access request data set.

[0118] It is worth noting that after each data reception cycle, the first merging buffer can receive and merge memory access requests according to a pre-set first byte throughput; for example, if the first byte throughput is 128 bytes, then after each data reception cycle, memory access requests can be received and cached in the first merging buffer at a throughput of 128 bytes. The first byte throughput can be set according to actual needs and is not limited here.

[0119] For example, a chip system includes two computing units, namely a first computing unit and a second computing unit. The first computing unit contains a first execution unit and a second execution unit, and the second computing unit contains a third execution unit and a fourth execution unit. During a data access process, both the first and second execution units need to read data from the cluster shared memory. Since the width of the data bus of the crossbar switch network is 256 bytes, the first execution unit generates a first memory access request because it needs to access the cluster shared memory. The data size of the first memory access request is 128 bytes, and the first byte throughput is also 128 bytes. After the first data reception cycle, the first memory access request is first cached in the first merge cache. The second execution unit generates a second memory access request because it needs to access the cluster's shared memory. The data size of the second memory access request is also 128 bytes. After the second data reception cycle, the first merge cache that caches the first memory access request will receive the second memory access request, thus obtaining a 256-byte memory access request data set. Then, based on the memory access request data set obtained by merging the first and second memory access requests, the corresponding data reading and processing can be performed using the crossbar switch network.

[0120] Reference Figure 7 , Figure 7A schematic diagram illustrating the execution unit sending memory access requests to the first merging buffer is disclosed. The throughput of the first merging buffer in each data reception cycle is 128 bytes. During cycle 0, a 128-byte memory access request is first cached in the first merging buffer; during cycle 1, another 128-byte memory access request is cached in the first merging buffer, thus forming a 256-byte set of memory access request data in the first merging buffer. With the crossbar switch network bus width also being 256 bytes, data reading processing can be performed based on the merged set of memory access request data.

[0121] In one embodiment, refer to Figure 8 Each computational unit includes a tensor processing engine, and the data merging unit is a first merging register. Step S201 may also include:

[0122] Step S501: Receive a memory access request initiated by the tensor processing engine in the corresponding computing unit through the first merging register.

[0123] In step S501, each computing unit of the chip system is equipped with a tensor processing engine, and each computing unit is also equipped with a first merging register. When the tensor processing engine of the computing unit needs to access the cluster shared memory, it will initiate a memory access request. The first merging register can receive and merge the memory access request generated by the tensor processing engine. For example, a chip system includes three computing units and cluster shared memory. The three computing units are a first computing unit, a second computing unit, and a third computing unit. Each of the first, second, and third computing units is equipped with a first merging register. The first computing unit is equipped with a first tensor processing engine, the second computing unit is equipped with a second tensor processing engine, and the third computing unit is equipped with a third tensor processing engine. During a data processing process, both the second tensor processing engine in the second computing unit and the third tensor processing engine in the third computing unit need to access the cluster shared memory. When the second tensor processing engine needs to access the cluster shared memory, it will generate a first memory access request, which will be cached in the first merging register of the second computing unit. When the third tensor processing engine needs to access the cluster shared memory, it will generate a first memory access request. A second memory access request is generated when the cluster's shared memory is accessed. This second memory access request is first cached in the first merging register of the third computing unit. Then, the size of the memory access requests cached in the first merging registers of both the second and third computing units is compared with the bus width of the crossbar network. If the size of the memory access request cached in the first merging register of the second computing unit meets the requirements, data can be read from the cluster's shared memory based on this cached request. Similarly, if the size of the memory access request cached in the first merging register of the third computing unit meets the requirements, data can be read from the cluster's shared memory based on this cached request. The memory access requests stored in the first merging registers of the second and third computing units are independent of each other; that is, memory access requests generated by the second tensor processing engine of the second computing unit are not cached in the first merging register of the third computing unit, nor are memory access requests generated by the third tensor processing engine of the third computing unit cached in the first merging register of the second computing unit.

[0124] It is worth noting that each computing unit is equipped with a first merging register, so that all memory access requests generated by cluster shared memory access initiated by all tensor processing engines within the computing unit can be cached in the corresponding first merging register, in preparation for subsequent data transmission.

[0125] It is worth noting that the first merging register in this embodiment is specifically optimized for the high-dimensional data access mode unique to tensor operations. When the tensor processing engine performs calculations such as matrix multiplication or convolution, the merging register will capture memory access requests from multiple parallel processing channels in real time and use its built-in stride access mode to identify the tensor processing engine, intelligently reorganizing the originally scattered matrix blocks or convolution kernel data accesses into continuous and aligned wide-range memory transactions.

[0126] In one embodiment, refer to Figure 9 Step S202 may also include:

[0127] Step S601: Based on the first merging register, the memory access request sent by the tensor processing submodule is processed by data merging to obtain a memory access request data set. The tensor processing engine includes multiple tensor processing submodules, which are tensor processing systolic arrays or addition trees.

[0128] In step S601, during the process of caching memory access requests generated by the tensor processing submodules initiating cluster shared memory access using the first merging register, the memory access requests sent by the tensor processing submodules are merged using the first merging register to obtain a set of memory access request data. A tensor processing engine may include multiple tensor processing submodules. When a tensor processing submodule initiates memory access, it generates a memory access request, which is stored in the first merging register of the corresponding computing unit. For example, a chip system includes two computing units: a first computing unit and a second computing unit. The first computing unit is equipped with a first tensor processing engine and a first merging register. The first tensor processing engine includes a first tensor processing submodule, a second tensor processing submodule, and a third tensor processing submodule. When the first tensor processing submodule initiates cluster shared memory access, a memory access request is generated and then cached in the first merging register. Similarly, when the second tensor processing submodule initiates cluster shared memory access, a memory access request is generated and also cached in the first merging register.

[0129] It is worth noting that the tensor processing submodule can be either a tensor processing systolic array or an addition tree. The tensor processing systolic array employs a spatial architecture, arranging a large number of processing units in a grid to form a data pipeline. Input data flows synchronously between rows and columns like a pulse wave, with multiplication and addition calculations performed between different units in each cycle, achieving highly parallel matrix multiplication and accumulation operations. The addition tree is a time-efficient reduction structure, typically organized as a binary tree, specifically responsible for quickly and with low latency accumulating and summing the numerous partial products output by the systolic array (such as calculation results from different channels or locations) to generate the final output element. The advantage of the tensor processing systolic array lies in maximizing the utilization of computational units and reducing data movement through regular data flow, making it particularly suitable for large-scale matrix multiplication. The addition tree optimizes the many-to-one reduction path, ensuring that high fan-in accumulation operations can still maintain a high clock frequency.

[0130] For example, a chip system includes two computing units, namely a first computing unit and a second computing unit; the first computing unit is equipped with a first tensor processing engine, and the second computing unit is equipped with a second tensor processing engine; the first computing unit is also equipped with a first merging register, and the second computing unit is also equipped with a second merging register; the first tensor processing engine includes a first tensor processing submodule and a second tensor processing submodule, and the second tensor processing engine includes a third tensor processing submodule and a fourth tensor processing submodule; during a data access process, both the first tensor processing submodule and the second tensor processing submodule need to read data from the cluster shared memory; since the width of the data bus of the crossbar switch network is 256 bytes, the first The tensor processing submodule generates a first memory access request due to the need to access the cluster's shared memory, and the data size of the first memory access request is 128 bytes. At this time, the first memory access request is first cached in the first merging register. The second tensor processing submodule generates a second memory access request due to the need to access the cluster's shared memory, and the data size of the second memory access request is also 128 bytes. The first merging register, which caches the first memory access request, will receive the second memory access request, thus obtaining a 256-byte memory access request data set. Then, based on the memory access request data set obtained by merging the first and second memory access requests, the corresponding data reading and processing can be performed using the crossbar switch network. Similarly, during a data access process, both the third and fourth tensor processing submodules need to read data from the cluster shared memory. Since the crossbar network's data bus width is 256 bytes, the third tensor processing submodule generates a third memory access request (64 bytes) to access the cluster shared memory. This request is first cached in the second merging register. The fourth tensor processing submodule generates a fourth memory access request (also 64 bytes), and the second merging register, which caches the third request, receives it, resulting in a 128-byte memory access request data set. However, because the crossbar network's data bus width is 256 bytes, data reading and processing cannot be performed immediately based on the third and fourth memory access requests. It is necessary to wait for either the third or fourth tensor processing submodule to initiate a new cluster shared memory access until the second merging register merges the resulting memory access request data set to reach 256 bytes.

[0131] Reference Figure 10 , Figure 10A schematic diagram is disclosed showing how a tensor processing systolic array or adder tree sends a memory access request to a merging register. In Group 1, the tensor processing systolic array or adder tree sends a 64-byte memory access request to the merging register each time. Since the bus width of the crossbar network is 256 bytes, only 4 sends are needed to obtain a satisfactory set of memory access request data in the merging register. Subsequent data reading and processing can then be performed based on this merged set of memory access request data. Similarly, in Group 2, the tensor processing systolic array or adder tree also sends a 64-byte memory access request to the merging register each time. Again, since the bus width of the crossbar network is 256 bytes, only 4 sends are needed to obtain a satisfactory set of memory access request data in the merging register. Subsequent data reading and processing can then be performed based on this merged set of memory access request data.

[0132] In one embodiment, refer to Figure 11 Each computing unit includes a replication engine, and the data merging unit is a second merging cache. Step S201 may also include:

[0133] Step S701: Receive memory access requests initiated by the replication engine in the corresponding computing unit through the second merging buffer.

[0134] In step S701, each computing unit of the chip system is equipped with a replication engine, and each computing unit is also equipped with a second merging cache. When the replication engine of the computing unit needs to access the cluster shared memory, it will initiate a memory access request, and the second merging cache can receive and merge the memory access request generated by the replication engine. For example, a chip system includes two computing units and a cluster shared memory. The two computing units are a first computing unit and a second computing unit. Each computing unit has a second merging cache internally. The first computing unit has a first replication engine internally, and the second computing unit has a second replication engine internally. During a data processing operation, the second replication engine in the second computing unit needs to access the cluster shared memory. When the second replication engine needs to access the cluster shared memory, it generates a first memory access request, which is first cached in the second merging cache. Then, the size of the memory access request cached in the second merging cache is compared with the width of the crossbar switch network bus. If the size of the memory access request cached in the second merging cache meets the requirements, data can be read from the cluster shared memory according to the memory access request cached in the second merging cache.

[0135] It is worth noting that each computing unit is equipped with a second merge cache, so that all memory access requests generated by the cluster shared memory access initiated by the replication engine within the computing unit can be cached in the second merge cache, in preparation for subsequent data transmission.

[0136] It is worth noting that the second merging cache in this embodiment is also a type of memory access optimization hardware in the chip system. It is also used to merge multiple small, scattered memory access requests into a memory access request data set, thereby improving the overall efficiency of the chip system.

[0137] In one embodiment, refer to Figure 12 Step S202 may also include:

[0138] Step S801: After each data reception cycle, the memory access requests sent by the replication submodule are processed by the second merging buffer according to the preset second byte throughput to obtain a set of memory access request data. The replication engine includes multiple replication submodules.

[0139] In step S801, during the process of caching and merging memory access requests generated by the replication submodule initiating cluster shared memory access using the second merging cache, it is necessary to use the second merging cache to process the memory access requests sent by the replication submodule according to the pre-set second byte throughput after each data reception cycle, so as to obtain the memory access request data set.

[0140] It is worth noting that after each data reception cycle, the second merging buffer can perform reception, caching, and merging processing on memory access requests according to a pre-set second byte throughput; for example, if the second byte throughput is 128 bytes, then after each data reception cycle, memory access requests can be received and cached into the second merging buffer at a throughput of 128 bytes. The second byte throughput can be set according to actual needs and is not limited here.

[0141] For example, a chip system includes two computing units, namely a first computing unit and a second computing unit. The first computing unit is equipped with a first replication engine, and the second computing unit is equipped with a second replication engine. The first replication engine includes a first replication submodule and a second replication submodule, and the second replication engine includes a third replication submodule and a fourth replication submodule. Both the first and second computing units are equipped with a second merging buffer. During a data access process, both the first and second replication submodules need to read data from the cluster shared memory. Since the width of the data bus of the crossbar switch network is 256 bytes, the first replication submodule generates a first memory access request because it needs to access the cluster shared memory. The data size is 128 bytes, and the second byte throughput is also 128 bytes. After the first data reception cycle, the first memory access request is first cached in the second merge cache of the first computing unit. The second replication submodule generates a second memory access request because it needs to access the cluster's shared memory, and the data size of the second memory access request is also 128 bytes. After the second data reception cycle, the second merge cache that caches the first memory access request will receive the second memory access request, thus obtaining a 256-byte memory access request data set. Then, based on the memory access request data set obtained by merging the first and second memory access requests, the corresponding data reading processing can be performed using the crossbar switch network.

[0142] Reference Figure 13 , Figure 13 A schematic diagram illustrating the replication engine sending memory access requests to the second merging buffer is disclosed. The throughput of the second merging buffer in each data reception cycle is 64 bytes. During cycle 0, a 64-byte memory access request is first cached in the second merging buffer; during cycle 1, another 64-byte memory access request is cached in the second merging buffer; during cycle 2, another 64-byte memory access request is cached in the second merging buffer; during cycle 3, another 64-byte memory access request is cached in the second merging buffer, thus forming a 256-byte set of memory access request data in the second merging buffer. With the crossbar network data bus width also being 256 bytes, data reading processing can be performed based on the merged set of memory access request data.

[0143] In one embodiment, refer to Figure 14 Step S203 may also include:

[0144] Step S901: Based on the address resolution unit in the chip system, the memory access request data set is parsed and processed to obtain the data read address information;

[0145] Step S902: Based on the data read address information, read the corresponding data from the cluster shared memory through the crossbar switch network and transfer it to the corresponding computing unit.

[0146] In step S901, when the memory access request data set is determined in the data merging unit, the memory access request data set can be parsed and processed by the address parsing unit in the chip system to obtain the corresponding data read address information. Subsequently, all data in the corresponding storage group can be read from the cluster shared memory according to the data read address information. Since the storage group is composed of multiple storage groups, the efficiency of data reading can be greatly improved.

[0147] It is worth noting that the storage group in this embodiment consists of multiple storage units. For example, the clustered shared memory includes 32 storage units, and four storage units are configured as one storage group, thus the clustered shared memory can be divided into eight storage groups. Each storage unit stores 64 bytes of data, so the size of one storage group is 256 bytes. The width of the crossbar network's data bus is 256 bytes, allowing the crossbar network to directly read all the data of a complete storage group from the clustered shared memory via the data bus. Each storage group in the clustered shared memory can be assigned a unique address marker. For example, the address marker for the first storage group is 001, the address marker for the second storage group is 002, the address marker for the third storage group is 003, and so on. Therefore, when the address resolution unit parses the memory access request data set to obtain data read address information 005, the crossbar network can be controlled to read all the data stored in the fifth storage group from the clustered shared memory.

[0148] In step S902, after parsing the data read address information, all data in the corresponding storage group can be read from the cluster shared memory and transferred to the corresponding computing unit via the crossbar network bus according to the data read address information. For example, when the execution unit of the computing unit initiates cluster shared memory access, after determining the data read address information, it can use the crossbar network to determine the corresponding storage group from the cluster shared memory based on the data read address information, and then transfer all data in the read storage group to the corresponding execution unit; or, when the replication engine of the computing unit initiates cluster shared memory access, after determining the data read address information, it can use the crossbar network to determine the corresponding storage group from the cluster shared memory based on the data read address information, and then transfer all data in the read storage group to the corresponding replication engine.

[0149] It is worth noting that after parsing the data read address information from the memory access request data set, the crossbar network can connect and build internal data transmission channels based on the data read address information. This allows data in the corresponding memory blocks to be directly transmitted to the corresponding computing units through the transmission channels built inside the crossbar network, making the data transmission process simpler, faster, and more reliable.

[0150] In steps S901 to S902, once the memory access request data set is determined in the data merging unit, the memory access request data set can be parsed and processed by the address parsing unit in the chip system to obtain the corresponding data read address information. After obtaining the data read address information, the corresponding memory bank can be read from the cluster shared memory and placed into the corresponding computing unit through the crossbar switch network bus. Through the above technical solution, the data reading process can be made simpler, more reliable, and more efficient.

[0151] For example, a chip system includes two computing units, namely a first computing unit and a second computing unit. The first computing unit is equipped with a tensor processing engine. During a data operation, the tensor processing engine needs to access the cluster shared memory. After the address resolution unit in the chip system parses the memory access request data set and obtains the data read address information 004, since the cluster shared memory includes four memory banks, namely the first memory bank, the second memory bank, the third memory bank, and the fourth memory bank, with the address of the first memory bank marked as 001, the address of the second memory bank marked as 002, the address of the third memory bank marked as 003, and the address of the fourth memory bank marked as 004, the crossbar switch network can then be used to read all the data in the fourth memory bank of the cluster shared memory into the tensor processing engine of the first computing unit. Then, the tensor processing engine in the first computing unit can split and process the received data.

[0152] In one embodiment, refer to Figure 15 The data processing method of this disclosure embodiment further includes:

[0153] Step S1001: After a preset first number of data receiving cycles, if the total data size of all memory access requests received through the first merging buffer does not reach the data size of the memory access request data set, then based on all received memory access requests, the corresponding data is read from the cluster shared memory into the corresponding computing unit through the cross-connect network.

[0154] In step S1001, during the process of caching and merging memory access requests generated by the execution unit initiating cluster shared memory access using the first merging buffer, it is necessary to use the first merging buffer to process the memory access requests sent by the execution unit according to a preset first byte throughput after each data reception cycle. However, after the preset first number of data reception cycles, if the total data size of all memory access requests received by the first merging buffer does not meet the data size requirement of the memory access request data set, it will not continue to wait for the next data reception cycle. Instead, it will directly read the corresponding data from the cluster shared memory and into the corresponding computing unit based on all currently received memory access requests through the crossbar network data bus. Through the above technical solution, if the total data size of all memory access requests received by the first merging buffer does not meet the data size requirement of the memory access request data set after the preset first number of data reception cycles, it is not necessary to wait again. This effectively prevents the data reading time from being too long, ensures the timeliness of data reading, and also effectively prevents the data waiting from entering an infinite loop, making data reading safer and more reliable.

[0155] It is worth noting that if the total size of all memory access requests received by the first merging buffer does not meet the requirement of the memory access request data set size, it will not wait for the next data reception cycle. Instead, it will directly read the corresponding data from the cluster shared memory and into the corresponding computing unit through the crossbar network data bus based on all currently received memory access requests. Although this does not ensure that data reading is always highly efficient, it can effectively maintain the timeliness and reliability of data reading. The first merging buffer can be set according to actual needs and is not limited here.

[0156] For example, a chip system includes two computing units, namely a first computing unit and a second computing unit. The first computing unit contains a first execution unit and a second execution unit, and the second computing unit contains a third execution unit and a fourth execution unit. During a data access process, both the first and second execution units need to read data from the cluster shared memory. Since the width of the data bus of the crossbar switch network is 256 bytes, the first execution unit generates a first memory access request because it needs to access the cluster shared memory. The data size of the first memory access request is 128 bytes, and the first byte throughput is also 128 bytes. After the first data reception cycle, the first memory access request is first cached in the first merge cache. The second execution unit generates a second memory access request because it needs to access the cluster's shared memory. The data size of the second memory access request is also 128 bytes. After the second data reception cycle, the first merge cache that caches the first memory access request will receive the second memory access request, thus obtaining a 256-byte memory access request data set. Then, based on the memory access request data set obtained by merging the first and second memory access requests, the corresponding data reading and processing can be performed using the crossbar switch network. However, the third execution unit generates a third memory access request due to the need to access the cluster's shared memory. The data size of the third memory access request is 32 bytes. After the first data reception cycle, the third memory access request is first cached in the first merge cache. The fourth execution unit generates a fourth memory access request due to the need to access the cluster's shared memory. The data size of the fourth memory access request is also 32 bytes. After the second data reception cycle, the first merge cache, which caches the third memory access request, receives the fourth memory access request. The first merge cache then merges the data into a 64-byte memory access request. After another 4 data reception cycles, neither the third nor the fourth execution unit issues any new access requests. At this point, 6 data reception cycles have passed, and the memory access requests merged in the first merge cache do not meet the data size requirement of the memory access request data set. Since the first number of attempts is set to 6, there will be no further waiting for data from the third and fourth execution units. Instead, data will be read directly from the cluster's shared memory using the crossbar switch network based on the 64-byte memory access request merged in the first merge cache, ensuring the timeliness and reliability of data reading.

[0157] In one embodiment, refer to Figure 16 The data processing method of this disclosure embodiment further includes:

[0158] In step S1101, after a preset second data reception cycle, if the total data size of all memory access requests received through the second merging buffer does not reach the data size of the memory access request data set, then based on all received memory access requests, the corresponding data is read from the cluster shared memory into the corresponding computing unit through the bus of the crossbar switch network.

[0159] In step S1101, during the caching and merging of memory access requests generated by the replication submodule's cluster shared memory access using the second merging buffer, it is necessary to use the second merging buffer to process the memory access requests sent by the execution unit according to a pre-set second byte throughput after each data reception cycle. However, after the pre-set second number of data reception cycles, if the total data size of all memory access requests received by the second merging buffer does not meet the data size requirement of the memory access request data set, it will not wait for the next data reception cycle. Instead, it will directly read the corresponding data from the cluster shared memory and into the corresponding computing unit based on all currently received memory access requests through the cross-connect network's data bus. Through the above technical solution, if the total data size of all memory access requests received by the second merging buffer does not meet the data size requirement of the memory access request data set after the pre-set second number of data reception cycles, it is not necessary to wait again. This effectively prevents excessively long data reading times, ensures timely data reading, and prevents data from entering an infinite loop, making data reading safer and more reliable. The second number can be set according to actual needs and is not limited here.

[0160] For example, a chip system includes two computing units, namely a first computing unit and a second computing unit. The first computing unit is equipped with a first replication engine, and the second computing unit is equipped with a second replication engine. The first replication engine includes a first replication submodule and a second replication submodule, and the second replication engine includes a third replication submodule and a fourth replication submodule. During a data access process, both the first replication submodule and the second replication submodule need to read data from the cluster shared memory. Since the bus width of the crossbar switch network is 256 bytes, the first replication submodule generates a first memory access request because it needs to access the cluster shared memory, and the data size of the first memory access request is 128 bytes. The first byte throughput is also 128 bytes. After the first data reception cycle, the first memory access request is first cached in the second merge cache. The second replication submodule generates a second memory access request because it needs to access the cluster's shared memory. The data size of the second memory access request is also 128 bytes. After the second data reception cycle, the second merge cache that caches the first memory access request will receive the second memory access request, thus obtaining a 256-byte memory access request data set. Then, based on the memory access request data set obtained by merging the first and second memory access requests, the corresponding data reading processing can be performed using the cross-connect network. However, the third replication submodule generates a third memory access request due to the need to access the cluster's shared memory. This third memory access request is 16 bytes in size. After the first data reception cycle, this request is cached in the second merge buffer. Similarly, the fourth replication submodule generates a fourth memory access request due to the need to access the cluster's shared memory. This fourth memory access request is also 16 bytes in size. After the second data reception cycle, the second merge buffer, which cached the third memory access request, receives the fourth memory access request. The second merge buffer then merges the requests into a 32-byte memory access request. After another 6 data reception cycles, neither the third nor the fourth replication submodule issues any new access requests. At this point, 8 data reception cycles have passed, and the merged memory access requests in the second merge buffer do not meet the size requirement of the memory access request data set. Since the second count is set to 8, subsequent data waiting for the third and fourth replication submodules will not be performed. Instead, data will be read directly from the cluster's shared memory using the crossbar switch network based on the 32-byte memory access request merged in the second merge buffer, ensuring both timeliness and reliability of data reading.

[0161] This disclosure also provides an electronic device 1700, comprising:

[0162] At least one processor, and,

[0163] A memory that is communicatively connected to at least one processor; wherein,

[0164] The memory stores instructions that are executed by at least one processor to cause the at least one processor to perform a data processing method as described in any of the above embodiments of the present disclosure when executing the instructions.

[0165] The following is combined Figure 17 The hardware structure of the electronic device is described in detail. The electronic device includes: a processor 1710, a memory 1720, an input / output interface 1730, a communication interface 1740, and a bus 1750.

[0166] The processor 1710 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure.

[0167] The memory 1720 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1720 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1720 and is called and executed by the processor 1710 using the data processing method of the embodiments of this disclosure.

[0168] The input / output interface 1730 is used to implement information input and output;

[0169] The communication interface 1740 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0170] Bus 1750 transmits information between various components of the device (e.g., processor 1710, memory 1720, input / output interface 1730, and communication interface 1740);

[0171] The processor 1710, memory 1720, input / output interface 1730 and communication interface 1740 are connected to each other within the device via bus 1750.

[0172] This disclosure also provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the data processing method of the above embodiments, which will not be described in detail here.

[0173] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0174] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0175] It should be understood that in the description of the embodiments of this disclosure, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0176] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0177] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0178] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0179] It should also be understood that the various implementation methods provided in this disclosure can be combined arbitrarily to achieve different technical effects.

[0180] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A chip system, characterized in that, include: Cluster shared memory, which includes multiple storage units; Multiple computing units; A cross-connect network is coupled between the plurality of said computing units and the plurality of said memory banks; A data merging unit, disposed within or coupled to the computing unit, is used to receive memory access requests initiated by the computing unit and to perform data merging processing on the memory access requests to obtain a memory access request data set. The size of the memory access request data set is an integer multiple of the data bus width of the crossbar network, and the memory access request data set is transmitted between the computing unit and the cluster shared memory through the crossbar network. The storage units are composed of multiple storage units, each storage unit stores a data block, the size of each data block is an integer multiple of the data bus width of the crossbar switch network, and all data in the same data block is accessed simultaneously. The data bus width of the crossbar network is greater than the port width of a single memory cell, and the number of crossbars in the crossbar network is less than the product of the number of computing units and the number of memory cells.

2. The chip system according to claim 1, characterized in that, Data in the cluster's shared memory is organized and addressed in blocks; the size of each block is an integer multiple of the width of the crossbar switch network's data bus, and all data within the same block is accessed simultaneously.

3. The chip system according to claim 1, characterized in that, Each computing unit includes multiple execution units. The data merging unit includes a first merging buffer, which is used to receive memory access requests initiated by multiple execution units and to perform data merging processing on the memory access requests.

4. The chip system according to claim 1, characterized in that, Each of the computing units includes a tensor processing engine, which includes multiple tensor processing sub-modules, wherein the tensor processing sub-modules are tensor processing systolic arrays or addition trees; The data merging unit includes a first merging register, which is used to merge the memory access requests sent by multiple tensor processing submodules.

5. The chip system according to claim 1, characterized in that, Each of the computing units includes a replication engine, which comprises multiple replication submodules. The data merging unit includes a second merging buffer, which is used to perform data merging processing on the memory access requests sent by multiple copying submodules.

6. A data processing method, characterized in that, The method is applied to a chip system, which includes a clustered shared memory, a crossbar network, a data merging unit, and multiple computing units. The clustered shared memory includes multiple memory banks. The crossbar network is coupled between the multiple computing units and the multiple memory banks. The data merging unit is disposed within or coupled to the computing unit. The data processing method includes: The data merging unit receives the memory access request initiated by the corresponding computing unit; The memory access request is merged based on the data merging unit to obtain a memory access request data set, wherein the size of the memory access request data set is an integer multiple of the data bus width of the crossbar switch network. The memory access request data set is transmitted from the corresponding computing unit to the cluster shared memory through the crossbar network, so as to read data from the cluster shared memory; The storage units are composed of multiple storage units, each storage unit stores a data block, the size of each data block is an integer multiple of the data bus width of the crossbar switch network, and all data in the same data block is accessed simultaneously. The data bus width of the crossbar network is greater than the port width of a single memory cell, and the number of crossbars in the crossbar network is less than the product of the number of computing units and the number of memory cells.

7. The data processing method according to claim 6, characterized in that, Each computing unit includes multiple execution units, and the data merging unit is a first merging cache. Receiving memory access requests initiated by the corresponding computing unit through the data merging unit includes: The first merging buffer receives the memory access request initiated by any of the execution units in the corresponding computing units.

8. The data processing method according to claim 7, characterized in that, The data merging unit performs data merging processing on the memory access requests to obtain a memory access request data set, including: After each data reception cycle, the memory access requests sent by the execution unit are processed by the first merging buffer according to a preset first byte throughput to obtain the memory access request data set.

9. The data processing method according to claim 6, characterized in that, Each of the computational units includes a tensor processing engine, and the data merging unit is a first merging register. Receiving memory access requests initiated by the corresponding computational unit through the data merging unit includes: The first merging register receives the memory access request initiated by the tensor processing engine in the corresponding computing unit.

10. The data processing method according to claim 9, characterized in that, The data merging unit performs data merging processing on the memory access requests to obtain a memory access request data set, including: The memory access request sent by the tensor processing submodule is processed by merging data based on the first merging register to obtain the memory access request data set. The tensor processing engine includes multiple tensor processing submodules, which are tensor processing systolic arrays or addition trees.

11. The data processing method according to claim 6, characterized in that, Each of the computing units includes a replication engine, and the data merging unit is a second merge cache. Receiving memory access requests initiated by the corresponding computing unit through the data merging unit includes: The second merging cache receives the memory access request initiated by the replication engine in the corresponding computing unit.

12. The data processing method according to claim 11, characterized in that, The data merging unit performs data merging processing on the memory access requests to obtain a memory access request data set, including: After each data reception cycle, the memory access requests sent by the replication submodule are processed by the second merging buffer according to a preset second byte throughput to obtain the memory access request data set. The replication engine includes multiple replication submodules.

13. The data processing method according to claim 6, characterized in that, The step of transmitting the memory access request data set from the corresponding computing unit to the cluster shared memory via the crossbar network, in order to read data from the cluster shared memory, includes: The address resolution unit in the chip system parses the memory access request data set to obtain data read address information. Based on the data read address information, the corresponding data is read from the cluster shared memory and into the corresponding computing unit through the crossbar switch network.

14. The data processing method according to claim 8, characterized in that, The data processing method further includes: After a preset first number of data reception cycles, if the total data size of all memory access requests received through the first merging buffer does not reach the data size of the memory access request data set, then based on all the received memory access requests, the corresponding data is read from the cluster shared memory and into the corresponding computing unit through the cross-connect network.

15. The data processing method according to claim 12, characterized in that, The data processing method further includes: After a preset second data reception cycle, if the total data size of all memory access requests received through the second merging buffer does not reach the data size of the memory access request data set, then based on all the received memory access requests, the corresponding data is read from the cluster shared memory and into the corresponding computing unit through the cross-connect network.

16. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for establishing communication between the processor and the memory. The program is executed by the processor to implement the data processing method as described in any one of claims 6 to 15.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the data processing method as described in any one of claims 6 to 15.

Images