Method for matrix data broadcasting in parallel processing

Abstract

The present invention relates to a computing system, and in particular to a method for broadcasting matrix data in parallel processing in a computing system. Systems, apparatuses, and methods are disclosed for efficiently executing multiple work units in a processor in parallel by reducing the number of memory accesses. A computing system includes a processor core having a parallel data architecture. One or more of a software application and firmware implement matrix operations and support broadcasting shared data to multiple compute units of the processor core. The application creates a thread group by matching compute kernels of the application with data items and grouping the resulting work units into a thread group. The application assigns the thread group to a compute unit based on detecting shared data between compute units. A single access request is generated instead of sending multiple read accesses to a memory subsystem to obtain the shared data. The single access request includes information identifying multiple compute units to receive the shared data when broadcast.

Description

Technical Field

[0001] This invention relates to a computing system, and more specifically, to a method for broadcasting matrix data in parallel processing within the computing system. Background Technology

[0002] Parallelization of tasks is used to increase the throughput of computing systems. To do this, compilers extract parallelized tasks from program code for parallel execution on system hardware. Processor cores consist of deep pipelines configured to execute multiple threads. To further increase parallel execution on hardware, multi-core architectures include multiple processor cores. Computing systems offload specific tasks to dedicated hardware, overcoming the performance limitations of conventional general-purpose cores. Some types of dedicated hardware include Single Instruction Multiple Data (SIMD) parallel architectures, others include Field Programmable Gate Arrays (FPGAs), and still others include other dedicated types of processing cores. When an architecture contains multiple cores of different types, it is called a heterogeneous multi-core architecture. For specific tasks such as graphics rendering, neural network training, and cryptography, heterogeneous multi-core architectures offer higher instruction throughput than homogeneous multi-core architectures.

[0003] Designers use a variety of parallel computing platforms and application programming interface (API) models to develop software applications for heterogeneous computing. Function calls within these platforms are called “computation kernels,” or simply “kernels.” Software, such as an operating system scheduler, matches these software kernels with one or more data records (e.g., data items) to produce one or more computational work units. Generally, SIMD architectures offer good computational performance and cost efficiency when performing such data-parallel workloads. However, performance degrades when storage bandwidth is limited and the application has high data reusability, leading to multiple memory accesses to the same data.

[0004] Given the above, it is desirable to have an efficient method to execute multiple work units in the processor in parallel by reducing the number of memory accesses. Summary of the Invention

[0005] In one aspect, the present invention provides an apparatus comprising: a first interface configured to communicate with a processor comprising a plurality of computing units, each computing unit configured to process instructions; and circuitry. The circuitry is configured to: receive, via the first interface, a first access request for shared data from one of the plurality of computing units; and, in response to determining that qualifying conditions are met, transmit a single access request for the shared data requested by the plurality of computing units.

[0006] In one embodiment, the qualifying condition is that the count of access requests for the shared data equals the number of computing units that require the shared data.

[0007] In another embodiment, the circuit is configured to: increment a count of access requests for each access request received from the processor for shared data; and determine that the qualifying condition is met in response to determining that the count is equal to the sum of a given computing unit and one or more other computing units.

[0008] In another embodiment, the circuit is configured to store broadcast characteristics for the shared data, including one or more of the following: a broadcast identifier, wherein the broadcast identifier is a unique identifier that identifies the shared data; and a base address, wherein the base address specifies a storage location for storing the beginning of the shared data.

[0009] In another embodiment, the circuit is configured to store broadcast characteristics for the shared data, including one or more of the following: a broadcast identifier that identifies the shared data; and a base address that specifies a storage location where the beginning of the shared data is stored.

[0010] In yet another embodiment, the circuit is configured to store in the broadcast feature a number equal to the sum of the given computing unit and the one or more other computing units.

[0011] In yet another embodiment, the circuit is configured to insert one or more of the following into the single access request: a broadcast identifier; and a mask identifying the given computing unit and the one or more other computing units.

[0012] In another embodiment, a cache controller is also included, the cache controller being configured to: receive the mask in the single access request; and include the broadcast identifier in the single access request.

[0013] In yet another embodiment, the cache controller is configured to: store the mask; and transmit the mask to the second processor during the broadcasting of the shared data to the plurality of computing units.

[0014] In another aspect, a method is provided, comprising: allocating a plurality of computing cores to a plurality of computing units of a processor via a system-on-a-chip (SoC), wherein a given computing core among the plurality of computing cores is configured to support a shared data broadcast protocol; in response to detecting an access request for the shared data from the given computing unit and determining that qualifying conditions are met, transmitting a single access request for the shared data to a memory subsystem via the SoC, the shared data being shared by the given computing unit and the one or more other computing units; and in response to receiving the shared data from the memory subsystem, broadcasting the shared data to the given computing unit and the one or more other computing units via the SoC.

[0015] In one implementation, the given computing kernel does not communicate with other computing kernels regarding the shared data.

[0016] In another embodiment, the qualifying condition is that the count of access requests for the shared data equals the number of computing units that require the shared data.

[0017] In another embodiment, the method further includes: incrementing the count of access requests in response to detecting an access request for the shared data.

[0018] In another embodiment, the method further includes: preventing the sending of access requests for the shared data in response to determining that the count of access requests for the shared data is not equal to the number of computing units that require the shared data.

[0019] In another embodiment, the shared data is further included as a broadcast feature, which includes one or more of the following: a broadcast identifier, wherein the broadcast identifier is a unique identifier that identifies the shared data; and a base address, wherein the base address specifies the storage location where the beginning of the shared data is stored.

[0020] In another aspect, a computing system is provided, comprising: a processing unit including a plurality of computing units; a memory subsystem coupled to the processing unit, wherein the memory subsystem is configured to store an operating system, applications, and user data; and wherein the operating system includes instructions executable by the processing unit to allocate a plurality of computing cores to the plurality of computing units, wherein a given computing core among the plurality of computing cores is configured to support a shared data broadcast protocol; wherein the applications include instructions executable by the processing unit to perform the following operations: in response to detecting an access request for shared data from the given computing unit and determining that a qualifying condition is met; transmitting a single access request for shared data to the memory subsystem, the shared data being shared by the given computing unit and the one or more other computing units; and in response to receiving the shared data from the memory subsystem, broadcasting the shared data to the given computing unit and the one or more other computing units.

[0021] In one implementation, the given computing kernel does not communicate with other computing kernels regarding the shared data.

[0022] In another embodiment, the qualifying condition is that the count of access requests for the shared data equals the number of computing units that require the shared data.

[0023] In another embodiment, the application includes instructions that can be further executed by the processing unit to store broadcast characteristics for the shared data, including one or more of the following: a broadcast identifier, wherein the broadcast identifier is a unique identifier that identifies the shared data; and a base address, wherein the base address specifies a storage location for storing the beginning of the shared data.

[0024] In yet another embodiment, the first access request from the given computing unit includes the broadcast identifier. Attached Figure Description

[0025] The advantages of the methods and mechanisms described herein can be better understood by referring to the following description in conjunction with the accompanying drawings, wherein: Figure 1 This is a block diagram of one implementation of a matrix.

[0026] Figure 2 This is a block diagram of one implementation of a matrix.

[0027] Figure 3 This is a block diagram of one implementation of a matrix.

[0028] Figure 4 This is a block diagram of one implementation of a matrix.

[0029] Figure 5 This is a block diagram of one implementation of a table used to support the broadcasting of shared data in a matrix.

[0030] Figure 6 This is a block diagram of one implementation of computer program code for supporting the broadcasting of matrix-shared data.

[0031] Figure 7 This is a flowchart of one implementation of a method for efficiently parallelizing multiple work units in a processor by reducing the number of memory accesses.

[0032] Figure 8 This is a flowchart of one implementation of the shared data broadcast protocol.

[0033] Figure 9 This is a block diagram of one implementation of a computing system.

[0034] While the invention is readily adaptable to various modifications and alternatives, specific embodiments are illustrated by way of example in the accompanying drawings, and are described in detail herein. However, it should be understood that the drawings and their detailed description are not intended to limit the invention to the specific forms disclosed; rather, the invention is intended to cover all modifications, equivalents, and alternatives falling within the scope of the invention as defined by the appended claims. Detailed Implementation

[0035] In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, those skilled in the art should recognize that various implementations can be practiced without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques have not been shown in detail to avoid obscuring the methods described herein. It should be understood that, for the sake of simplicity and clarity, the elements shown in the accompanying drawings are not necessarily drawn to scale. For example, the dimensions of some elements may be enlarged relative to other elements.

[0036] Various systems, apparatuses, methods, and computer-readable media are disclosed for efficiently parallelizing multiple work units in a processor by reducing the number of memory accesses. In various embodiments, the computing system includes memory for storing an operating system, software applications developed by the designer, and user data and result data of the software applications. The computing system also includes a processing unit that accesses memory and executes instructions from the operating system and the applications. The processing unit includes one or more processor cores having a single-instruction multiple-data (SIMD) parallel architecture. In one embodiment, an accelerated processing unit on a motherboard includes a SIMD processor core in addition to a general-purpose processor. In another embodiment, a video graphics card includes a SIMD processor core. One or more software applications include instructions for directing the SIMD core to perform generalized matrix multiplication (GEMM) operations, such as multiplying two input matrices together to generate a third output matrix.

[0037] In one implementation, GEMM operations are used in neural network applications. For example, this application includes algorithms for training deep neural networks. Another software application is a graphics shader program that instructs the SIMD processor core on how to render pixels to control lighting and shadow effects. Furthermore, this application includes pixel interpolation algorithms for geometric transformations. Pixel interpolation extracts new pixel values ​​at arbitrary coordinates from existing data. Other software applications also include instructions for instructing the SIMD core to perform GEMM operations for other scientific and commercial uses. These software application instructions operate on matrices with hundreds to thousands of rows and columns, resulting in millions to billions of floating-point operations (FLOPs).

[0038] In some implementations, designers use parallel computing platforms and corresponding application programming interface (API) models to develop software applications using GEMM operations. An example of a parallel computing platform is the OpenCL® (Open Computing Language) framework. The OpenCL framework (generally referred to herein as "OpenCL") includes a C-like language. For video graphics applications, an example of this language is GLSL (OpenGL Shading Language). Function calls in the C-like language are called OpenCL kernels, software kernels, compute kernels, or simply "kernels." The operating system includes instructions executable by a processor (e.g., a general-purpose processor) to distribute multiple compute kernels of the software application performing GEMM operations to multiple compute units of the SIMD processor core. In some implementations, the software application includes code segments that include instructions for supporting a shared data broadcast protocol, where shared data is broadcast to multiple compute units. In a shared data broadcast protocol, multiple compute units that need shared data generate requests for the data, but only generate a data fetch once. When the fetched data (e.g., from cache, memory, etc.) is returned, it is broadcast to the compute unit that requested the data. In this way, multiple fetch requests do not unnecessarily consume memory bus bandwidth. In various implementations, the computing kernels executing on the computing units of the SIMD processor core do not communicate with each other about shared data, and do not receive instructions from any other computing kernel specifying that the shared data is shared with any other computing unit.

[0039] In some implementations, one or more computing kernels include instructions for a broadcast feature for storing shared data. In other implementations, additional code includes these instructions. The feature includes a broadcast identifier, which is a unique identifier of the shared data, and a base address, which specifies the storage location at the beginning of the shared data storage. In some implementations, the feature also includes a summation of the total number of computing units using the shared data. In some implementations, program code other than the kernel itself includes instructions executable by processor circuitry to increment a count of computing units that sent the read access request for the shared data when an instruction detects a read access request. In some implementations, the computing kernel includes instructions for instructing a thread's current iteration to request the shared data, and other program code (e.g., code outside the kernel) increments the counter accordingly. When the count of computing units for the shared data equals the summation of the stored data, processor-executable instructions communicate a single access request for the shared data to the memory subsystem.

[0040] In other implementations, the instructions monitor the number of threads in the thread group requesting shared data among multiple computing units. This sum is the total number of threads from the multiple computing units. In one implementation, when the processor executes the instructions, the processor generates a single access request when the count equals the sum of 32 threads from each of the four computing units. Although in one implementation, the computing unit includes 64 parallel channels for executing 64 threads, the processor generates a single read access request after each of the four computing units generates one or more read access requests to provide data to 32 of the 64 channels. It is possible and predictable to determine when a single read access request will be generated from other channels and other portions of the total number of channels. In other implementations, the instructions monitor the amount of shared data requested by multiple read access requests from multiple computing units, rather than the number of computing units or the number of parallel channels within a computing unit.

[0041] In one implementation, the compute kernel includes instructions for inserting a mask that uses shared data to identify compute units within a single access request. In another implementation, the compute kernel includes instructions for generating a mask based on a broadcast identifier. In such an implementation, the single access request includes a broadcast identifier but not a mask. In other implementations, the single access request includes a mask but not a broadcast identifier. When the compute kernel determines that the memory subsystem has returned shared data, it broadcasts the shared data to the compute unit identified by the mask. Therefore, the performance degradation impact based on memory subsystem bandwidth limitations is reduced due to the single access request and the broadcast of the retrieved shared data.

[0042] Reference Figure 1An embodiment of matrix 100 used in the operation is illustrated. As shown, matrices A 110 and B 120 are input to matrix operation 130, which produces matrix C 150. Matrix operation 130 and matrices 110, 120, and 150 are used for training neural networks, pixel interpolation during video graphics rendering, or one of a variety of other scientific and commercial applications. In various embodiments, matrix operation 130 is a general matrix and matrix multiplication (GEMM) operation. For example, in some embodiments, matrix operation 130 is a multiplication and accumulation (MAC) operation for determining the dot product of two values ​​of two input matrices. In other embodiments, matrix operation 130 is another mathematical operation for generating a third matrix (such as matrix C 150) from two input matrices (such as matrix A 110 and matrix B 120). A computational unit 140 for allocating the input data values ​​of matrices 110 and 120 to processor cores is also shown, utilizing a single-instruction multiple-data (SIMD) parallel architecture. In one embodiment, the SIMD processor includes 16 compute units, each with 64 execution channels. In other embodiments, the SIMD processor core includes another number of compute units and another number of channels for each compute unit.

[0043] Matrix A 110 is shown as comprising 16 blocks labeled A32 to A47. In one embodiment, each of these 16 blocks comprises K data items, where K is a positive non-zero integer. In such an embodiment, matrix A 110 has a number of data items equal to 16 multiplied by K. Similarly, matrix B 120 is shown as comprising one block labeled B32. This single block has a number of data items equal to 1 multiplied by K, or K data items. The value K can be any number, so matrices 110 and 120 can be very large. For example, the number of data items K can range from tens to hundreds or even thousands. When the range reaches thousands, the number of operations in matrix operation 130 ranges from millions to billions. In other embodiments, the value K represents the amount of data, which comprises multiple data items. For example, if the value K represents 64 bytes, and the size of the data item is 32 bits (4 bytes), then the value K represents (64 / 4) data items or 16 data items.

[0044] Matrix C 150 is shown as comprising 16 x 1 blocks labeled (A32, B32) to (A47, B32). Each block of matrix C 150 comprises a combination of a block of matrix A 110 and a block B32 of matrix B 120, executed by matrix operation 130. Therefore, data items in matrices 110 and 120 are combined by matrix operation 130 in the order indicated by the block labels in matrix C 150. In some embodiments, the SIMD processor core executes the steps of matrix operation 130 in a concurrent and parallel manner based on instructions from a software application. In various embodiments, the computing system includes memory for storing a software application written by a software developer, as well as user data and result data of the software application. Here, user data is the data items of matrices A 110 and B 120, while result data is the data items of matrix C 150.

[0045] In one implementation, the SIMD processor core includes 16 compute units (CUs) as shown in allocation 140; however, in other implementations, the SIMD processor core includes a different number of compute units. Using allocation 140, compute unit 0 (CU0) of the SIMD processor core combines data in block A32 of matrix A 110 with data in block B32 of matrix B 120 using the steps of matrix operation 130. Software applications written by software developers include the steps of matrix operation 130. For example, the software application uses function calls with definitions defined in a specific library. Software developers use one of several types of parallel computing platforms and application programming interface (API) models to develop software applications. Function calls in these platforms are called “compute kernels,” or simply “kernels.”

[0046] In various implementations, the memory also stores instructions for the operating system. In some implementations, the operating system instructions are executed by a general-purpose processor core in the computing system. In one implementation, an application or firmware matches these software cores with one or more data records (e.g., data items) to create one or more computing work units. Work units are divided into multiple workgroups, also known as “thread groups.” Each thread group has a unique identifier (ID) assigned to it. Thread groups are not assigned to computing units by a scheduler, but rather the allocation is managed by the software application or firmware. For example, when a computing unit has 64 parallel execution channels, the application or firmware divides the work units into 64 groups, and assigns the thread groups of the 64 work units to specific computing units. The application or firmware performs the allocation shown in computing unit allocation 140 (or allocation 140).

[0047] In a similar manner, based on the application or firmware, computation unit 1 (CU1) of the SIMD processor core is assigned to combine data in block A33 of matrix A 110 with data in block B32 of matrix B 120 using matrix operation 130. Similarly, computation unit 15 (CU15) of the SIMD processor core uses matrix operation 130 to combine data in block A47 of matrix A 110 with data in block B32 of matrix B 120, and so on for computation units between CU1 and CU15. Based on assignment 140, the data from matrices 110 and 120 are loaded into computation units 0-15 of the SIMD processor core multiple times as needed. For example, if K represents 256 data items, each data item is 4 bytes in size, and each computing unit has 64 execution channels, each channel supporting operations on 4-byte data items, then the SIMD processor core uses (256 / 64) loads, or 4 loads, to fill the 64 parallel execution channels with data from blocks A32 and B32.

[0048] Generally, SIMD architectures offer good computational performance and cost efficiency when performing parallel data workloads. However, performance degrades when storage bandwidth is limited and the application has high data reuse, leading to multiple memory accesses to the same data. As shown in allocation 140, data reuse is typical in GEMM computations. For example, each computation unit CU0-CU15 loads and uses the same data from block B32 of matrix B120.

[0049] In the illustrated implementation, without modifying the software application, the software application generates 32 data extracts, with each computing unit having two data extracts. For example, computing unit 0 (CU0) uses two data extracts for data in blocks A32 and B32. Similarly, computing unit 8 (CU8) uses two data extracts for data in blocks A40 and the same block B32. However, if the software application is modified to include broadcast support for reused data, the software application will generate 17 data extracts instead of 32. For example, if the software application includes instructions to support only one data extract in block B32 and broadcast the acquired data to computing units 0-15, the software application generates 16 data extracts for data in blocks A32-A47 and a single data extract for data in block B32. The number of data extracts is reduced from 32 to 17.

[0050] In one example, one or more buses in the memory subsystem are smaller than the amount of data in block B32 of matrix B120. Therefore, for a “single” data fetch of data in block B32, multiple data fetches occur, and the bus is used multiple times while data is transferred within block B32. For example, when the width of the bus to the memory storing data in block B32 is 64 bytes, but the amount of data in block B32 is 512 bytes, a data fetch of block B32 will use eight separate data transfers on the bus. When data in block B32 is fetched multiple times, for example, 16 times for each of the sixteen compute units in one implementation, the resources of the memory subsystem are heavily loaded, and latency increases. Using the example above, without broadcast support, there are (16 x 8) or 128 data fetches for 512 bytes of block B32. Performance degrades with increasing latency. However, if the software application supports broadcasting data in block B32 and fetches that data only once, instead of fetching for each of the sixteen computation units, then there are only 8 data fetches for the 512 bytes of block B32 compared to 128 data fetches. Therefore, instead of loading the memory subsystem as before, latency is reduced and performance is improved.

[0051] Turn now Figure 2 An embodiment of matrix 200 used in operation is shown. The control logic described above is numbered the same. As shown, matrices A 210 and B 220 are input to matrix operation 130, which produces matrix C 250. Similar to matrix 100, matrix operation 130 and matrices 210, 220, and 250 are used for training neural networks, pixel interpolation in video graphics rendering processes, or one of a variety of other scientific and commercial applications. In various embodiments, matrix operation 130 is a general matrix and matrix multiplication (GEMM) operation. The allocation 240 of the input data values ​​of matrices 210 and 220 to the computational units of the SIMD processor core is also shown.

[0052] Matrix A 210 is shown as comprising eight blocks labeled A32 to A39. In one embodiment, each of these eight blocks comprises K data items. In such an embodiment, matrix A 210 has eight times the number of data items equal to K. Similarly, matrix B 220 is shown as comprising two blocks labeled B32 and B33. Matrix B 220 has twice the number of data items equal to K. As mentioned earlier, K is a positive non-zero integer. The value K can be any number, so matrices 210 and 220 can be very large. In other embodiments, the value K represents the amount of data, which comprises multiple data items. For example, if the value K represents 512 bytes, and the size of a data item is 64 bits (8 bytes), then the value K represents (512 / 8) data items or 64 data items.

[0053] The illustrated matrix C 250 comprises 8 x 2 blocks labeled (A32, B32) and (A32, B33) to (A39, B32) and (A39, B33). Each block of matrix C 250 comprises a combination of a block of matrix A 210 and a block of matrix B 220, executed via matrix operation 130. Therefore, data items in matrices 210 and 220 are combined via matrix operation 130 in the order displayed by block labeling in matrix C 250. In some embodiments, the SIMD processor core executes the steps of matrix operation 130 in a concurrent and parallel manner based on instructions from a software application.

[0054] In one embodiment, the SIMD processor core includes 16 compute units (CUs), as shown in compute unit allocation 240 (or allocation 240), although in other embodiments, the SIMD processor core includes a different number of compute units. Using allocation 240, compute unit 0 (CU0) of the SIMD processor core combines data in block A32 of matrix A 210 with data in block B32 of matrix B 220 using the steps of matrix operation 130. Compute unit 1 (CU1) of the SIMD processor core is allocated to combine data in block A33 of matrix A 210 with data in block B32 of matrix B 220 using the steps of matrix operation 130. Similarly, compute unit 15 (CU15 of the SIMD processor core) combines data in block A39 of matrix A 210 with data in block B33 of matrix B 220 using the steps of matrix operation 130, and so on for compute units between CU1 and CU15.

[0055] In some implementations, based on allocation 240, the data from matrices 210 and 220 is loaded multiple times as needed into computation units 0-15 of the SIMD processor core. For example, if K represents 64 data items, each with a size of 8 bytes, and each computation unit has 32 execution channels, each channel supporting operations on 8-byte data items, then the SIMD processor core uses (64 / 32) loading, or two loadings, to fill the 32 parallel execution channels with data from blocks A32 and B32. As shown in allocation 240, data reuse is typical in GEMM computations. For example, each computation unit CU0-CU7 loads and uses the same data from block B32 of matrix B 220. Similarly, each computation unit CU8-CU15 loads and uses the same data from block B33 of matrix B 220.

[0056] Without modifying the software application using matrix operation 130, in the illustrated embodiment, the circuitry of the processor executing the software application generates 32 data extractions, with 2 data extractions generated for each of the 16 computing units. For example, computing unit 0 (CU0) uses 2 data extractions for data in blocks A32 and B32. Similarly, computing unit 7 (CU7) uses 2 data extractions for data in blocks A39 and the same block B32. However, if the software application is modified to include broadcast support for reused data (shared data) in matrix B 220, the software application will generate 18 data extractions instead of 32. For example, if the software application includes instructions to support only one data extraction in block B32 and broadcast the extracted data to computing units 0-7, then for the leftmost column of matrix C 250, the software application generates 8 data extractions for data in blocks A32-A39 and a single data extraction for data in block B32. The number of data extractions in the leftmost column of matrix C250 was reduced from 16 to 9.

[0057] If the software application further includes instructions to support data extraction only once in box B33 and broadcast the extracted data to computation units 8-15, then for the rightmost column of matrix C250, the software application generates 8 data extractions for the data in blocks A32-A39 and a single data extraction for the data in block B33. The number of data extractions for the rightmost column of matrix C250 is reduced from 16 to 9. The total number of data extractions is reduced from (16 + 16) or 32 to (9 + 9) or 18. If the software application further includes instructions to support the extraction of shared data from blocks A32-A39 of matrix A210 in a single operation and to broadcast the extracted shared data to computation units 0-15, then the software application generates 8 data extractions for the data in blocks A32-A39 and performs a single data extraction for the data in blocks B32 and B33, thus providing (8 + 1 + 1) data extractions or 10 data extractions. As previously described with respect to matrix 100, when the software application supports data broadcasting, the data is extracted only once, and the memory subsystem does not load without broadcast support as before, thereby reducing latency and improving performance.

[0058] Now go to Figure 3An embodiment of matrix 300 used in operation is shown. The control logic described above is numbered the same. As shown, matrices A 310 and B 320 are input to matrix operation 130, which produces matrix C 350. Similar to matrices 100 and 200, matrix operation 130 and matrices 310, 320, and 350 are used for neural network training, pixel interpolation during video graphics rendering, or one of a variety of other scientific and commercial applications. In various embodiments, matrix operation 130 is a GEMM operation. The allocation 340 of the input data values ​​of matrices 310 and 320 to the computational units of the SIMD processor core is also shown.

[0059] Matrix A 310 is shown as comprising four blocks labeled A32 to A35. In one embodiment, each of the four blocks comprises K data items. In such an embodiment, matrix A 310 has a number of data items equal to four times K. Similarly, matrix B 320 is shown as comprising four blocks labeled B32 to B35. Matrix B 320 has a number of data items equal to four times K. As previously stated, K is a positive non-zero integer. The value K can be any number, so matrices 310 and 320 can be very large. In other embodiments, the value K represents the amount of data, which comprises multiple data items.

[0060] Matrix C 350 is shown as comprising 4 x 4 blocks. The top row of matrix C 350 comprises blocks labeled (A32, B32), (A32, B33), (A32, B34), and (A32, B35). Each block of matrix C 350 comprises a combination of a block of matrix A310 and a block of matrix B320, executed by matrix operation 130. Therefore, data items in matrices 310 and 320 are combined by matrix operation 130 in the order indicated by the block labels in matrix C 350. In some embodiments, the SIMD processor core executes the steps of matrix operation 130 in a concurrent and parallel manner based on instructions from a software application.

[0061] In one implementation, the SIMD processor core includes 16 compute units (CUs), as shown in compute unit allocation 340 (or allocation 340), but in other implementations, the SIMD processor core includes a different number of compute units. As shown in allocation 340, data reuse is typical in GEMM computation. For example, each compute unit CU0-CU3 loads and uses the same data from block B32 of matrix B320. Similarly, each compute unit CU4-CU7 loads and uses the same data from block B33 of matrix B320, and so on.

[0062] Without modifying the software application using matrix operation 130, in the illustrated embodiment, the software application generates 32 data extractions, with each computation unit performing two data extractions. For example, computation unit 0 (CU0) uses two data extractions for the data in blocks A32 and B32. Similarly, computation unit 3 (CU3) uses two data extractions for the data in blocks A35 and the same block B32. However, if the software application is modified to include broadcast support for reused data (shared data) in matrix B320, in one embodiment, the software application will generate 20 data extractions instead of 32. For example, if the software application includes instructions to support extracting shared data only once in block B32 and broadcasting the extracted shared data to computation units 0-3, then for the leftmost column of matrix C350, the software application generates 4 data extractions for the data in blocks A32-A35 and a single data extraction for the data in block B32. The number of data extractions for the leftmost column of matrix C250 is reduced from 8 to 5.

[0063] If the software application additionally includes instructions to support retrieving shared data from block B33 only once and broadcasting the retrieved data to computation units 4-7, then for the second leftmost column of matrix C350, the software application generates a total of 5 data retrievals for the data in blocks A32-A35 and the data in block B33. If the software application includes instructions to retrieve and broadcast shared data in blocks B34 and B35 in a similar manner to retrieving and broadcasting shared data in blocks B32 and B33, then the total number of retrievals is (4 x 5), or 20 data retrievals, instead of 32 data retrievals. If the software application also includes instructions that support extracting shared data from blocks A32-A35 of matrix A310 only once and broadcasting the extracted data to computation units 0-15, then the software application will generate 4 data extractions for the data in blocks A32-A35, a single data extraction for the data in block B32, and a single data extraction for the data in block B33. This will provide (4 + 1 + 1) or 6 data extractions for the leftmost two columns of matrix C350.

[0064] For the rightmost two columns of matrix C 350, two additional data extractions are performed to extract data from blocks B34 and B35. Therefore, when the software application includes instructions supporting broadcasting data from matrices A 310 and B 320, the total number of data extractions increases from 6 to 8 to generate matrix C 350 from matrices 310 and 320. If the software application includes instructions supporting broadcasting only data from matrix B 320, then the number of data extractions when generating matrix C 350 from matrices 310 and 320 decreases from 32 to 20. In either case, as previously described for matrices 100 and 200, when the software application supports data broadcasting, the memory subsystem will not load as it would without broadcast support, resulting in reduced latency and improved performance.

[0065] Now go to Figure 4 The diagram illustrates one implementation of matrix 400 used in operation. The control logic and data values ​​described earlier are numbered the same. The resulting matrix C 350 and input matrices 310 and 320 are shown again. To reduce memory accesses, data in the columns of matrix B 320 is associated with a broadcast identifier (ID). In one implementation, instructions from the software application assign broadcast identifier (ID) 0 to data in the leftmost column of matrix B 320, which is block B32. The thread group (TG) assigned to data in block B32 is also assigned to broadcast identifier 0. Thus, as shown in broadcast assignment 460, thread group 0-3 (TG 0-3) is assigned to broadcast identifier 0.

[0066] Thread groups 0-3 use data from block B32 of matrix B320 to generate the leftmost column of matrix C350. The assignment of broadcast identifier 0 instructs that the data extraction for block B32 should occur once, not four times. For example, data for a specific data item in block B32 should be extracted once, instead of each of the four logics in the four compute units 0-3 (CU 0-3) generating a read access request for that specific data item. When executing instructions from the GEMM software application, each of the four CU 0-3 still generates a read access request for the same data in block B32, but a read access request is not sent to the memory subsystem until each of the four read access requests has been generated. In another example, each compute unit CU8-CU11 generates one read access request for the same data in block B34. However, a read access request is not sent to the memory subsystem until each of the four read access requests has been generated.

[0067] Once each of the four read access requests has been generated, a single read access request is generated and sent to the memory subsystem. This single read access request includes an indication specifying that the read access request is associated with a broadcast of the requested data. In one embodiment, this indication is a bit storing an assertion value to indicate the broadcast of the requested data. The single read access request also includes an indication of which computing units receive the requested data during the broadcast. In some embodiments, the single read access request includes a bitmask specifying which computing units receive the requested data during the broadcast. In other embodiments, the single read access request includes a broadcast identifier, which is subsequently translated or decoded to determine which computing units receive the requested data during the broadcast. Thus, when data for block B32 is retrieved from the memory subsystem at one time, that data is simultaneously provided to computing units CU0-CU3. Similarly, when data for block B34 is retrieved from the memory subsystem at one time, that data is simultaneously provided to computing units CU8-CU11.

[0068] Although only the broadcast identifier for matrix B 320 is shown, it should be noted that in other embodiments, broadcast identifiers exist for blocks A32-A35 of matrix A 310. In one embodiment, the data allocated to block A32, as well as thread groups (TGs) 0, 4, 8, and 12 of computing units CU0, CU4, CU8, and CU12, are also assigned to broadcast identifier 4 (not shown). Similarly, the data allocated to block A33, as well as thread groups (TGs) 1, 5, 9, and 13 of computing units CU1, CU5, CU9, and CU13, are also assigned to broadcast identifier 5 (not shown). In such embodiments, the software application includes instructions for acquiring data for each thread group before executing any one of the thread groups. For example, when the broadcast identifier is used for both matrices 310 and 320, computing unit CU0 generates read access requests for both block A 32 and block B32. However, a single access request for block B32 is generated using broadcast identifier 0 only after read access requests from computing units C1-C3 have been generated. Similarly, a single access request for block A32 using broadcast identifier 4 (not shown) is not generated until read access requests from computing units C4, C8, and C12 are generated.

[0069] Turn now Figure 5This illustrates one implementation of table 500 for supporting data broadcasting for matrix operations. In some implementations, instructions from a software application, such as a GEMM software application, include instructions for setting up table 500. In one implementation, a specific computational kernel includes these instructions. In another implementation, the application includes table setup instructions prior to function calls used to define the computational kernel using GEMM operations. Table 500 includes fields 512-524 for storing information used during the broadcasting of shared data during matrix operations such as GEMM operations. Table 500 is stored in a group of registers, random access memory, content-addressable memory, or other locations. In some implementations, table 500 is set up as shown, with each entry assigned to specific shared data to be broadcast. In other implementations, each entry is a separate table. Table 500 provides a mapping (assignment) from thread groups to computational units.

[0070] Field 512 stores the broadcast identifier for shared data. (As previously stated...) Figure 4 As shown, each block B32-B35 of matrix B320 has a unique broadcast identifier that distinguishes the shared data of a particular block (e.g., B32) from the shared data of another block (e.g., B35). Field 514 stores an indication of the number of thread groups accessing the shared data identified by the corresponding broadcast identifier. In the earlier example shown for matrix B320, four of the sixteen computing units share the data for each block B32-B35. For example, computing units CU0-CU3 share the data for block B32, computing units CU4-CU7 share the data for block B33, and so on. Field 516 stores an indication of the iteration size, which is used to determine the amount of shared data to be extracted. For each individual data extraction of shared data, a specific amount of shared data is extracted or retrieved. The indication of the amount of shared data is specified as a portion of the total amount of shared data, the number of data items, the absolute amount of data, or something else. In some implementations, the amount of shared data to be extracted is specified in the original read access request for the shared data.

[0071] Field 518 stores the base address of shared data for a specific block, such as block B32 of matrix B320. This base address specifies the memory location where the beginning of the shared data is stored. As shown, the base address of the shared data corresponding to broadcast identifier 0 is 0x04DE_8F10. Here, the symbol "0x" represents a hexadecimal value. The base address is also a reference address. To obtain the absolute address of other data in the shared data, a specific shift (or offset) value is added to the base address. When the first read access request targets the first quarter of the shared data, the base address and a value indicating the size of the quarter of the shared data are used in the first read access request. When the second read access request targets the second quarter of the shared data, the size of the quarter of the shared data is added as a shift to the base address to provide the target address. Additionally, a value indicating the size of the quarter of the shared data is used in the second read access request.

[0072] In one implementation, instructions of a software application executing on the processor core also compare the target address of a read access request with the base address in field 518. When the target address is within the range of the base address in field 518, the corresponding read access request targets shared data, and this read access request should be converted into a single read access request that supports broadcasting of the shared data. Field 520 stores a count of the number of incomplete read access requests for the corresponding shared data. As previously described, field 514 stores an indication of the number of thread groups accessing the shared data identified by the corresponding broadcast identifier.

[0073] Whenever an instruction from the software application causes the logic in the processor core to detect an incomplete read access request for specific shared data, the corresponding count in field 520 is updated. In one embodiment, updating the value stored in field 520 includes incrementing the value. In such an embodiment, the initial value stored in field 520 is zero. In other embodiments, the initial value is equal to the value stored in field 514, and updating the value stored in field 520 includes decrementing the value. When the value stored in field 520 is not equal to a threshold, an instruction from the software application causes the logic in the processor core to prevent sending a read access request for the shared data. When updating the value stored in field 520 includes incrementing the value, the threshold is the value stored in field 514. When updating the value stored in field 520 includes decrementing the value, the threshold is zero.

[0074] In some implementations, when the value stored in field 520 equals a threshold, instructions from the software application cause logic in the processor core to generate a single read access request for broadcasting shared data. In other implementations, circuitry in the processor core determines that the value stored in field 520 equals a threshold and generates a single read access request without executing specific instructions instructing the circuitry to do so. As previously described, in one implementation, the single read access request includes bits storing an assertion value to indicate the broadcast of the requested data. The single read access request also includes an indication of which computing units receive the requested data during the broadcast. In some implementations, the single read access request includes a broadcast identifier stored in field 512, which is subsequently converted or decoded to determine which computing units receive the requested data during the broadcast. In other implementations, the single read access request includes a bitmask specifying which computing units receive the requested data during the broadcast.

[0075] Field 522 stores a bitmask specifying which computing units receive the requested data during the broadcast. As shown, the shared data corresponding to broadcast identifier 0 includes a bitmask of 0xF000, which specifies that computing units CU0-CU3 use the shared data. The shared data corresponding to broadcast identifier 1 includes a bitmask of 0x0F00, which specifies that computing units CU4-CU7 use the shared data, and so on. In embodiments where the broadcast identifier is decoded later to generate the bitmask, field 522 is not used in table 500. In some embodiments, table 500 includes field 524, which stores the size of the shared data. In this example, the size of the shared data is 2 kilobytes (KB). In another embodiment, field 524 includes the end address of the shared data, so the addresses in fields 518 and 524 provide the address range of the shared data. In other embodiments, logic uses the iteration size in field 516 and the base address in field 518 to determine the address range and / or size of the shared data.

[0076] Now go to Figure 6This illustrates one implementation of source code 600 that utilizes broadcast support for shared data. In various implementations, the computing system includes system memory for storing the operating system, software applications developed by the designers, and user and result data of the software applications. The computing system also includes a processing unit that accesses the system memory, stores information in a cache memory subsystem, and executes instructions from the operating system and applications. The processing unit includes one or more processor cores with a Single Instruction Multiple Data (SIMD) parallel architecture. In one implementation, an accelerated processing unit on the motherboard includes a SIMD processor core in addition to a general-purpose processor. In another implementation, a video graphics card includes a SIMD processor core. Code 600 represents an example of a software application stored in system memory. In some implementations, portions of Code 600 are also instructions for firmware stored in memory other than system memory or a cache memory subsystem (e.g., erasable read-only memory or other random access memory). Code 600 includes instructions for directing the SIMD processor core or other circuitry to perform GEMM operations. Code 600 also includes code segments 610, 620, and 630 for supporting broadcasting of shared data.

[0077] In some implementations, software developers use parallel computing platforms and corresponding application programming interface (API) models to develop software applications using GEMM operations. An example of a parallel computing platform is the OpenCL® (Open Computing Language) framework. The OpenCL framework improves the computational performance of a wide range of data-parallel applications used in gaming, entertainment, science, and healthcare. The OpenCL framework (generally referred to as "OpenCL" in this document) includes a C-like language. For video graphics applications, an example of this language is GLSL (OpenGL Shading Language). Function calls in the C-like language are called OpenCL kernels, software kernels, computation kernels, or simply "kernels."

[0078] In one implementation, code 600 is matched with data items to form work units and thread groups, the data items being assigned to computational units of the SIMD processor core. As previously shown, in one implementation, this is achieved by creating table 500 ( Figure 5 The allocation is performed using code segments 610, 620, and 630. These segments include instructions for supporting the broadcasting of shared data to multiple compute units. However, the compute kernel executing on a compute unit of a SIMD processor core does not receive instructions from any other compute kernel instructing that the shared data be shared with any other compute unit. In various implementations, code 600 is a computer program comprising a compute kernel and a collection of internal functions. Software developers typically define the compute kernel, while the internal functions are typically defined in libraries.

[0079] In some implementations, code 600 operates on specific columns of a matrix (e.g., the previous matrix B 320). The columns of matrix B 320 are specific blocks, such as blocks B32, B33, etc. Each computation unit (e.g., CU 0-15 in the previous example) executes the instructions of code 600, but operates on different data items. The earlier examples shown for matrices 100, 200, 300, and 400 illustrate that computation units are grouped differently when sharing data. Sharing is based on the input matrix. Code segment 610 generates a mapping from thread groups to computation units. For example, code segment 610 constructs a mapping similar to Table 500 (… Figure 5 The table (of type B) is used to construct one or more tables to support the broadcasting of shared data, and the table also assigns thread groups to specific computational units. In some implementations, code segment 610 is placed in an outer loop, which includes code 600 within that loop. In one implementation, each iteration of the "for loop" in code 600 operates on a partition or subset of columns of a matrix, such as matrix B350. For example, each iteration operates on a quarter of the data in block B32 of matrix B350. In some implementations, field 516 of table 500 specifies the iteration size.

[0080] In various embodiments, when code segment 620 detects the generation of a read access request for shared data, code segment 620 generates a broadcast start message. In some embodiments, code segment 600 includes a hint from the static compiler indicating which read access requests target shared data and which broadcast identifier corresponds to the shared data. For example, the static compiler compares the target address of the read access request with the address range of the shared data. Therefore, code segment 620 detects the compiler hint. In other embodiments, the compiler hint is not used, and code segment 620 performs the comparison of the target address of the read access request with the address range of the shared data. The message includes an indication specifying the start of broadcast data, the corresponding broadcast identifier, and the target address.

[0081] In one implementation, instructions in code 600 between code segment 620 and code segment 630 retrieve data from another matrix (e.g., matrix A 310 in the previous example or another matrix A). These instructions also store the retrieved data in the local data store (LDS) of the corresponding computing unit. The instructions perform the retrieval and storage separately for each data item. Code segment 630 detects the start of a broadcast message for matrix B and updates a count, such as a count of the number of read access requests for the same shared data. Code segment 630 also compares this count to the total number of thread groups sharing the shared data. This threshold is the same as the threshold stored in field 514 of table 500. When the count equals the threshold, code segment 630 generates a single, individual read access request and sends it to the memory subsystem to retrieve the requested data.

[0082] The generated single read access request includes an indication that specifies the read access request is associated with a broadcast of the requested data. In one implementation, this indication is a bit that stores an assertion value to indicate the broadcast of the requested data. The single read access request also includes an indication of which computing units receive the requested data during a subsequent broadcast. In some implementations, the single read access request includes a bitmask specifying which computing units receive the requested data during the broadcast, such as a bitmask in field 522 of table 500. In other implementations, the single read access request includes a broadcast identifier, which is then translated or decoded to determine which computing units receive the requested data during the broadcast. Thus, when the memory subsystem provides the requested shared data based on a single memory access, that data is simultaneously provided to the local data store of the specified computing unit. Code following code segment 630 reads data from the local data store and uses that data to perform matrix operations, such as GEMM operations. The "for loop" of code 600 is repeated as needed.

[0083] Now for reference Figure 7 This illustrates one embodiment of a method 700 for efficiently parallelizing multiple work units in a processor by reducing the number of memory accesses. For the purposes of discussion, this embodiment (and) is shown in sequence. Figure 8 The steps in ) are described. However, it should be noted that in various embodiments of the described methods, one or more of the described elements are performed simultaneously, in an order different from that shown, or are omitted entirely. Other additional elements may also be performed as needed. Any of the various systems or apparatuses described herein are configured to implement methods 700 and 800.

[0084] Software developers design applications with GEMM operations. In one implementation, the logic and circuitry of the SIMD processor core execute the application's instructions to perform GEMM operations. In one implementation, the application includes components similar to (…). Figure 6 The application generates multiple computing cores. It also assigns computing cores sharing the same data to computing units by loading broadcast characteristics into a table, each core capable of supporting broadcasting of shared data (box 702). The application matches its computing cores with records of one or more data items to generate one or more computing work units. The work units are divided into multiple thread groups, each with its own assigned unique identifier (ID). The application assigns multiple thread groups to computing units within the SIMD processor core. In one implementation, this is achieved by utilizing information in one or more of fields 512-524. Figure 5 Table 500 is used to perform the allocation.

[0085] In some implementations, the application includes hints from the static compiler indicating which read access requests target shared data and which broadcast identifier corresponds to the shared data. For example, the static compiler compares the target address of the read access request with the address range of the shared data. If no memory access for shared data is detected (the "No" branch of condition box 704), the application processes other instructions of the compute kernel (box 706). If the end of the compute kernel is reached (the "Yes" branch of condition box 708), method 700 completes (box 710). However, if the end of the compute kernel is not reached (the "No" branch of condition box 708), the control flow of method 700 returns to condition box 704, where it is determined whether a memory access to shared data has occurred.

[0086] If a memory access to target shared data is detected, for example, if the application detects a compiler prompt (the "Yes" branch of condition box 704), then when the instructions of the application or firmware are executed, the SIMD processor core and / or cache controller performs a single access to the shared data based on broadcast characteristics (box 712). In one embodiment, the application includes instructions for inserting a mask that identifies the compute unit using the shared data into a single access request. In another embodiment, the compute core includes instructions for generating a mask based on a broadcast identifier. In such an embodiment, the single access request includes a broadcast identifier but does not include a mask. In other embodiments, the single access request includes a mask but does not include a broadcast identifier. When the instructions of the application or firmware are executed, and the SIMD processor core and / or cache controller determines that the memory subsystem has returned shared data, the shared data is broadcast to the appropriate compute unit according to broadcast characteristics (e.g., a mask) (box 714).

[0087] Now for reference Figure 8 A flowchart 800 illustrates a method for implementing a shared data broadcast protocol. By loading broadcast characteristics into a table, the application performing GEMM operations also assigns groups of threads sharing the same data to computing units. In one implementation, this is achieved by utilizing information in one or more of fields 512-524 to create (… Figure 5 Table 500 is used to perform allocation. The computational kernel executing against the thread group conveys read access requests for the shared data. Access to the target shared data is detected (box 802). One or more instructions in the application and circuitry of the processor increment a count of the number of read requests from the thread group for the same shared data (box 804). In one implementation, the application updates field 520 of table 500.

[0088] If the threshold number of read requests from the thread group has not been reached (the "No" branch of condition box 806), the application blocks the extraction of shared data (box 808). No read access requests are issued to the memory subsystem. In one implementation, instructions from one or more applications and circuitry of the processor compare the updated value in field 520 of table 500 with the threshold in field 514 of table 500. If the threshold number of read requests from the thread group is reached (the "Yes" branch of condition box 806), the count is reset (box 810). Shared data is extracted using a single read access request (box 812). For example, a single read access request with one or more of a broadcast identifier and a bitmask, as described previously, is transmitted to the memory subsystem. When shared data is returned from the memory subsystem, the shared data is broadcast to the appropriate computing unit based on broadcast characteristics such as the mask (box 814).

[0089] Reference Figure 9 The diagram illustrates one embodiment of a computing system 900. A processing node 910 includes one or more processing units, such as units 915 and 930. Unit 915 includes one or more processor cores 912 and an associated cache memory subsystem 914. Unit 930 includes one or more processor cores 932 and a data storage buffer 934. The processing node 910 also includes a memory controller 920 and interfaces 940a and 940b. The memory controller 920 includes logic and queues for servicing memory requests and responses and for communicating with memory 950 based on a specific communication protocol. Interfaces 940a-940b also include logic and queues for servicing requests and responses and for communicating with specific devices based on appropriate communication protocols. In one embodiment, interface 940b communicates with a video processing unit 960. In one embodiment, at least the illustrated functionality of the processing node 910 is integrated onto a single integrated circuit. In some embodiments, the functionality of the computing system 900 is integrated onto a system-on-a-chip (SoC). In other embodiments, the functionality of the video processing unit 960 is integrated into a video graphics card inserted into a motherboard, which also includes a processing node 910 connected to a memory 950.

[0090] For ease of illustration, other components such as phase-locked loops (PLLs) and other clock generation circuits, power management units, shared cache memory subsystems, etc., are not shown. In one embodiment, processor core 912 utilizes a general-purpose microarchitecture. Processor core 932 is not a mirror silicon image of processor core 912. Instead, processor core 932 has a different microarchitecture than the one used by processor core 932. In one embodiment, processor core 932 has a microarchitecture that provides high instruction throughput for computationally intensive tasks, such as a parallel data architecture. In one embodiment, processor core 932 is a Single Instruction Multiple Data (SIMD) core. Examples of SIMD cores include graphics processing units (GPUs), digital signal processing (DSP) cores, or others. Similarly, processor core 962 of video processing unit 960 also has a SIMD architecture. Each processor core 932 and 962 includes multiple compute units 970, each compute unit having multiple parallel execution channels 972. In one embodiment, processor core 962 has 16 compute units, each compute unit having 64 parallel execution channels. In other embodiments, a different number of compute units and parallel execution channels are used.

[0091] Typically, processor core 912 accesses cache memory subsystem 914 for both data and instructions. If the requested block is not found in cache memory subsystem 914, processor core 912 generates a read access request and sends it to memory 950 via memory controller 920. Similarly, when the requested data is not found in buffer 934, processor core 932 generates a memory access request to send it to memory 950. In one embodiment, one or more of cache memory subsystem 914 and buffer 934 include high-speed cache memories configured to store data blocks and are implemented as a cache hierarchy. Both cache memory subsystem 914 and buffer 934 include cache array memory and a corresponding cache controller.

[0092] Typically, packet processing logic 916 responds to control packets received on the link coupled to processing node 910, generates control packets in response to processor core 912 and / or cache memory subsystem 914, generates probe commands and response packets in response to transactions selected by memory controller 920 for service, and routes the packets (for such packets, node 910 is an intermediate node) to other nodes via interface logic 940a.

[0093] In some implementations, processor core 962 executes graphics-intensive applications, such as vertex shaders or pixel shaders, on a large number of objects (vertices or pixels). Since each object is processed independently of other objects but uses the same sequence of operations, the SIMD architecture using compute unit 970 and channel 972 can significantly improve performance. In some implementations, processor core 932 executes applications for training neural networks, which also process objects independently of other objects but use the same sequence of operations. Therefore, these types of applications also benefit from the SIMD architecture using compute unit 970 and channel 972.

[0094] In various embodiments, memory 950 includes one or more of hard disk drives, solid-state drives, other types of flash memory, portable solid-state drives, tape drives, etc. Memory 950 stores operating system 952, application programs 954, and data 956. Although a single operating system and a single application are shown, in other embodiments, a greater number of these software components are stored in memory 950. Operating system 952 includes instructions for starting processing node 910, allocating tasks to hardware circuitry, managing resources of computing system 900, and hosting one or more virtual environments.

[0095] In some implementations, the general-purpose processor core 912 executes instructions from the operating system 952. In some implementations, the application 954 uses GEMM operations for neural network training, pixel interpolation during graphics rendering, or other purposes. In one implementation, the application 954 matches its software kernel with one or more data records (e.g., data items in data 956) to generate one or more computational work units. The work units are divided into multiple thread groups, each with its own assigned unique identifier (ID). The application 954 also includes code for supporting the broadcasting of shared data, such as broadcast support code 955. In one implementation, code 955 is similar to code 600 (…). Figure 6 Code segments 610, 620, and 630.

[0096] Code 955 and one or more of the firmware, depending on the application being processed, assign thread groups to computing units 970 in one or more of SIMD processor cores 932 and 962. In one implementation, this is achieved by creating table 500 ( Figure 5 The allocation is performed using a specific method. For example, in one implementation, application 954 allocates a thread group for pixel interpolation to the SIMD processor core 962 of video processing unit 960, and application 954 allocates multiple thread groups for neural network training to the SIMD processor core 932 of unit 930. Other types of allocations for these and other applications are possible and can be anticipated. When software application 954 includes code 955 for supporting broadcasting of shared data, the shared data is fetched only once, the memory subsystem is not loaded without broadcast support, latency is reduced, and system performance is improved.

[0097] In various embodiments, the program instructions of the software application are used to implement the previously described methods and / or mechanisms. The program instructions describe the behavior of the hardware using a high-level programming language (e.g., C). Alternatively, a hardware design language (HDL), such as Verilog, may be used. The program instructions are stored on a non-transitory computer-readable storage medium. Various types of storage media can be used. During use, the computing system can access the storage medium to provide the program instructions and accompanying data to the computing system to execute the program. The computing system includes at least one or more memories and one or more processors that execute the program instructions.

[0098] It should be emphasized that the above embodiments are merely non-limiting examples of implementation methods. Many variations and modifications will become apparent to those skilled in the art once the above disclosure is fully understood. It is intended that the following claims be interpreted as encompassing all such variations and modifications.

Claims

1. A computing device comprising: The first interface is configured to communicate with a processor comprising a plurality of computing units, each of which is configured to process instructions; and The circuit is configured as follows: A first access request for shared data is received from one of a plurality of computing units via the first interface; and In response to determining that a qualifying condition is met, a single access request is transmitted for the shared data requested by multiple computing units out of a plurality of computing units, wherein the qualifying condition is that the count of access requests for the shared data is equal to the number of computing units that require the shared data.

2. The computing device of claim 1, wherein the circuitry is configured to prevent the transmission of access requests for the shared data until the count equals the number of computing units that require the shared data.

3. The computing device according to claim 1, wherein, The circuit is configured as follows: For each access request received from the processor for shared data, increment the count of the access requests; as well as In response to determining that the count equals the sum of the given computing unit and one or more other computing units, it is determined that the qualification condition is met.

4. The computing device according to claim 1, wherein, The circuit is configured to store broadcast characteristics for the shared data, including one or more of the following: Broadcast identifier, wherein the broadcast identifier is a unique identifier that identifies the shared data; and The base address specifies the storage location where the beginning of the shared data is stored.

5. The computing device according to claim 1, wherein, The circuit is configured to store broadcast characteristics for the shared data, including one or more of the following: A broadcast identifier that identifies the shared data; and The base address specifies the storage location where the beginning of the shared data is stored.

6. The computing device according to claim 5, wherein, The circuit is configured to store in the broadcast feature a number equal to the sum of a given computing unit and one or more other computing units.

7. The computing device according to claim 5, wherein, The circuit is configured to insert one or more of the following into the single access request: The broadcast identifier; and A mask that identifies a given computational unit and one or more other computational units.

8. The computing device of claim 7, further comprising a cache controller, the cache controller being configured to: Receive the mask in the single access request; and This includes the broadcast identifier in the single access request.

9. The computing device according to claim 8, wherein, The cache controller is configured as follows: Store the mask; as well as During the broadcasting of the shared data to the plurality of computing units, the mask is transmitted to the second processor.

10. A calculation method, comprising: Through a System-on-a-Chip (SoC), a plurality of computing cores are allocated to a plurality of computing units of a processor, wherein a given computing core among the plurality of computing cores is configured to share a data broadcast protocol; In response to: An access request for the shared data requested by multiple computing units among a plurality of computing units was detected from a given computing unit; and The qualification condition is determined to be met, wherein the qualification condition is that the count of access requests for the shared data is equal to the number of computing units that require the shared data; The system-on-chip transmits a single access request for shared data to the memory subsystem, the shared data being shared by the given computing unit and one or more other computing units; and In response to receiving the shared data from the memory subsystem, the shared data is broadcast via the system-on-chip to the given computing unit and the one or more other computing units.

11. The method according to claim 10, wherein, The given computing kernel does not communicate with other computing kernels regarding the shared data.

12. The method of claim 10, further comprising preventing the transmission of access requests for the shared data until the count equals the number of computing units that require the shared data.

13. The method of claim 10, further comprising: In response to detecting an access request for the shared data, the count of the access requests is incremented.

14. The method of claim 10, further comprising: In response to determining that the count of access requests for the shared data is not equal to the number of computing units that require the shared data, access requests for the shared data are prevented from being sent.

15. The method of claim 10, further comprising a broadcast feature for storing the shared data, the broadcast feature including one or more of the following: Broadcast identifier, wherein the broadcast identifier is a unique identifier that identifies the shared data; and The base address specifies the storage location where the beginning of the shared data is stored.

16. A computing system comprising: A processing unit, which includes a plurality of computing units; A memory subsystem coupled to the processing unit, wherein the memory subsystem is configured to store the operating system, applications, and user data; and The operating system includes instructions executable by the processing unit to allocate a plurality of computing cores to the plurality of computing units, wherein a given computing core among the plurality of computing cores is configured to support a shared data broadcast protocol. The application includes instructions that can be performed by the processing unit: In response to: An access request for the shared data requested by multiple computing units out of a plurality of computing units was detected from a given computing unit; and The qualification condition is determined to be met, wherein the count of access requests for the shared data is equal to the number of computing units that require the shared data; A single access request for shared data, which is shared by the given computing unit and one or more other computing units, is transmitted to the memory subsystem; and In response to receiving the shared data from the memory subsystem, the shared data is broadcast to the given computing unit and the one or more other computing units.

Images