A graphics processor capable of maintaining data coherency within a core
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING FENGHUA CHUANGZHI TECHNOLOGY CO LTD
- Filing Date
- 2026-03-10
- Publication Date
- 2026-06-16
AI Technical Summary
In graphics processing units (GPUs), because multiple computing cores execute tasks in parallel, it is difficult to effectively maintain data consistency between cores, resulting in high communication overhead, limited performance, and inaccurate calculation results.
A multi-level broadcast link mechanism is adopted. The first-level and second-level link modules identify and intercept requests that need to maintain data consistency, and broadcast them to the cache modules of multiple computing cores to ensure data consistency, including memory barriers, flushing, and handling of invalid requests.
It achieves data consistency within the graphics processor core in a short time, reduces communication overhead, improves computing performance and system reliability, and supports parallel computing tasks.
Smart Images
Figure CN122220127A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data processing technology, and more specifically, relates to a graphics processor that can maintain the consistency of data within the kernel. Background Technology
[0002] In modern Graphics Processing Units (GPUs), data consistency between cores is a critical challenge due to the parallel execution of tasks by multiple computing cores. As the number of computing cores increases and task complexity rises, ensuring data consistency between cores becomes even more critical. Existing technologies typically employ cache coherency protocols to ensure data consistency across multiple cores. These protocols maintain data consistency by interacting and synchronizing between caches. However, these protocols often require significant communication overhead, and their performance can be limited when faced with a large number of concurrent operations. Furthermore, some concurrent operations can lead to data inconsistencies or latency, thereby affecting the correctness and efficiency of computation. Summary of the Invention
[0003] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a graphics processor that can maintain the consistency of data within the core. It utilizes a multi-level broadcast link to realize data broadcasting and synchronization between computing cores, which can reduce communication overhead and achieve the consistency of data within the GPU core in a shorter time.
[0004] To achieve the above objectives, according to one aspect of the present invention, a graphics processor is provided, including a computing core; the computing core has multiple sub-computing units, multiple first-level link modules, multiple first-level cache modules, a second-level link module, and a second-level cache module; the multiple sub-computing units, multiple first-level link modules, and multiple first-level cache modules correspond one-to-one, each first-level link module is connected to a corresponding sub-computing unit and a first-level cache module, the second-level link modules are respectively connected to multiple first-level link modules, and the second-level cache modules are respectively connected to multiple first-level cache modules; Multiple first-level link modules are used to identify the request type of the corresponding sub-computing unit. When the corresponding sub-computing unit issues a request that requires maintaining data consistency, the request is intercepted and transmitted to the second-level link module. The second-level link module is used to broadcast the request to multiple first-level link modules. After receiving the request from the second-level link module, the multiple first-level link modules are also used to forward the received request to the corresponding first-level cache module. After receiving the request forwarded by the corresponding first-level link module, the multiple first-level cache modules perform corresponding operations to maintain the data consistency of the computing core.
[0005] In some implementations, when the request that needs to maintain data consistency is a memory barrier request, the first-level cache module corresponding to the sub-computing unit that issued the memory barrier request writes the data of the corresponding address segment into the second-level cache module after receiving the request forwarded by the corresponding first-level link module.
[0006] In some implementations, after receiving a request forwarded by the corresponding first-level link module, the first-level cache module of other sub-computing units in the computing core, excluding the sub-computing unit that issued the memory barrier request, processes the data at the corresponding address, so that when the corresponding sub-computing unit performs a read operation, it reads the data at the corresponding address from the second-level cache module.
[0007] In some implementations, after receiving a request forwarded by the corresponding first-level link module, the first-level cache module of the sub-computing unit other than the sub-computing unit that issued the memory barrier request in the computing core sets the first flag bit of the data at the corresponding address to a first value. This enables the corresponding sub-computing unit to initiate a data read request to the second-level cache module when performing a read operation, based on the first flag bit of the data being set to the first value. The second-level cache module is used to send the requested data back to the corresponding first-level cache module after receiving the data read request.
[0008] In some implementations, when the request that needs to maintain data consistency is a refresh request, after receiving the request from the second-level link module forwarded by the corresponding first-level link module, multiple first-level cache modules check whether there is cached data for the corresponding address range. If so, they further check whether the data has been written locally. If so, they write the data for the corresponding address range into the second-level cache module.
[0009] In some implementations, when the request that needs to maintain data consistency is a refresh request, multiple first-level cache modules, when caching data for the corresponding address range, further check whether the second flag bit of the corresponding cache line is the second value. If the second flag bit of the corresponding cache line is the second value, the data of the cache line is written to the second-level cache module.
[0010] In some implementations, when a request that requires maintaining data consistency is an invalid request, multiple first-level cache modules, after receiving a request forwarded from a second-level link module by the corresponding first-level link module, process the data at the corresponding address to invalidate that part of the data.
[0011] In some implementations, when a request that requires maintaining data consistency is an invalid request, multiple first-level cache modules, after receiving a request forwarded from a second-level link module by the corresponding first-level link module, set the third flag of the data at the corresponding address to a third value.
[0012] In some implementations, multiple first-level link modules are also used to forward requests to the corresponding first-level cache modules when the corresponding sub-computing unit issues a request that does not require maintaining data consistency.
[0013] In some implementations, multiple first-level link modules are used to intercept and transmit the request to the second-level link module when the corresponding sub-computing unit issues a request that requires maintaining data consistency, and to suspend receiving requests from the corresponding sub-computing unit; the first-level link module corresponding to the sub-computing unit that issued the request that requires maintaining data consistency resumes receiving requests from the sub-computing unit after receiving the request from the second-level link module.
[0014] According to another aspect of the present invention, a graphics processor is provided, comprising a plurality of computing cores, each computing core comprising a plurality of sub-computing units, a plurality of first-level link modules, a plurality of first-level cache modules, a second-level link module, and a second-level cache module; in each computing core, the plurality of sub-computing units, the plurality of first-level link modules, and the plurality of first-level cache modules correspond one-to-one, each first-level link module is connected to the corresponding sub-computing unit and the first-level cache module, the second-level link modules are respectively connected to the plurality of first-level link modules, and the second-level cache modules are respectively connected to the plurality of first-level cache modules; The graphics processor also includes a third-level link module and a third-level cache module; the third-level link module connects the third-level cache module and the second-level link module of multiple computing cores, and the third-level cache module connects the third-level link module and the second-level cache module of multiple computing cores. In each computing core, multiple first-level link modules are used to intercept and transmit requests that require maintaining data consistency from the corresponding sub-computing unit to the second-level link module. The second-level link module is used to determine whether the request that requires maintaining data consistency is a local request or a global request. If the request is determined to be a global request, the request is transmitted to the third-level link module. The third-level link module is used to broadcast requests to multiple first-level link modules across multiple computing cores; each first-level link module is also used to forward the request to the corresponding first-level cache module after receiving the request from the third-level link module; after receiving the request forwarded by the corresponding first-level link module, the multiple first-level cache modules across multiple computing cores perform corresponding operations to maintain the data consistency of the graphics processor.
[0015] In some implementations, when the request that needs to maintain data consistency is a memory barrier request, the first-level cache module corresponding to the sub-computing unit that issued the memory barrier request writes the data of the corresponding address segment into the corresponding second-level cache module after receiving the request forwarded by the corresponding first-level link module, and then further writes it into the third-level cache module.
[0016] In some implementations, after receiving a request forwarded by the corresponding first-level link module, the first-level cache module of other sub-computing units in the graphics processor, excluding the sub-computing unit that issued the memory barrier request, processes the data at the corresponding address, so that the corresponding sub-computing unit reads the data at the corresponding address from the third-level cache module when performing a read operation.
[0017] In some implementations, when the request that needs to maintain data consistency is a refresh request, multiple first-level cache modules of multiple computing cores, after receiving the request forwarded by the corresponding first-level link module, check whether there is cached data for the corresponding address segment. If so, they further check whether the data has been written locally. If so, they write the data for the corresponding address segment to the third-level cache module.
[0018] In some implementations, when a request that requires maintaining data consistency is an invalid request, multiple first-level cache modules of multiple computing cores, upon receiving a request forwarded by the corresponding first-level link module, process the data at the corresponding address to invalidate that part of the data.
[0019] In some implementations, in each computing core, when the second-level link module confirms that the request to maintain data consistency is a local request, it broadcasts the request to multiple first-level link modules in the computing core. The multiple first-level link modules in the computing core are also used to forward the request to the corresponding first-level cache module after receiving the request from the second-level link module. After receiving the request forwarded by the corresponding first-level link module, the multiple first-level cache modules in the computing core perform corresponding operations to maintain the data consistency of the computing core.
[0020] In summary, compared with existing technologies, the technical solutions conceived in this invention have the following beneficial effects: They utilize broadcast links to achieve data broadcasting and synchronization between computing cores. Specifically, through broadcast links, computing cores can send dirty data (i.e., newly generated data) from their caches to shared memory, ensuring that all computing cores can access the latest data. This ensures that the cached data of multiple computing cores within a GPU core and multiple sub-computing modules within a single computing core remains consistent, avoiding system errors and inaccurate calculation results caused by data consistency issues. It achieves data consistency within the GPU core in a shorter time, making data consistency maintenance more efficient and reliable, reducing communication overhead and latency, and significantly improving computing performance and system reliability. Furthermore, the design of the broadcast link fully considers the needs of parallel computing tasks, providing better support and performance improvements for parallel computing. Attached Figure Description
[0022] Figure 1 This is a schematic diagram of a single-core processor. Figure 2 This is a schematic diagram of another type of single-core processor; Figure 3 This is a schematic diagram of a multi-core processor. Figure 4 This is a schematic diagram of the structure of a graphics processor according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a graphics processor according to another embodiment of the present invention. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of this application. Therefore, the drawings and description are considered exemplary in nature and not restrictive.
[0024] like Figure 1 As shown, a single-core processor includes a computing core, which in turn includes a sub-computation unit. The computing core is responsible for instruction execution, and during instruction execution, it retrieves metadata from main memory (not shown in the figure) and generates new data. The actual instruction execution is completed by the sub-computation unit within the computing core; that is, all instructions that need to be executed within the processor are completed by this sub-computation unit.
[0025] Sub-computation units include internal cache units. When the processor executes a program, it loads frequently used instructions from main memory into the internal cache unit to reduce the latency of fetching instructions from main memory each time and improve processing speed. Instructions are usually not modified during execution; therefore, the internal cache unit only supports read operations from the sub-computation unit, requiring only one instruction to be read from main memory and stored in the internal cache unit.
[0026] During instruction execution, the sub-computation unit caches dirty data generated by the executed instructions in the first-level cache module via write requests and retrieves the required data from the first-level cache module via read requests. It is important to note that the dirty data generated by the sub-computation unit is only temporarily written to the first-level cache module, and then all dirty data is written back at a later point in time, i.e., written back to the second-level cache module or even the third-level cache module.
[0027] Figure 1The single-core processor shown essentially eliminates data consistency issues. Because it has only one computing core, and this core has only one sub-computation unit, the only end user of any dirty data generated by the executed instructions is the sub-computation unit that produced that data. Therefore, unless a hardware failure occurs, leading to the loss of cached data or transmission errors, data consistency problems will not arise within this single sub-computation unit.
[0028] With the development of big data and cloud computing, there are further requirements for the performance of graphics processing units (GPUs). For example, GPUs are typically required to include multiple parallel computing cores and multi-level caches. To ensure the normal operation of GPUs, it is necessary to ensure the consistency of data and instructions between each computing core and each level of cache. That is, when each computing core competes to read and write cached data in cache lines that have related and competitive relationships, it can ensure that the cached data it retrieves from the cache is the latest data in that cache line, and that its read and write operations are performed on the latest data.
[0029] To meet the demands of high computing power, current popular graphics processing unit (GPU) architectures employ a multi-core approach, with even a single computing core containing multiple sub-computing units. For example... Figure 2 As shown, the processor has a computing core C0, which contains N+1 sub-computing units U0 to UN (each sub-computing unit is associated with...). Figure 1 The structure shown is consistent.
[0030] Since the computing core C0 is internally configured with multiple parallel sub-computing units U0 to UN, when a computing / rendering task is assigned to the computing core C0 and completed, the computing core C1 will perform further instruction allocation internally, distributing all instructions for this task a second time. In other words, a computing / rendering task received by the computing core C0 is completed in parallel by the sub-computing units U0 to UN.
[0031] This raises the issue of data consistency and synchronization. For example, during instruction execution, both sub-computation unit U0 and sub-computation unit U1 read back the data 0x1234_5678 corresponding to address 0x00ff from main memory and store it in the first-level cache modules of sub-computation unit U0 and sub-computation unit U1, respectively. Subsequently, during execution, sub-computation unit U1 generates a new data 0x0000_1234, whose corresponding address is exactly 0x00ff.
[0032] Because the system employs a write-back strategy, the new data 0x0000_1234 is temporarily stored in the local first-level cache module (i.e., the first-level cache module corresponding to sub-computation unit U1). If sub-computation unit U0 needs to use the data corresponding to this address 0x00ff again, it will be hit in the local first-level cache module (i.e., the first-level cache module corresponding to sub-computation unit U0), and therefore can only retrieve the old data 0x1234_5678, and cannot retrieve the new data 0x0000_1234 generated by sub-computation unit U1. It is evident that parallel computation units cannot retrieve new data generated by each other, but can only retrieve old data, which creates a data consistency problem and can lead to computational errors.
[0033] Furthermore, in such Figure 3 The multi-core processor shown has M+1 computing cores C0 to CM, each with multiple sub-computing units. This significantly enhances the parallel processing capability of the graphics processing unit (GPU), meeting the demands of increasingly demanding computing scenarios. However, it also complicates data consistency issues. With multiple computing cores C0 to CM, the GPU needs to maintain data consistency across all internal cache modules, and the numerous parallel cache modules undoubtedly place a heavy burden on maintaining this consistency.
[0034] like Figure 4 As shown, a graphics processor according to one embodiment of the present invention includes a computing core C0. The computing core C0 includes N+1 sub-computing units U0 to UN, N+1 first-level link modules, N+1 first-level cache modules, a second-level link module, and a second-level cache module. The N+1 first-level link modules, N+1 first-level cache modules, and N+1 sub-computing units U0 to UN correspond one-to-one. Each first-level link module connects to its corresponding sub-computing unit and first-level cache module. The second-level link modules connect to the N+1 first-level link modules, and the second-level cache modules connect to the N+1 first-level cache modules.
[0035] Each sub-computing unit also includes an internal cache unit, which stores the instructions that the corresponding sub-computing unit needs to execute. The sub-computing unit is used to retrieve instructions from the corresponding internal cache unit and execute them, as well as to issue requests based on the executed instructions.
[0036] The first-level link module is used to identify the request type of its corresponding sub-computing unit and determine whether data consistency needs to be maintained based on the request type.
[0037] When it's confirmed that maintaining data consistency is unnecessary, such as when the request is a general read / write request, no operation is performed, and the request is directly sent to the first-level cache module. In this case, it can be understood that the request sent by the sub-computing unit is transparently passed from the first-level link module to the first-level cache module, and the first-level link module is inactive. Specifically, when the request is a read request, the read request arrives at the first-level cache module from the sub-computing unit, and the sub-computing unit performs a read operation on the first-level cache module. When the request is a write request, the write request arrives at the first-level cache module from the sub-computing unit, and the sub-computing unit performs a write operation on the first-level cache module, writing the resulting dirty data into the first-level cache module. In some implementations, when the first-level link transparently passes the request sent by the sub-computing unit, the original request order remains unchanged.
[0038] When it is confirmed that data consistency needs to be maintained, the first-level link module intercepts the request and transmits it to the second-level link module, suspending the reception of requests from the corresponding sub-computing unit. At this time, other first-level link modules continue to receive requests initiated by their respective sub-computing units normally, unaffected, which helps improve path performance. The second-level link module broadcasts the request from the first-level link module to the N+1 first-level link modules corresponding to all N+1 sub-computing units U0 to UN. After receiving the request from the second-level link module, the N+1 first-level link modules forward the received request to their corresponding N+1 first-level cache modules. After receiving the forwarded request from the second-level link module, the N+1 first-level cache modules perform the corresponding operations to maintain data consistency from sub-computing units U0 to UN. In some implementations, the first-level link module corresponding to the sub-computing unit that issued the request requiring data consistency maintenance resumes receiving requests from that sub-computing unit after receiving the request from the second-level module.
[0039] In some implementations, the request to maintain data consistency is a memory barrier request. After receiving a request from a second-level link module forwarded by the corresponding first-level link module, the first-level cache module corresponding to the sub-computing unit that issued the memory barrier request writes the data at the corresponding address into the second-level cache module, or even into the next-level cache module (third-level cache module) or main memory. The remaining N first-level cache modules, after receiving requests from second-level link modules forwarded by their respective first-level link modules, process the data at the corresponding address so that the corresponding sub-computing unit does not directly read the data at the corresponding address from the first-level cache module when performing a read operation.
[0040] In some implementations, after receiving a request from a second-level link module forwarded by the corresponding first-level link module, the remaining N first-level cache modules set the first flag bit of the data at the corresponding address to a first value. The sub-computing unit is further configured to check the first flag bit of the data in the first-level cache module when requesting to retrieve data from the corresponding first-level cache module (read operation). If the first flag bit of the data is the first value, the sub-computing unit initiates a read data request to the second-level cache module. Upon receiving the read data request, the second-level cache module sends the requested data back to the first-level cache module corresponding to the sub-computing unit, and the sub-computing unit then retrieves the data from the corresponding first-level cache module. If the first flag bit of the data is not the first value, the sub-computing unit directly retrieves the data from the corresponding first-level cache module.
[0041] For example, after the remaining N first-level cache modules receive a request from the second-level link module forwarded by the corresponding first-level link module, they set the data flag re-fetch of the corresponding address to 1 (initially 0; when the flag re-fetch is 0, the sub-computing unit can directly obtain data from the corresponding first-level cache module). The next time the corresponding sub-computing unit needs data at the corresponding address, it first performs a read operation on the first-level cache module. According to the flag re-fetch being 1, it will not read directly from the first-level cache module, but will instead send a request to the second-level cache module to read the data. The second-level cache module then sends the requested data back to the first-level cache module.
[0042] In some implementations, the request requiring data consistency maintenance is a refresh request. After receiving a request from a second-level link module forwarded by the corresponding first-level link module, the N+1 first-level cache modules check if they have cached data for the corresponding address segment. If so, they further check if the data has been written locally. If so, they write the data for the corresponding address segment to the second-level cache module, or even to the next-level cache module (third-level cache module) or main memory. In some implementations, when the N+1 first-level cache modules have cached data for the corresponding address segment, they further check if the second flag bit of the corresponding cache line is the second value. If the second flag bit of the corresponding cache line is the second value, they write the data for the cache line to the second-level cache module. For example, if the first-level cache module has cached data for the corresponding address segment, it further checks if the dirty flag bit of the corresponding cache line is 1. If the dirty flag bit is 1, it writes the data for the cache line to the second-level cache module.
[0043] In some implementations, requests requiring data consistency maintenance are considered invalid requests. After receiving a request forwarded from a second-level link module by the corresponding first-level link module, the N+1 first-level cache modules process the data at the corresponding address, invalidating that data. In some implementations, after receiving a request forwarded from a second-level link module by the corresponding first-level link module, the N+1 first-level cache modules set the third flag bit of the data at the corresponding address to a third value. For example, setting the valid flag bit of the corresponding address to 0.
[0044] In some implementations, since the second-level cache module is closer to the processing core than the third-level cache module, the second-level cache module has a smaller capacity but a faster access speed compared to the third-level cache module. It is used to store data copies of the most frequently accessed main memory locations. When a sub-computing unit cannot find data in the first-level cache, it will access the second-level cache to reduce the time to access the data.
[0045] Continuing with the previous example of sub-computation units U0 and U1, during operation, sub-computation unit U1 generates a new data 0x0000_1234, whose corresponding address is 0x00ff. This new data 0x0000_1234 is temporarily stored in the first-level cache module corresponding to sub-computation unit U1. When data consistency needs to be maintained, sub-computation unit U1 initiates a memory barrier request. The first-level link module corresponding to sub-computation unit U1 intercepts the memory barrier request and transmits it to the second-level link module. The second-level link module then broadcasts the request to all first-level link modules, and the first-level link modules forward the request to the corresponding first-level cache module, completing the link transmission and broadcast process.
[0046] After receiving a memory barrier request, the first-level cache module corresponding to sub-computing unit U1 writes the dirty data 0x0000_1234 at address 0x00ff to the second-level cache module, thus achieving data synchronization between sub-computing units. After receiving a memory barrier request, the first-level cache module corresponding to sub-computing unit U0 sets the re-fetch flag at address 0x00ff to 1. Sub-computing unit U0 will not directly read from the corresponding first-level cache module for a read request at address 0x00ff; instead, it will send a read request to the second-level cache module, which will then send the requested data 0x0000_1234 back to the first-level cache module. In this way, sub-computing unit U0 can successfully read back the new data 0x0000_1234 generated by sub-computing unit U1 for a read request at address 0x00ff.
[0047] Memory barrier requests, refresh requests, and invalidation requests can be initiated as needed to ensure data consistency within the graphics processor core. This invention, by introducing a multi-level link module, enables data broadcasting and synchronization between sub-computing units within the computing core. This effectively achieves data consistency within the core in a short time, ensuring that cached data across multiple sub-computing units within the core remains consistent, thereby improving the computing performance and reliability of the graphics processor.
[0048] like Figure 5 As shown, another embodiment of the graphics processor of the present invention includes multiple computing cores, each computing core including multiple sub-computing units, multiple first-level link modules, multiple first-level cache modules, second-level link modules, and second-level cache modules. For example, the graphics processor includes M+1 computing cores C0 to CM. Taking computing core C0 as an example, computing core C0 includes N+1 sub-computing units U0 to UN, N+1 first-level link modules, N+1 first-level cache modules, second-level link modules, and second-level cache modules. The graphics processor also includes a third-level link module and a third-level cache module. The third-level link module connects the third-level cache module and the second-level link modules of computing cores C0 to CM, and the third-level cache module connects the third-level link module and the second-level cache modules of computing cores C0 to CM.
[0049] Each sub-computing unit also includes an internal cache unit, which stores the instructions that the corresponding sub-computing unit needs to execute. The sub-computing unit is used to retrieve instructions from the corresponding internal cache unit and execute them, as well as to issue requests based on the executed instructions.
[0050] The first-level link module is used to identify the request type of its corresponding sub-computing unit and determine whether data consistency needs to be maintained based on the request type.
[0051] When it's confirmed that maintaining data consistency is unnecessary, such as when the request is a general read / write request, no operation is performed, and the request is directly sent to the first-level cache module. In this case, it can be understood that the request sent by the sub-computing unit is transparently passed from the first-level link module to the first-level cache module, and the first-level link module is inactive. Specifically, when the request is a read request, the read request arrives at the first-level cache module from the sub-computing unit, and the sub-computing unit performs a read operation on the first-level cache module. When the request is a write request, the write request arrives at the first-level cache module from the sub-computing unit, and the sub-computing unit performs a write operation on the first-level cache module, writing the resulting dirty data into the first-level cache module. In some implementations, when the first-level link transparently passes the request sent by the sub-computing unit, the original request order remains unchanged.
[0052] When it's confirmed that data consistency needs to be maintained, the first-level link module intercepts the request and transmits it to the second-level link module, pausing the reception of requests from the corresponding sub-computing unit. At this time, other first-level link modules continue to receive requests initiated by their respective sub-computing units normally, unaffected, which helps improve path performance. The second-level link module identifies whether the request is a local or global request. When it confirms the request is a local request, it proceeds according to... Figure 4 The illustrated embodiment maintains data consistency within the computing core.
[0053] When the second-level link module confirms that the request is a global request, it transmits the request to the third-level link module. The third-level link module then broadcasts the request from the second-level link module to all first-level link modules corresponding to all sub-computing units across all computing cores. Upon receiving the request from the third-level link module, all first-level link modules forward the request to their corresponding first-level cache modules. After receiving the forwarded request from the third-level link module, all first-level cache modules perform corresponding operations to maintain data consistency across all computing cores. In some implementations, the first-level link module corresponding to the sub-computing unit that issued the request requiring data consistency maintenance resumes receiving requests from that sub-computing unit after receiving the request from the third-level link module.
[0054] In some implementations, the request to maintain data consistency is a memory barrier request. After receiving a request from a third-level link module forwarded by the corresponding first-level link module, the first-level cache module corresponding to the sub-computing unit that issued the memory barrier request writes the data for the corresponding address segment into the corresponding second-level cache module, and then further writes it into the third-level cache module and even main memory. The first-level cache modules of the remaining sub-computing units (including other sub-computing units within the same computing core as the sub-computing unit that issued the data consistency request, and all sub-computing units in other computing cores outside the same core) process the data at the corresponding address after receiving a request from a third-level link module forwarded by the corresponding first-level link module, ensuring that the corresponding sub-computing unit does not directly read the data at the corresponding address from the first-level cache module when performing a read operation.
[0055] In some implementations, after receiving a request from the third-level link module forwarded by the corresponding first-level link module, the first-level cache module corresponding to the remaining sub-computing unit sets the first flag bit of the data at the corresponding address to a first value. The sub-computing unit is also used to check the first flag bit of the data in the first-level cache module when requesting to retrieve data from the corresponding first-level cache module (read operation). If the first flag bit of the data is the first value, the sub-computing unit initiates a read data request to the third-level cache module. Upon receiving the read data request, the third-level cache module sends the requested data back to the first-level cache module corresponding to the sub-computing unit, and the sub-computing unit then retrieves the data from the corresponding first-level cache module. If the first flag bit of the data is not the first value, the sub-computing unit directly retrieves the data from the corresponding first-level cache module.
[0056] For example, after the first-level cache module corresponding to the remaining sub-computing unit receives a request from the third-level link module forwarded by the corresponding first-level link module, it sets the data flag re-fetch of the corresponding address to 1 (initially 0; when the flag re-fetch is 0, the sub-computing unit can directly obtain data from the corresponding first-level cache module). The next time the corresponding sub-computing unit needs data at the corresponding address, it first performs a read operation on the first-level cache module. According to the flag re-fetch being 1, it will not read directly from the first-level cache module, but will instead send a request to the third-level cache module to read the data. The third-level cache module then sends the requested data back to the first-level cache module.
[0057] In some implementations, the request requiring data consistency maintenance is a refresh request. Upon receiving a request from a third-level link module forwarded by the corresponding first-level link module, all first-level cache modules of the computing cores check if they have cached data for the corresponding address segment. If so, they further check if the data has been written locally. If so, they write the data for the corresponding address segment to the third-level cache module or even main memory. In some implementations, when the first-level cache modules of all computing cores cache data for the corresponding address segment, they further check if the second flag bit of the corresponding cache line is the second value. If the second flag bit of the corresponding cache line is the second value, they write the data of the cache line to the third-level cache module. For example, if the first-level cache module caches data for the corresponding address segment, it further checks if the dirty flag bit of the corresponding cache line is 1. If the dirty flag bit is 1, it writes the data of the cache line to the third-level cache module.
[0058] In some implementations, requests requiring data consistency maintenance are invalid requests. Upon receiving a request from a third-level link module forwarded by the corresponding first-level link module, all first-level cache modules of all computing cores process the data at the corresponding address, invalidating that data. In some implementations, upon receiving a request from a third-level link module forwarded by the corresponding first-level link module, all first-level cache modules of all computing cores set the third flag bit of the data at the corresponding address to a third value. For example, setting the valid flag bit of the corresponding address to 0.
[0059] A second-level cache module is set in the computing core with multiple sub-computing units to maintain the consistency of data within the computing core, corresponding to local data consistency requests; a third-level cache module is set between the multiple computing cores of the graphics processor to maintain the consistency of data between the multiple computing cores, corresponding to global data consistency requests.
[0060] Traditional architectures, such as those using the snoop mechanism to ensure data consistency, require iterating through all routing nodes for each write request whenever a write request occurs. The system checks if the relevant data exists in the local cache; if so, it updates the data; otherwise, it does nothing. This invention provides a novel solution to reduce the number of internal communications within the caching architecture. It utilizes multi-level link modules to intercept requests related to data consistency, such as memory barrier requests and cache flush / invalidate (CFI) requests, and transmits and broadcasts these requests within the link. Through link transmission, the broadcast link can achieve data consistency in a short time, ensuring that cached data remains consistent across multiple computing cores and among multiple sub-computing modules within a single computing core.
[0061] Specifically, for write operations, it's unnecessary to traverse all nodes every time. Instead, data can be temporarily stored in a local cache. The concept of a local cache is relative; for a specific sub-computation unit, it refers to the first-level cache module, while for a specific computing core, it refers to the second-level cache module. After program control, a local memory barrier request or a global memory barrier request (or a flush request and invalidation request) is initiated to flush dirty data from the local cache to the next-level cache. This significantly reduces internal communication overhead while ensuring data consistency across multiple cores.
[0062] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.
[0063] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.
[0064] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more (two or more) executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.
[0065] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).
[0066] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.
[0067] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.
[0068] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A graphics processor, characterized in that, It includes a computing core; the computing core has multiple sub-computing units, multiple first-level link modules, multiple first-level cache modules, second-level link modules, and second-level cache modules; the multiple sub-computing units, the multiple first-level link modules, and the multiple first-level cache modules correspond one-to-one, each first-level link module connects to the corresponding sub-computing unit and the first-level cache module, the second-level link modules connect to the multiple first-level link modules respectively, and the second-level cache modules connect to the multiple first-level cache modules respectively; The plurality of first-level link modules are used to identify the request type of the corresponding sub-computing unit. When the corresponding sub-computing unit issues a request that requires maintaining data consistency, the request is intercepted and transmitted to the second-level link module. The second-level link module is used to broadcast the request to the plurality of first-level link modules. The plurality of first-level link modules are also used to forward the received request to the corresponding first-level cache module after receiving the request from the second-level link module. After receiving the request forwarded by the corresponding first-level link module, the plurality of first-level cache modules perform corresponding operations to maintain the data consistency of the computing core.
2. The graphics processor as described in claim 1, characterized in that, When the request that needs to maintain data consistency is a memory barrier request, the first-level cache module corresponding to the sub-computing unit that issued the memory barrier request writes the data of the corresponding address segment into the second-level cache module after receiving the request forwarded by the corresponding first-level link module.
3. The graphics processor as described in claim 2, characterized in that, In the computing core, the first-level cache module corresponding to the other sub-computing units besides the sub-computing unit that issued the memory barrier request processes the data at the corresponding address after receiving the request forwarded by the corresponding first-level link module, so that the corresponding sub-computing unit reads the data at the corresponding address from the second-level cache module when performing a read operation.
4. The graphics processor as described in claim 3, characterized in that, In the computing core, the first-level cache modules corresponding to the sub-computing units other than the sub-computing unit that issued the memory barrier request, after receiving the request forwarded by the corresponding first-level link module, set the first flag bit of the data at the corresponding address to a first value. This enables the corresponding sub-computing unit to initiate a read data request to the second-level cache module when performing a read operation, based on the first flag bit of the data being set to the first value. The second-level cache module, upon receiving the read data request, sends the requested data back to the corresponding first-level cache module.
5. The graphics processor as described in claim 1, characterized in that, When the request that requires maintaining data consistency is a refresh request, after receiving the request from the second-level link module forwarded by the corresponding first-level link module, the multiple first-level cache modules check whether there is cached data for the corresponding address segment. If so, they further check whether the data has been written locally. If so, they write the data for the corresponding address segment into the second-level cache module.
6. The graphics processor as described in claim 5, characterized in that, When the request that needs to maintain data consistency is a refresh request, the multiple first-level cache modules, when caching data for the corresponding address range, further check whether the second flag bit of the corresponding cache line is the second value. If the second flag bit of the corresponding cache line is the second value, the data of the cache line is written to the second-level cache module.
7. The graphics processor as claimed in claim 1, characterized in that, When a request that requires maintaining data consistency is invalid, the multiple first-level cache modules, after receiving a request forwarded from the second-level link module by the corresponding first-level link module, process the data at the corresponding address to invalidate that part of the data.
8. The graphics processor as described in claim 7, characterized in that, When a request that requires maintaining data consistency is invalid, the multiple first-level cache modules, after receiving a request from the second-level link module forwarded by the corresponding first-level link module, set the third flag of the data at the corresponding address to the third value.
9. The graphics processor as described in any one of claims 1 to 8, characterized in that, The multiple first-level link modules are also used to forward the request to the corresponding first-level cache module when the corresponding sub-computing unit issues a request that does not require maintaining data consistency.
10. The graphics processor as claimed in any one of claims 1 to 8, characterized in that, The plurality of first-level link modules are used to intercept the request and transmit it to the second-level link module when the corresponding sub-computing unit issues a request that needs to maintain data consistency, and to suspend receiving requests from the corresponding sub-computing unit; the first-level link module corresponding to the sub-computing unit that issued the request that needs to maintain data consistency resumes receiving requests from the sub-computing unit after receiving the request from the second-level link module.
11. A graphics processor, characterized in that, It includes multiple computing cores, each computing core including multiple sub-computing units, multiple first-level link modules, multiple first-level cache modules, second-level link modules, and second-level cache modules; in each computing core, the multiple sub-computing units, the multiple first-level link modules, and the multiple first-level cache modules correspond one-to-one, each first-level link module connects to the corresponding sub-computing unit and first-level cache module, the second-level link modules connect to the multiple first-level link modules respectively, and the second-level cache modules connect to the multiple first-level cache modules respectively; The graphics processor further includes a third-level link module and a third-level cache module; the third-level link module connects the third-level cache module and the second-level link modules of the plurality of computing cores, and the third-level cache module connects the third-level link module and the second-level cache modules of the plurality of computing cores; In each computing core, the multiple first-level link modules are used to intercept and transmit the request to the second-level link module when the corresponding sub-computing unit issues a request that needs to maintain data consistency; the second-level link module is used to determine whether the request that needs to maintain data consistency is a local request or a global request, and if the request is determined to be a global request, it transmits the request to the third-level link module. The third-level link module is used to broadcast the request to the multiple first-level link modules of the multiple computing cores; each first-level link module is also used to forward the request to the corresponding first-level cache module after receiving the request from the third-level link module; after receiving the request forwarded by the corresponding first-level link module, the multiple first-level cache modules of the multiple computing cores perform corresponding operations to maintain the data consistency of the graphics processor.
12. The graphics processor as claimed in claim 11, characterized in that, When the request that needs to maintain data consistency is a memory barrier request, the first-level cache module corresponding to the sub-computing unit that issued the memory barrier request writes the data of the corresponding address segment into the corresponding second-level cache module after receiving the request forwarded by the corresponding first-level link module, and then further writes it into the third-level cache module.
13. The graphics processor as claimed in claim 12, characterized in that, In the graphics processor, the first-level cache module corresponding to the other sub-computing units besides the sub-computing unit that issued the memory barrier request processes the data at the corresponding address after receiving the request forwarded by the corresponding first-level link module, so that the corresponding sub-computing unit reads the data at the corresponding address from the third-level cache module when performing a read operation.
14. The graphics processor as claimed in claim 11, characterized in that, When the request that requires maintaining data consistency is a refresh request, the multiple first-level cache modules of the multiple computing cores, after receiving the request forwarded by the corresponding first-level link module, check whether there is cached data for the corresponding address segment. If so, they further check whether the data has been written locally. If so, they write the data for the corresponding address segment into the third-level cache module.
15. The graphics processor as claimed in claim 11, characterized in that, When a request that requires maintaining data consistency is invalid, the multiple first-level cache modules of the multiple computing cores, upon receiving the request forwarded by the corresponding first-level link module, process the data at the corresponding address to invalidate that part of the data.
16. The graphics processor as claimed in any one of claims 11 to 15, characterized in that, In each computing core, when the second-level link module confirms that the request to maintain data consistency is a local request, it broadcasts the request to the multiple first-level link modules in the computing core. The multiple first-level link modules in the computing core are also used to forward the request to the corresponding first-level cache module after receiving the request from the second-level link module. After receiving the request forwarded by the corresponding first-level link module, the multiple first-level cache modules in the computing core perform corresponding operations to maintain the data consistency of the computing core.