Network-on-chip congestion control method and system

Abstract

The invention provides a network-on-chip congestion control method and system, and belongs to the technical field of network-on-chip, and the method comprises the steps: responding to any memory reading request, and obtaining current response data; when it is judged that the current response data is approximable data according to the current approximation flag bit of the current response data, any target entry data in a preset output cache table is compared with the current response data, and a precision difference is obtained; when it is judged that the approximate entry data exists, setting a target bit position in the flag bit field, and discarding the current response data; when it is judged that the approximate entry data does not exist, the current response data serves as new target entry data to be stored in an output cache table; and packaging the target entry data into a data packet, and sending the data packet to one or more streaming multiprocessors. According to the method and the device, a plurality of independent memory reading responses aiming at approximate data are converted into single data responses sent through multicast, so that the overall performance of the general parallel computing processor is improved.

Description

Technical Field

[0001] This invention relates to the field of on-chip network technology, and in particular to an on-chip network congestion control method and system. Background Technology

[0002] General-purpose parallel computing architectures, particularly those represented by general-purpose graphics processing units (GPGPUs), have become the mainstream in the field of high-performance computing. These architectures achieve large-scale data-parallel computing by integrating dozens or even hundreds of streaming multiprocessors (SMs), thus providing powerful computing support for applications such as scientific computing and artificial intelligence. To support the data supply for these parallel processing units, this architecture typically relies on a high-bandwidth memory system connected by a network-on-chip (NoC).

[0003] In current general-purpose parallel computing architectures, the standard memory access process is as follows: when a streaming multiprocessor needs to access data in global memory, it generates a memory read request and sends it to the memory controller (MC) that manages the corresponding memory address partition via the on-chip network. Upon receiving the request, the memory controller retrieves the data from external Dynamic Random Access Memory (DRAM) and then sends the response data packet containing that data back to the original streaming multiprocessor via the on-chip network. However, because the number of streaming multiprocessors in the architecture far exceeds the number of memory controllers, under high load, a large number of streaming multiprocessors concurrently send memory access requests to a few memory controllers. This results in multiple memory controllers needing to inject a large number of response packets back to the on-chip network within a very short time. Since the physical interface bandwidth of each memory controller connecting to the on-chip network is limited, it cannot send all response packets simultaneously. This creates severe network congestion at the memory controller's output, leading to low utilization of high-bandwidth memory and directly limiting the computational efficiency of the streaming multiprocessors, resulting in poor overall performance of the general-purpose parallel computing processor. Summary of the Invention

[0004] This invention provides an on-chip network congestion control method, system, electronic device, storage medium, and computer program product to address the shortcomings of low bandwidth and memory utilization and low computational efficiency of streaming multiprocessors in the prior art, thereby improving the overall performance of general-purpose parallel computing processors.

[0005] This invention provides an on-chip network congestion control method, comprising the following steps: In response to any memory read request, obtain the current response data; the current response data includes the current approximate flag bit; When the current response data is determined to be approximate based on the current approximation flag, any target entry data in the preset output cache table is compared with the current response data to obtain the precision difference; wherein, the target entry data includes a target approximation flag and a flag field, the target approximation flag is the same as the current approximation flag, and the flag field includes multiple bits, each bit corresponding to the identifier of a stream multiprocessor; When it is determined that there is approximate entry data, the target bit in the flag field is set to the bit position, and the current response data is discarded; the approximate entry data is the target entry data whose precision difference is less than a preset threshold, and the target bit is the bit corresponding to the identifier of the stream multiprocessor to which the current response data belongs; When it is determined that no approximate entry data exists, the current response data is stored as the new target entry data in the output cache table; The data of each target entry is packaged into a data packet, and the data packet is sent to one or more streaming multiprocessors.

[0006] According to an on-chip network congestion control method provided by the present invention, when the bit position is 1, it is used to indicate that the target entry data belongs to the corresponding identifier of the stream multiprocessor; when the bit position is 0, it is used to indicate that the target entry data does not belong to the corresponding identifier of the stream multiprocessor.

[0007] According to an on-chip network congestion control method provided by the present invention, when it is determined that there is approximate entry data, setting the target bit in the flag field and discarding the current response data includes: Each target entry data is compared with the current response data in turn until it is determined that there is an approximate entry data whose accuracy difference is less than a preset threshold; Set the target bit position of the approximate entry data to 1.

[0008] According to the on-chip network congestion control method provided by the present invention, after storing the current response data as new target entry data in the output cache table when it is determined that no approximate entry data exists, the method further includes: Set the bit position corresponding to the identifier of the stream multiprocessor to which the current response data belongs in the flag field of the new target entry data to 1.

[0009] According to an on-chip network congestion control method provided by the present invention, the step of packaging each target entry data into a data packet and sending the data packet to one or more stream multiprocessors includes: Based on the status of the on-chip network transmit queue, schedule the target entry data from the output buffer table; The target entry data is packaged into a data packet, and the identifier of the stream multiprocessor corresponding to the bit set to 1 is marked in the header of the data packet according to the flag field. Based on the packet header, the data packet is sent to the corresponding stream multiprocessor via multicast.

[0010] The on-chip network congestion control method provided by the present invention further includes: Get memory request; When the memory request is a memory read request, the memory read request is sent to the arbitrator; The memory read request is processed by the arbitrator. When the arbitration result is a memory read request, the memory read request and read address are sent to the L2 cache, and the L2 cache reads the current response data from the internal cache or external dynamic random access memory.

[0011] The on-chip network congestion control method provided by the present invention further includes: Get memory request; When the memory request is a memory write request, the memory write request is written to the write buffer and then sent to the arbitrator; The memory write request is processed by the arbitrator. When the arbitration result is a memory write request, the memory write request and the data in the write buffer are sent to the L2 cache together, and the L2 cache writes the data to the internal cache or external dynamic random access memory according to the write address.

[0012] According to an on-chip network congestion control method provided by the present invention, the step of obtaining a memory request includes: Get the memory access requests of the streaming multiprocessor to the L2 cache; The memory access request is converted into a memory request and placed into the on-chip network receive queue; When it is determined that the output cache table has free resources, a memory request is dequeued from the on-chip network receive queue.

[0013] The present invention also provides an on-chip network congestion control system, comprising the following modules: The first processing module is used to respond to any memory read request and obtain the current response data; the current response data includes the current approximate flag bit; The second processing module is used to compare any target entry data in the preset output cache table with the current response data when the current response data is determined to be approximate based on the current approximation flag bit, and obtain the precision difference; wherein, the target entry data includes a target approximation flag bit and a flag bit field, the target approximation flag bit is the same as the current approximation flag bit, and the flag bit field includes multiple bits, each bit bit corresponding to the identifier of a stream multiprocessor; The third processing module is used to set the target bit in the flag field and discard the current response data when it is determined that there is approximate entry data; the approximate entry data is the target entry data whose precision difference is less than a preset threshold, and the target bit is the bit corresponding to the identifier of the stream multiprocessor to which the current response data belongs; The third processing module is also used to store the current response data as new target entry data into the output cache table when it is determined that no approximate entry data exists; The fourth processing module is used to package the data of each target entry into a data packet and send the data packet to one or more stream multiprocessors.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the on-chip network congestion control methods described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the on-chip network congestion control method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the on-chip network congestion control method as described above.

[0017] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: First, the response data and its key approximation flags were acquired, providing the necessary prerequisites for subsequent differential processing. Next, by comparing the approximate current response data with the target entry data in the output cache table, a quantified precision difference was provided to determine whether merging was possible. When approximate entry data was determined to exist, the flag field was updated and the merging operation of the current response data was discarded, directly merging multiple memory response requests into one, reducing the number of data packets injected into the on-chip network from the source. For data that could not be merged, it was stored as new target entry data in the cache table, ensuring the integrity of data processing and the correctness of calculations. Finally, the efficiency of a single data transmission was maximized by packaging the target entry data and sending it using multicast. Overall, this solution significantly reduced response network traffic from the memory controller to the streaming multiprocessor, effectively alleviating on-chip network congestion, improving bandwidth utilization and the computational efficiency of the streaming multiprocessor, thereby enhancing the overall performance of the general-purpose parallel computing processor. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts of the on-chip network congestion control method provided by the present invention.

[0020] Figure 2 This is the second flowchart of the on-chip network congestion control method provided by the present invention.

[0021] Figure 3 This is the third flowchart of the on-chip network congestion control method provided by the present invention.

[0022] Figure 4 This is the fourth flowchart of the on-chip network congestion control method provided by the present invention.

[0023] Figure 5 This is the fifth flowchart of the on-chip network congestion control method provided by the present invention.

[0024] Figure 6 This is the sixth flowchart of the on-chip network congestion control method provided by the present invention.

[0025] Figure 7 This is the seventh flowchart of the on-chip network congestion control method provided by the present invention.

[0026] Figure 8 This is a schematic diagram of the on-chip network congestion control system provided by the present invention.

[0027] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] It should be noted that in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terms "upper," "lower," etc., indicating orientation or positional relationships according to the accompanying drawings, are only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the system or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0030] The terms "first," "second," etc., used in this invention are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0031] The following is combined Figures 1-9 This invention describes the on-chip network congestion control method, system, electronic device, storage medium, and computer program product provided by the present invention.

[0032] Reference Figure 1 , Figure 1 This is one of the flowcharts illustrating the on-chip network congestion control method provided by the present invention, such as... Figure 1 As shown, steps 101 to 105 are included: Step 101: Respond to any memory read request and obtain the current response data; the current response data includes the current approximate flag.

[0033] In one specific embodiment of the present invention, the first step is to respond to any memory read request and obtain the current response data; the current response data includes the current approximate flag. In general-purpose parallel computing architectures, a large number of concurrent memory requests put enormous pressure on the memory controller and cause congestion in the Network-on-Chip (NoC). To effectively alleviate this congestion, it is necessary to screen and process the response data returned from the memory system.

[0034] In this embodiment, the current response data refers to the actual data read from the L2 cache or external Dynamic Random Access Memory (DRAM) based on the address contained in the memory read request of the Streaming Multiprocessor (SM). The current approximation flag is a flag used to characterize whether the current response data is allowed to be approximated. The value of this flag is determined during the compilation phase and accompanies the memory request issued from the Streaming Multiprocessor, transmitted through the on-chip network, and finally to the memory controller. For example, the approximation flag can be a single bit; when its value is 1, it indicates that the current response data is approximable; when its value is 0, it indicates that the current response data is not approximable, and its calculation must guarantee the original precision.

[0035] One specific implementation of this step is as follows: First, the stream multiprocessor obtains memory access requests to the L2 cache, converts these requests into memory requests, and places them in the on-chip network receive queue. After determining that the output cache table has free resources, a memory request is dequeued from the on-chip network receive queue. When this memory request is determined to be a memory read request, it is sent to an arbitrator for processing. After the arbitrator determines to process the memory read request, it sends the memory read request and the corresponding read address to the L2 cache. Finally, the L2 cache completes the data read based on the read address, thereby obtaining the current response data containing the current approximation flag.

[0036] Step 102: When the current response data is determined to be approximate based on the current approximation flag, any target entry data in the preset output cache table is compared with the current response data to obtain the precision difference; wherein, the target entry data includes a target approximation flag and a flag field, the target approximation flag is the same as the current approximation flag, and the flag field includes multiple bits, each bit corresponding to the identifier of a stream multiprocessor.

[0037] After obtaining the current response data, subsequent operations need to be determined based on its attributes. For data that can be approximated, network traffic can be reduced by merging data responses with similar content. Performing a comparison and obtaining the precision difference is a necessary prerequisite for determining whether the data is similar and whether it can be merged.

[0038] In this embodiment, the preset output buffer table is a data structure located within the memory controller, used to temporarily store response data to be sent to the on-chip network. Each row in the table constitutes a target entry data, representing the content of a data packet to be sent. The flag field is a component of the target entry data, recording which streaming multiprocessors have requested the data in that entry. This field consists of multiple bits, the total number of which is equal to the total number of streaming multiprocessors in the architecture, and each bit uniquely corresponds to the identifier of a streaming multiprocessor. Precision difference is a quantification of the numerical difference between the current response data and the target entry data.

[0039] The specific implementation of this step is as follows: First, determine the current approximation flag of the current response data. The comparison process is initiated only when this flag indicates that the data is approximable. Next, access the output cache table and select a target entry data from it. The selection condition is that the target approximation flag of the target entry data must also be approximable. Subsequently, compare the data value stored in the target entry data with the data value of the current response data to calculate the precision difference between the two.

[0040] Step 103: When it is determined that there is approximate entry data, the target bit in the flag field is set to the bit position and the current response data is discarded; the approximate entry data is the target entry data with a precision difference of less than a preset threshold, and the target bit is the bit corresponding to the identifier of the stream multiprocessor to which the current response data belongs.

[0041] When a newly returned data instance is found to be sufficiently similar to existing data in the cache, there is no need to create a separate data packet for this new data. Multiple data responses can be merged into one by updating the destination information of the existing entry and discarding the new data, thereby directly reducing the number of data packets injected into the on-chip network. Therefore, after calculating the precision difference, a merging decision needs to be made based on this difference.

[0042] In this embodiment, the preset threshold is a pre-defined critical value used to measure data similarity, which defines the maximum error range that the approximate calculation can tolerate. The approximate entry data refers to the target entry data found in the output cache table whose data value differs from the precision of the current response data by less than the preset threshold. The target bit is the bit in the flag field of the approximate entry data whose position corresponds to the identifier of the streaming multiprocessor that initiated this memory read request.

[0043] One specific implementation of this step is as follows: compare the precision difference obtained in the previous step with a preset threshold. If the precision difference is less than the preset threshold, the target entry data being compared is determined to be approximate entry data. At this point, an update operation is performed on the approximate entry data, that is, find the target bit in its flag field and set the value of that bit, for example, from 0 to 1. After completing the setting operation, no further storage or forwarding processing will be performed on the current response data; instead, it will be directly discarded.

[0044] Step 104: When it is determined that there is no similar entry data, store the current response data as the new target entry data in the output cache table.

[0045] If an approximate response data has a precision difference exceeding the allowable range compared to all existing approximate data in the output cache table, or if the response data itself is not approximate, then that data cannot be merged. In this case, a new entry must be created for it to ensure that it can be sent independently to the requester, thereby guaranteeing the correctness of the calculation.

[0046] In this embodiment, the absence of approximate entry data is a determination result, indicating that no target entry data with a precision difference of less than a preset threshold from the current response data could be found in the output cache table. The new target entry data is a completely new entry created in the output cache table based on the current response data and its related information; it will become the basis for subsequent data packet transmission.

[0047] One specific implementation of this step is as follows: After traversing and comparing all target entry data that meet the conditions in the output buffer table, if the determination results are all that the precision difference is not less than a preset threshold, it is confirmed that there is no approximate entry data. At this time, an available blank position is found in the output buffer table, and the current response data and its associated attributes, such as the current approximation flag and its corresponding stream multiprocessor identifier, are written into the blank position to form a new target entry data.

[0048] Step 105: Pack the data of each target entry into a data packet and send the data packet to one or more streaming multiprocessors.

[0049] After storing or updating the target entry data in the output cache table, this data must be passed to the corresponding requester to ensure that all streaming multiprocessors that initiated the memory request can receive the data they need, so that they can continue to execute subsequent computing tasks.

[0050] In this embodiment, a data packet is a basic unit constructed from target entry data in the output cache table for transmission in the on-chip network. One or more stream multiprocessors refer to the destination of the data packet. When a target entry data corresponds to a single original memory request, its destination is a stream multiprocessor; while when a target entry data is the result of approximate merging, it corresponds to multiple original memory requests, and its destination is multiple stream multiprocessors.

[0051] One specific implementation of this step is as follows: A target entry data is retrieved from the output buffer table, and the data content stored in that entry is encapsulated into a standard data packet. During encapsulation, the flag field in the target entry data is parsed to determine which streaming multiprocessors the data packet needs to be sent to. Finally, the routing and forwarding function of the on-chip network is used to transmit the data packet to all identified destination streaming multiprocessors.

[0052] In summary, steps 101 to 105 first obtain the response data and its key approximation flags, providing the necessary prerequisites for subsequent differential processing. Next, by comparing the approximate current response data with the target entry data in the output cache table, a quantified precision difference is provided to determine whether merging is possible. When approximate entry data is determined to exist, the flag field is updated and the merging operation of the current response data is discarded, directly merging multiple memory response requests into one, reducing the number of data packets injected into the on-chip network from the source. For data that cannot be merged, it is stored as new target entry data in the cache table, ensuring the integrity of data processing and the correctness of calculation. Finally, by packaging the target entry data and sending it using multicast, the efficiency of a single data transmission is maximized. Overall, this solution significantly reduces response network traffic from the memory controller to the streaming multiprocessor, effectively alleviating on-chip network congestion, improving bandwidth utilization, thereby reducing memory bottlenecks, and ultimately improving the overall performance of the general-purpose parallel computing processor.

[0053] In one specific embodiment of the present invention, the implementation of the flag field is further defined. When the bit is 1, it indicates that the target entry data belongs to the stream multiprocessor of the corresponding identifier; when the bit is 0, it indicates that the target entry data does not belong to the stream multiprocessor of the corresponding identifier.

[0054] Specifically, the flag field consists of multiple bits, the total number of which corresponds to the total number of streaming multiprocessors in a general-purpose parallel computing architecture. Each bit in the field has a fixed mapping relationship with the unique identifier of a streaming multiprocessor. When a bit in the flag field is 1, it indicates that the target entry data corresponding to that bit was requested by the streaming multiprocessor whose identifier corresponds to that bit. Conversely, when a bit is 0, it indicates that the target entry data does not belong to the corresponding streaming multiprocessor. For example, in an architecture with 16 streaming multiprocessors, the flag field can consist of 16 bits, where bit 0 corresponds to the streaming multiprocessor with identifier 0, bit 1 corresponds to the streaming multiprocessor with identifier 1, and so on. If the value of bit 5 is 1, it indicates that the streaming multiprocessor with identifier 5 requested this entry data. By adopting this method, all destinations corresponding to a data response can be clearly recorded and managed in a highly compact and efficient manner, providing accurate and direct routing information for subsequent multicast transmissions.

[0055] In one specific embodiment of the present invention, the process of determining and performing approximate merging is defined in detail. Step 103 specifically includes the following steps: Step 201: Compare each target entry data with the current response data in turn until it is determined that there is approximate entry data with an accuracy difference less than a preset threshold.

[0056] Step 202: Set the target bit position of the approximate entry data to 1.

[0057] In this embodiment, "comparing each target entry data with the current response data sequentially" refers to a sequential traversal and comparison process. This process accesses each target entry data in the output cache table in sequence and performs precision difference calculation. "Until existence is determined" indicates that this traversal process does not always need to complete a scan of the entire table; once the first matching approximate entry data is found, the search process can be terminated early. Setting the target bit to 1 utilizes the implementation method of the flag field described above. By modifying the value of the bit representing the stream multiprocessor to which the current response data belongs to, a new destination is added to the distribution list of the approximate entry data.

[0058] The specific implementation process of this method is as follows: After obtaining an approximate current response data, a traversal process is initiated, starting from the first entry in the output cache table. First, it is determined whether the entry is also approximate data. If so, the precision difference between its data and the current response data is calculated and compared with a preset threshold. If the precision difference is less than the preset threshold, the entry is determined to be approximate entry data, and the traversal is stopped immediately. Next, based on the stream multiprocessor identifier to which the current response data belongs, the corresponding target bit is located in the flag field of the approximate entry data, and the value of this bit is set to 1. If the precision difference is not less than the preset threshold, or the current entry is unapproximate data, the traversal continues to the next entry, repeating the above comparison process until an approximate entry data is found or all entries have been traversed.

[0059] In one specific embodiment of the present invention, the process of creating new target entry data is defined in detail. In one possible embodiment, step 104 specifically includes the following steps: Step 301: Set the bit position corresponding to the identifier of the stream multiprocessor to which the current response data belongs in the flag field of the new target entry data to 1.

[0060] The specific implementation process of this method is as follows: After determining that no approximate entry data exists, a usable blank entry is first allocated in the output cache table, and the current response data and its approximation flag are written into it. Simultaneously, the unique identifier of the streaming multiprocessor that initiated the memory request is obtained. Based on this identifier, the bit corresponding to the identifier is found in the flag field of the newly created target entry data, and its value is set to 1. For all other bits in the flag field of this new entry, their values ​​are kept at their initial 0 state.

[0061] In one specific embodiment of the present invention, the process of finally sending the data in the output buffer table to the streaming multiprocessor is defined in detail. Step 105 specifically includes the following steps: Step 401: Based on the status of the on-chip network transmit queue, schedule the target entry data from the output buffer table.

[0062] Step 402: Pack the target entry data into a data packet, and mark the identifier of the stream multiprocessor corresponding to the bit set to 1 in the packet header according to the flag field.

[0063] Step 403: Based on the packet header, send the data packet to the corresponding stream multiprocessor via multicast.

[0064] The specific implementation process of this method is as follows: First, the memory controller continuously monitors the status of the on-chip network's transmit queue. When there are idle resources in the transmit queue, a scheduling operation is triggered to retrieve a target entry data from the output buffer table. After obtaining the target entry data, the packetization stage begins. In this stage, the data content in the entry is read, and a data packet is created. Simultaneously, the flag field of the entry is parsed, and all stream multiprocessor identifiers corresponding to bits with a value of 1 are recorded in the packet header to indicate the destination. After the data packet is constructed, it is sent to the on-chip network's transmit queue. Upon receiving the data packet, the on-chip network's routing hardware parses its header and identifies the multiple destination identifiers contained within. Finally, the network hardware performs a multicast operation, copying the data packet and forwarding it to all the corresponding stream multiprocessors identified in the header.

[0065] By implementing this method, firstly, the scheduling step ensures that the injection of data packets matches the network's capacity, avoiding the exacerbation of network congestion due to indiscriminate sending. Secondly, the packetization and multicasting steps are closely integrated, completely translating the effect of approximate merging into actual network bandwidth savings. By identifying multiple destinations in a single packet header and sending it via multicast, multiple memory requests are responded to in a single network transmission. This significantly reduces the total number of data packets injected into the network, saves network bandwidth, and alleviates the load and contention on the on-chip network router, thereby achieving the final distribution of data responses with maximum efficiency and directly achieving the technical effects of alleviating network congestion and improving processor performance.

[0066] In one specific embodiment of the present invention, the upstream process for obtaining the current response data is defined in detail. Before step 101, the method further includes the following steps: Step 501: Obtain memory request.

[0067] Step 502: When the memory request is a memory read request, send the memory read request to the arbitrator.

[0068] Step 503: Process memory read requests via the arbitrator.

[0069] Step 504: When the arbitration result is a memory read request, the memory read request and read address are sent to the L2 cache, and the L2 cache reads the current response data from the internal cache or external dynamic random access memory.

[0070] In this implementation, the arbitrator is a decision-making unit used to resolve resource access conflicts. In the memory controller, memory read requests and memory write requests compete for access to the L2 cache. The role of the arbitrator is to fairly and orderly determine which request can continue to execute at any given time, based on a preset scheduling policy, thereby managing the traffic accessing the L2 cache.

[0071] The specific implementation process of this method can be further refined as follows: A memory access request from a streaming multiprocessor arrives at the memory controller node in the form of a data packet via the on-chip network. The node's on-chip network interface receives the data packet and parses its header and payload, extracting information such as the request type, memory address, source streaming multiprocessor identifier, and approximation flag. This information is then assembled into a structured memory request. The memory request is then placed in a first-in-first-out (FIFO) on-chip network receive queue to wait. The memory controller continuously monitors the status of the output cache table. Only when it determines that there is a usable blank entry in the output cache table will a memory request be dequeued from the head of the on-chip network receive queue. This is the specific action of acquiring a memory request. Next, the type of the memory request is determined. If it is a memory read request, it is sent to an input port of the arbitrator. The arbitrator selects a request from multiple input ports (including ports from memory write requests) for processing according to its internal scheduling strategy, such as round-robin or priority strategy. When the memory read request is selected by the arbitrator, the arbitrator grants it access to the L2 cache. At this point, the memory read request and its included read address are formally sent to the L2 cache. The L2 cache controller, based on the read address, first searches its internal cache tags. If the search is successful, the corresponding data is directly and quickly read from its internal cache; if the search fails, the L2 cache controller initiates a new read operation from external dynamic random access memory, retrieving the data after a longer memory access latency. Regardless of whether the data originates from the internal cache or external memory, the L2 cache ultimately sends the read data, along with its original approximation flag, as the current response data to the subsequent approximation merging process.

[0072] In one specific embodiment of the present invention, the processing flow for memory write requests is also defined in detail. Before step 101, the method further includes the following steps: Step 601: Obtain memory request.

[0073] Step 602: When the memory request is a memory write request, write the memory write request to the write buffer and then send it to the arbitrator.

[0074] Step 603: Process memory write requests via the arbitrator.

[0075] Step 604: When the arbitration result is a memory write request, the memory write request and the data in the write buffer are sent to the L2 cache together, and the L2 cache writes the data to the internal cache or external dynamic random access memory according to the write address.

[0076] In this implementation, the write buffer is a dedicated temporary storage space within the memory controller. Its function is to temporarily store data about to be written to the L2 cache or dynamic random access memory, along with its corresponding write address and other information. By introducing the write buffer, the action of receiving write requests is decoupled from the actual write operation, allowing the memory controller to quickly respond to write operations from streaming multiprocessors without waiting for the slower backend write process to complete.

[0077] The specific implementation process of this method can be further refined as follows: First, the memory controller obtains a memory request from the on-chip network receive queue and determines its type. If it is determined to be a memory write request, the write address, the data to be written, and the source multiprocessor identifier contained in the request are stored as a complete entry in the write buffer. After the data is successfully written to the buffer, a request handle (or pointer) representing the write operation is sent to a dedicated input port of the arbitrator to request access to the L2 cache. During its scheduling cycle, the arbitrator selects between memory read requests and memory write requests according to its internal policy. When the arbitrator finally chooses to process the memory write request and grants it access, the memory controller reads the previously stored complete write address and data corresponding to the request handle from the write buffer according to the arbitration result. Then, the write address and data are sent to the L2 cache together. After receiving the write request, the L2 cache controller performs a write operation according to the write address, such as updating the data of the corresponding cache line in its internal cache, and decides whether and when to further write the data to the external dynamic random access memory according to the cache policy.

[0078] In one specific embodiment of the present invention, the initial acquisition phase of a memory request is defined in detail. Acquiring a memory request specifically includes the following steps: Step 701: Obtain the memory access requests of the streaming multiprocessor to the L2 cache.

[0079] Step 702: Convert the memory access request into a memory request and put it into the on-chip network receive queue.

[0080] Step 703: When it is determined that there are free resources in the output buffer table, dequeue a memory request from the on-chip network receive queue.

[0081] In this implementation, a memory access request refers to a raw request generated by the streaming multiprocessor and transmitted in the form of data packets in the on-chip network. A memory request is a standardized data structure generated and used internally by the memory controller after the memory access request is received and parsed. The on-chip network receive queue is a first-in-first-out (FIFO) buffer queue located at the memory controller interface. It is used to temporarily store requests that have been converted into memory requests but have not yet been processed by internal logic, thus smoothing bursts of network traffic and decoupling the speed of network reception from internal processing.

[0082] The specific implementation process of this method can be further refined as follows: When a thread in a streaming multiprocessor executes a memory access instruction and its L1 cache misses, the hardware generates a memory access request. This memory access request is encapsulated into one or more network packets. The packet header contains the address of the destination memory controller, the identifier of the source streaming multiprocessor, the request type (read or write), the memory address, and approximate flags determined during the compilation phase. These packets are routed to the designated memory controller node via the on-chip network. The node's network interface hardware receives these packets, reassembles any fragmented packets, and then parses all the information within the packets. After parsing, the hardware assembles this discrete information (address, identifier, flags, etc.) into a fixed-format internal data structure, i.e., a memory request, and pushes this memory request to the tail of the on-chip network receive queue. Simultaneously, the memory controller's control logic continuously monitors the usage of its output buffer table. When an entry in the output buffer table is successfully scheduled for transmission, freeing up a free entry, the control logic determines that the output buffer table has free resources. This determination serves as an enable signal, allowing a memory request to be retrieved from the head of the on-chip network receive queue. The delisted memory request is then sent to subsequent processing stages such as request type determination and arbitration.

[0083] Reference Figure 8 , Figure 8 This is a schematic diagram of the on-chip network congestion control system provided by the present invention. The system includes: The first processing module is used to respond to any memory read request and obtain the current response data; the current response data includes the current approximate flag bit; The second processing module is used to compare any target entry data in the preset output cache table with the current response data when the current response data is determined to be approximate based on the current approximation flag bit, and obtain the precision difference; wherein, the target entry data includes a target approximation flag bit and a flag bit field, the target approximation flag bit is the same as the current approximation flag bit, and the flag bit field includes multiple bits, each bit corresponding to the identifier of a stream multiprocessor; The third processing module is used to set the target bit in the flag field and discard the current response data when it is determined that there is approximate entry data; the approximate entry data is the target entry data with a precision difference of less than a preset threshold, and the target bit is the bit corresponding to the identifier of the stream multiprocessor to which the current response data belongs; The third processing module is also used to store the current response data as the new target entry data in the output cache table when it is determined that there is no similar entry data. The fourth processing module is used to package the data of each target entry into a data packet and send the data packet to one or more stream multiprocessors.

[0084] In one possible implementation, the third processing module is further configured to: The data of each target item is compared with the current response data in turn until it is determined that there is an approximate item data with a precision difference of less than a preset threshold. Set the target bit position of the approximate entry data to 1.

[0085] In one possible implementation, the third processing module is further configured to set the bit position corresponding to the identifier of the stream multiprocessor to which the current response data belongs in the flag field of the new target entry data to 1.

[0086] In one possible implementation, the fourth processing module is further configured to: Based on the status of the on-chip network transmit queue, schedule the target entry data from the output buffer table; Pack the target entry data into a data packet, and mark the identifier of the stream multiprocessor corresponding to the bit set to 1 in the packet header according to the flag field; Based on the packet header, the data packets are sent to the corresponding stream multiprocessor via multicast.

[0087] In one possible implementation, the first processing module is further configured to: Get memory request; When the memory request is a memory read request, the memory read request is sent to the arbitrator; Memory read requests are handled by an arbitrator; When the arbitration result is a memory read request, the memory read request and read address are sent to the L2 cache, and the L2 cache reads the current response data from the internal cache or external dynamic random access memory.

[0088] In one possible implementation, the first processing module is further configured to: Get memory request; When the memory request is a memory write request, the memory write request is written to the write buffer and then sent to the arbitrator; Memory write requests are handled by an arbitrator; When the arbitration result is a memory write request, the memory write request and the data in the write buffer are sent to the L2 cache together, and the L2 cache writes the data to the internal cache or external dynamic random access memory according to the write address.

[0089] In one possible implementation, the first processing module is further configured to: Get the memory access requests of the streaming multiprocessor to the L2 cache; The memory access request is converted into a memory request and placed into the on-chip network receive queue; When it is determined that there are free resources in the output buffer table, a memory request is dequeued from the on-chip network receive queue.

[0090] It should be noted that the on-chip network congestion control system provided by the present invention can execute the on-chip network congestion control method of any of the above embodiments during actual operation, which will not be elaborated in this embodiment.

[0091] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930, and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other through the communication bus 940. The processor 910 can call logic instructions in the memory 930 to execute an on-chip network congestion control method. This method includes: responding to any memory read request and obtaining current response data; the current response data includes a current approximation flag; when the current response data is determined to be approximate based on the current approximation flag, comparing any target entry data in a preset output buffer table with the current response data to obtain the precision difference; wherein the target entry data includes a target approximation flag and a flag field, the target approximation flag being the same as the current approximation flag, and the flag field including multiple bits, each bit corresponding to the identifier of a streaming multiprocessor; when it is determined that approximate entry data exists, setting the target bit in the flag field and discarding the current response data; the approximate entry data is target entry data with a precision difference less than a preset threshold, and the target bit is the bit corresponding to the identifier of the streaming multiprocessor to which the current response data belongs; when it is determined that no approximate entry data exists, storing the current response data as new target entry data in the output buffer table; packaging each target entry data into a data packet and sending the data packet to one or more streaming multiprocessors.

[0092] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer is able to execute the on-chip network congestion control method provided in the above embodiments.

[0094] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the on-chip network congestion control method provided in the above embodiments.

[0095] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for congestion control in an on-chip network, characterized in that, include: Respond to any memory read request and obtain the current response data; The current response data includes the current approximate flag bit; When the current response data is determined to be approximate based on the current approximation flag, any target entry data in the preset output cache table is compared with the current response data to obtain the precision difference; wherein, the target entry data includes a target approximation flag and a flag field, the target approximation flag is the same as the current approximation flag, and the flag field includes multiple bits, each bit corresponding to the identifier of a stream multiprocessor; When it is determined that there is approximate entry data, the target bit in the flag field is set to the bit position, and the current response data is discarded; the approximate entry data is the target entry data whose precision difference is less than a preset threshold, and the target bit is the bit corresponding to the identifier of the stream multiprocessor to which the current response data belongs; When it is determined that no approximate entry data exists, the current response data is stored as the new target entry data in the output cache table; The data of each target entry is packaged into a data packet, and the data packet is sent to one or more streaming multiprocessors.

2. The on-chip network congestion control method according to claim 1, characterized in that, When the bit position is 1, it indicates that the target entry data belongs to the corresponding stream multiprocessor; when the bit position is 0, it indicates that the target entry data does not belong to the corresponding stream multiprocessor.

3. The on-chip network congestion control method according to claim 2, characterized in that, When it is determined that there is similar entry data, setting the target bit in the flag field and discarding the current response data includes: Each target entry data is compared with the current response data in turn until it is determined that there is an approximate entry data whose accuracy difference is less than a preset threshold; Set the target bit position of the approximate entry data to 1.

4. The on-chip network congestion control method according to claim 2, characterized in that, After storing the current response data as new target entry data in the output cache table when it is determined that no approximate entry data exists, the method further includes: Set the bit position corresponding to the identifier of the stream multiprocessor to which the current response data belongs in the flag field of the new target entry data to 1.

5. The on-chip network congestion control method according to claim 2, characterized in that, The step of packaging the data of each target entry into a data packet and sending the data packet to one or more streaming multiprocessors includes: Based on the status of the on-chip network transmit queue, schedule the target entry data from the output buffer table; The target entry data is packaged into a data packet, and the identifier of the stream multiprocessor corresponding to the bit set to 1 is marked in the header of the data packet according to the flag field. Based on the packet header, the data packet is sent to the corresponding stream multiprocessor via multicast.

6. The on-chip network congestion control method according to claim 1, characterized in that, Also includes: Get memory request; When the memory request is a memory read request, the memory read request is sent to the arbitrator; The memory read request is processed by the arbitrator. When the arbitration result is a memory read request, the memory read request and read address are sent to the L2 cache, and the L2 cache reads the current response data from the internal cache or external dynamic random access memory.

7. The on-chip network congestion control method according to claim 1, characterized in that, Also includes: Get memory request; When the memory request is a memory write request, the memory write request is written to the write buffer and then sent to the arbitrator; The memory write request is processed by the arbitrator. When the arbitration result is a memory write request, the memory write request and the data in the write buffer are sent to the L2 cache together, and the L2 cache writes the data to the internal cache or external dynamic random access memory according to the write address.

8. The on-chip network congestion control method according to claim 6 or 7, characterized in that, The memory acquisition request includes: Get the memory access requests of the streaming multiprocessor to the L2 cache; The memory access request is converted into a memory request and placed into the on-chip network receive queue; When it is determined that the output cache table has free resources, a memory request is dequeued from the on-chip network receive queue.

9. An on-chip network congestion control system, characterized in that, include: The first processing module is used to respond to any memory read request and obtain the current response data; The current response data includes the current approximate flag bit; The second processing module is used to compare any target entry data in the preset output cache table with the current response data when the current response data is determined to be approximate based on the current approximation flag bit, and obtain the precision difference; wherein, the target entry data includes a target approximation flag bit and a flag bit field, the target approximation flag bit is the same as the current approximation flag bit, and the flag bit field includes multiple bits, each bit bit corresponding to the identifier of a stream multiprocessor; The third processing module is used to set the target bit in the flag field and discard the current response data when it is determined that there is approximate entry data; the approximate entry data is the target entry data whose precision difference is less than a preset threshold, and the target bit is the bit corresponding to the identifier of the stream multiprocessor to which the current response data belongs; The third processing module is also used to store the current response data as new target entry data into the output cache table when it is determined that no approximate entry data exists; The fourth processing module is used to package the data of each target entry into a data packet and send the data packet to one or more stream multiprocessors.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the on-chip network congestion control method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the on-chip network congestion control method as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the on-chip network congestion control method as described in any one of claims 1 to 8.

Images