Method and apparatus for implementing multi-stream MDL parallel non-blocking aggregation

Abstract

The application provides a method and device for implementing multi-stream MDL parallel non-blocking aggregation. A DL to be aggregated into an MDL is dequeued from a to-be-aggregated queue, and it is determined whether data preparation of the DL is completed: if not, the DL is pre-aggregated and stored in a first waiting queue; if yes, the DL is pre-aggregated. When the DL is the last one, it is determined whether data preparation of the aggregated DL is completed: if yes, the MDL is allowed to be issued; if not, a pre-aggregation end position of the MDL is set. Based on a data preparation completion state of the DL entering the first waiting queue, a mutual exclusion logic is used to trigger the DL to be dequeued and sent into a post-aggregation process. For the DL dequeued from the first waiting queue, if pre-aggregation of the associated MDL is not ended, the DL is stored in a second waiting queue; if the pre-aggregation process is ended, the DL is post-aggregated, and when data preparation of the aggregated DL is completed, the MDL is allowed to be issued. The application realizes multi-stream MDL parallel non-blocking aggregation.

Description

Technical Field

[0001] This application belongs to the field of chip communication, and specifically relates to a method and apparatus for implementing multi-stream MDL parallel non-blocking aggregation. Background Technology

[0002] In I / O hardware acceleration chips, multiple small I / Os that are contiguous in physical space are typically aggregated into a larger I / O for transmission to improve I / O performance. Each small I / O corresponds to an on-chip cache, denoted as a DL (Data Lane). Multiple DLs pointing to contiguous physical space can be aggregated into a large cache, denoted as an MDL (Multi-Data Lane). Data preparation for each small I / O is serial and out-of-order. When the data for multiple DLs pointed to by an MDL is ready, the MDL can be transmitted to the next-level system; otherwise, it must wait for the DL data to be ready. Summary of the Invention

[0003] The purpose of this application is to provide a method and apparatus for implementing parallel non-blocking aggregation of multi-stream MDL, which aims to reduce the complexity of multi-stream MDL aggregation.

[0004] According to a first aspect of this application, a method for implementing parallel non-blocking aggregation of multi-stream MDL is provided, comprising:

[0005] The DLs to be aggregated into the MDL are dequeued from the aggregation queue. The data preparation of the DL is then checked: if the data preparation is incomplete, the DL is pre-aggregated and stored in the first waiting queue, then it is determined whether this DL is the last one; if the data preparation is complete, the DL is pre-aggregated, and then it is determined whether this DL is the last one. If this DL is the last one, it means that all DLs to be aggregated into the MDL have been dequeued from the aggregation queue. At this point, a counter is used to determine whether the data preparation of all aggregated DLs is complete: if all are complete, the MDL is allowed to be sent out; if not all are complete, the pre-aggregation end setting of the MDL is set. For DLs entering the first waiting queue, based on the data preparation completion status of the DL, mutual exclusion logic is used to trigger the DL dequeueing and subsequent aggregation process.

[0006] If a DL is dequeued from the first waiting queue and its associated MDL pre-aggregation process has not yet ended, the DL is stored in the second waiting queue to wait; if its associated MDL pre-aggregation process has ended, the DL is subjected to post-aggregation processing. When all aggregated DL data is ready, the MDL is allowed to be issued.

[0007] In an optional implementation, determining whether the data preparation of the DL is complete further includes:

[0008] If data preparation is not complete, increment the first counter; if data preparation is complete, increment both the first and second counters.

[0009] After the last DL undergoes pre-aggregation processing, if the count values ​​of the first counter and the second counter are equal, it is determined that the aggregated DL data preparation is complete.

[0010] After DL is dequeued from the first waiting queue, the process also includes:

[0011] If pre-aggregation ends, increment the second counter;

[0012] After performing the post-aggregation process on the DL, the following is further included:

[0013] If the count values ​​of the first counter and the second counter are equal, then it is determined that the aggregated DL data is ready to be fully prepared.

[0014] The data preparation completion status based on DL, utilizing mutual exclusion logic to perform boundary processing on the first waiting queue, further includes:

[0015] The prepared data DL is cached. When the DL arrives at the head of the cache, it is determined whether the DL is in the first waiting queue. If it is in the first waiting queue, the DL is read from the cache and dequeued from the first waiting queue; if it is not in the first waiting queue, the DL is read from the cache and discarded.

[0016] The step of retrieving the DL from the first waiting queue, storing the DL in the second waiting queue before the pre-aggregation is completed, and performing post-aggregation processing after the pre-aggregation is completed further includes:

[0017] If the pre-aggregation end bit corresponding to the DL is set, the second counter is incremented. If the count values ​​of the first counter and the second counter are equal, it is determined that the aggregated DL data is fully prepared and the MDL is allowed to be sent. If they are not equal, it continues to wait for the next judgment.

[0018] If the pre-aggregation end bit corresponding to the DL is not set, then the DL is stored in the second waiting queue.

[0019] The step of storing the DL in the second waiting queue for waiting further includes:

[0020] For a DL in the second waiting queue, if its corresponding pre-aggregation end bit is set, it is dequeued from the second waiting queue and the second counter is incremented.

[0021] If the count values ​​of the first counter and the second counter are equal, it is determined that the aggregated DL data is fully prepared and the MDL can be distributed.

[0022] According to a second aspect of this application, an implementation apparatus for multi-stream MDL parallel non-blocking aggregation is provided, comprising:

[0023] The pre-aggregation unit is used to dequeue the DL to be aggregated into the MDL from the aggregation queue and determine whether the data preparation of the DL is complete. If the data preparation is not complete, the DL is pre-aggregated and stored in the first waiting queue. If the data preparation is complete, the DL is pre-aggregated.

[0024] Determine if the DL is the last DL. If it is, check the statistics counter to see if all aggregated DL data preparation is complete. If all are complete, allow the MDL to be sent out. If not, set the pre-aggregation end position of the MDL.

[0025] For a DL entering the first waiting queue, based on the DL's data preparation completion status, the aggregation process after the DL is dequeued and sent in is triggered using mutual exclusion logic.

[0026] The post-aggregation unit is used to store the DL in the second waiting queue for waiting if the MDL pre-aggregation process associated with the DL has not been completed, and to perform post-aggregation processing on the DL if the MDL pre-aggregation process associated with the DL has been completed. When all the aggregated DL data is ready, the MDL is allowed to be sent out.

[0027] Compared with related technologies, the technical solution of this application has at least the following advantages:

[0028] Building upon the implementation of parallel non-blocking aggregation of multi-stream MDL, only two queues are used, reducing the complexity of the multi-queue linked list control logic, minimizing logical resources, and facilitating timing convergence. Furthermore, since the number of queues is fixed and does not increase with the number of MDLs, the scalability of MDLs is enhanced.

[0029] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures and processes shown in the description and the accompanying drawings. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram of the serial design architecture for implementing MDL aggregation using related technologies.

[0032] Figure 2 This is a schematic diagram of the parallel design architecture for implementing MDL aggregation using related technologies.

[0033] Figure 3 This is a flowchart of a method for implementing multi-stream MDL parallel non-blocking aggregation according to an exemplary embodiment of this application.

[0034] Figure 4 This is a flowchart of multi-stream MDL parallel non-blocking aggregation according to an exemplary embodiment of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] Serial design architecture in related technologies, such as Figure 1 As shown, the DL data to be aggregated in the current MDL is not all ready, but the data for subsequent DL aggregations is ready. Therefore, the current MDL cannot be sent, thus blocking the sending of subsequent DL aggregations with ready data.

[0037] Parallel design architectures in related technologies, such as Figure 2 As shown in the diagram. To address the blocking issue, queues corresponding to the number of MDLs are prepared at the MDL granularity. Assuming the number of MDLs is n, n queues are needed to store the DL pointed to by each MDL. These n queues convert the aggregation of DLs from serial to parallel. When the DL data stored in a queue is ready, the MDL represented by that queue can be distributed. Although this method can solve the blocking problem, the number of queues n increases with the number of MDLs, rapidly increasing the complexity of the queue control logic, increasing hardware logic resources and timing convergence risks.

[0038] Meanwhile, in the two design architectures mentioned above, since the DL data preparation process and the DL aggregation process are two parallel processes, the time slice for judging the completion of DL data preparation will inevitably encounter the interaction boundary problem between the two processes. The judgment of the completion of DL data preparation aggregated in the same MDL is either serial or requires aggregation marking at the DL granularity, consuming more logical resources to achieve parallel processing. For example, according to Figure 2 The mechanism is such that the number of aggregation tags is equal to the number of MDLs, and increases with the number of MDLs.

[0039] Based on the above analysis, this application provides a method and apparatus for implementing parallel non-blocking aggregation of multi-stream MDLs. It uses fewer queues and combines counter statistics to achieve parallel non-blocking aggregation of multi-stream MDLs. Taking a single MDL as an example, when multiple DLs are aggregated to a certain MDL, a first counter (denoted as dlcnt) is used to count the number of DLs that can be aggregated by this MDL, and a second counter (denoted as dldonecnt) counts the number of DLs among the aggregated DLs whose data is ready. If a DL's data is not ready, the DL and its corresponding MDL information are stored in a waiting queue until its data is ready. Then, the DL is removed from the queue, and dldonecnt is updated. When dlcnt equals dldonecnt, it indicates that the data of all DLs contained in this MDL is ready and can be sent to the next level. Furthermore, for boundary handling, a buffer is used for isolation between two parallel processes. The process state of the boundary point DL is determined at the buffer's entry and exit points to resolve the interaction boundary problem.

[0040] See Figure 3 The flowchart illustrates, for example, the implementation method of MDL parallel non-blocking aggregation provided in this application includes:

[0041] Step 301: Dequeue the DL to be aggregated into MDL from the aggregation queue and determine whether the data preparation of the DL is complete; if the data preparation is not complete, pre-aggregate the DL and store it in the first waiting queue; if the data preparation is complete, pre-aggregate the DL.

[0042] Determine if the DL is the last DL. If it is, check the statistics counter to see if all aggregated DL data preparation is complete. If all are complete, allow the MDL to be sent out. If not, set the pre-aggregation end position of the MDL.

[0043] For a DL entering the first waiting queue, based on the DL's data preparation completion status, the aggregation process after the DL is dequeued and sent in is triggered using mutual exclusion logic.

[0044] like Figure 4As shown in the flowchart, for example, the DL to be aggregated queue and waiting queues 0 and 1 are three queues formed by doubly linked lists, sharing physical storage space, with a total depth equal to the number of DLs. The depth of each queue is dynamically adjustable. The DL to be aggregated queue is used to implement the aggregation algorithm. DLs that can be aggregated into MDL are dequeued sequentially, and last is 1, indicating that the DL is the last DL to be aggregated into MDL. Waiting queues 0 and 1 are used to wait for DL ​​data preparation to be completed. Waiting queue 0 is called the first waiting queue, storing DLs whose data preparation is not yet complete in the pre-aggregation process and their corresponding MDL information. Waiting queue 1 is called the second waiting queue, storing DLs whose data preparation is complete, but whose corresponding MDL pre-aggregation process is not yet complete, and their MDL information. dlcnt[] is the first counter, with MDL as the granularity, representing the number of DLs aggregated by each MDL. dldonecnt[] is the second counter, with MDL as the granularity, representing the number of DLs whose data preparation is complete in the DLs aggregated by each MDL. predone[] is a set of bitmap flags, with MDL as the granularity, indicating whether the pre-aggregation process of each MDL is complete.

[0045] For example, in the pre-aggregation process, after the DL is dequeued from the queue to be aggregated, it is determined whether its data is ready: if ready, dlcnt[mdl] is incremented and dldonecnt[mdl] is incremented; if not ready, only dlcnt[mdl] is incremented, dldonecnt[mdl] remains unchanged, and the DL and the corresponding MDL information are stored in the first waiting queue, i.e., waiting queue 0.

[0046] Check the `last` flag. When `last` is 1, it means that all DLs aggregated to the MDL have been dequeued from the aggregation queue. Check if `dlcnt[mdl]` equals `dldonecnt[mdl]`. If they are equal, it means that all DL data aggregated to the MDL is ready, and the MDL can be sent down to the next level. If they are not equal, it means that some DL data is not ready, and `predone[mdl]` is set to 1, indicating that the MDL can start the subsequent aggregation process.

[0047] In a further embodiment, if the DL data is not ready, it continues to wait until the data is ready, and then is dequeued from the waiting queue 0 by mutual exclusion logic to perform boundary processing through mutual exclusion logic.

[0048] For example, when determining whether DL data is ready, if the data is ready but the determination logic hasn't sampled it, a DL that shouldn't be in waiting queue 0 might enter waiting queue 0. However, if the DL is in the process of entering waiting queue 0 when the data is ready, triggering the DL to be dequeued from waiting queue 0 at this time might fail to find the DL. Both of these factors will cause aggregation failure.

[0049] Therefore, during the boundary processing phase, the prepared deep learning (DL) data is cached. When it reaches the head of the cache, it is determined whether it is in waiting queue 0. If it is in waiting queue 0, it is read from the cache and dequeued from waiting queue 0; if it is not in waiting queue 0, it is read from the cache and discarded.

[0050] Step 302: For a DL dequeued from the first waiting queue, if the MDL pre-aggregation process associated with the DL has not been completed, the DL is stored in the second waiting queue for waiting; if the MDL pre-aggregation process associated with the DL has been completed, the DL is subjected to post-aggregation processing. When all the aggregated DL data is ready, the MDL is allowed to be sent out.

[0051] In the post-aggregation phase, after the DL that has completed data preparation is dequeued from the waiting queue 0, predone[mdl] is determined.

[0052] If predone[mdl] is 1, then dldonecnt[mdl] is incremented, and then it is checked whether it is equal to dlcnt[mdl]. If they are equal, the MDL is started and sent; if they are not equal, the process continues to wait for the next check.

[0053] If predone[mdl] is 0, then the DL and the corresponding MDL information are stored in the second waiting queue, i.e., waiting queue 1.

[0054] For a DL in waiting queue 1, if its corresponding predone[mdl] is 1, it is dequeued, and dldonecnt[mdl] is incremented, and it is checked whether it is equal to dlcnt[mdl]. If they are equal, the MDL is initiated and issued; otherwise, it continues to wait for the next check. If its corresponding predone[mdl] is 0, it continues to wait in the queue.

[0055] As can be seen, the implementation method of multi-stream MDL parallel non-blocking aggregation provided in this application has at least the following advantages compared with related technologies: Based on the implementation of multi-stream MDL parallel non-blocking aggregation, only two queues are used, reducing the complexity of the multi-queue linked list control logic, reducing logic resources, and facilitating timing convergence. At the same time, since the number of queues is fixed and does not increase with the number of MDLs, the scalability of MDLs is enhanced.

[0056] Accordingly, exemplarily, this application provides an implementation apparatus for multi-stream MDL parallel non-blocking aggregation in a second aspect, comprising:

[0057] The pre-aggregation unit is used to dequeue the DL to be aggregated into the MDL from the aggregation queue and determine whether the data preparation of the DL is complete. If the data preparation is not complete, the DL is pre-aggregated and stored in the first waiting queue. If the data preparation is complete, the DL is pre-aggregated.

[0058] Determine if the DL is the last DL. If it is, check the statistics counter to see if all aggregated DL data preparation is complete. If all are complete, allow the MDL to be sent out. If not, set the pre-aggregation end position of the MDL.

[0059] For a DL entering the first waiting queue, based on the DL's data preparation completion status, the aggregation process after the DL is dequeued and sent in is triggered using mutual exclusion logic.

[0060] The post-aggregation unit is used to store the DL in the second waiting queue for waiting if the MDL pre-aggregation process associated with the DL has not been completed, and to perform post-aggregation processing on the DL if the MDL pre-aggregation process associated with the DL has been completed. When all the aggregated DL data is ready, the MDL is allowed to be sent out.

[0061] The above-described device can be implemented using the multi-stream MDL parallel non-blocking aggregation implementation method provided in the first aspect embodiment. For specific implementation details, please refer to the description in the first aspect embodiment, which will not be repeated here.

[0062] It is understood that the circuit structures, names, and parameters described in the above embodiments are merely examples. Those skilled in the art can also make readily conceived combinations and adjustments to the structural features of the above embodiments according to their needs, and the concept of this application should not be limited to the specific details of the above examples.

[0063] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for implementing parallel non-blocking aggregation of multi-stream MDL, characterized in that, include: The DL to be aggregated into the MDL is dequeued from the aggregation queue, and it is determined whether the data preparation of the DL is complete. If the data preparation is not complete, the DL is pre-aggregated and then stored in the first waiting queue. If the data preparation is complete, the DL is pre-aggregated. Determine if the DL is the last DL. If it is, check the statistics counter to see if all aggregated DL data preparation is complete. If all are complete, allow the MDL to be sent out. If not, set the pre-aggregation end position of the MDL. For a DL entering the first waiting queue, based on the DL's data preparation completion status, the aggregation process after the DL is dequeued and sent in is triggered using mutual exclusion logic. For a DL dequeued from the first waiting queue, if the MDL pre-aggregation process associated with the DL has not been completed, the DL is stored in the second waiting queue for waiting; if the MDL pre-aggregation process associated with the DL has been completed, the DL is subjected to post-aggregation processing, and when all the aggregated DL data is ready, the MDL is allowed to be sent out.

2. The method for implementing multi-stream MDL parallel non-blocking aggregation according to claim 1, characterized in that, The step of determining whether the data preparation of the DL is complete further includes: If data preparation is not complete, increment the first counter; if data preparation is complete, increment both the first and second counters. After the last DL undergoes pre-aggregation processing, if the count values ​​of the first counter and the second counter are equal, it is determined that the aggregated DL data preparation is complete. After DL is dequeued from the first waiting queue, the process also includes: If pre-aggregation ends, increment the second counter; After performing the post-aggregation process on the DL, the following is further included: If the count values ​​of the first counter and the second counter are equal, then it is determined that the aggregated DL data is ready to be fully prepared.

3. The method for implementing multi-stream MDL parallel non-blocking aggregation according to claim 1, characterized in that, The data preparation completion status based on DL, utilizing mutual exclusion logic to perform boundary processing on the first waiting queue, further includes: The prepared data DL is cached. When the DL arrives at the head of the cache, it is determined whether the DL is in the first waiting queue. If it is in the first waiting queue, the DL is read from the cache and dequeued from the first waiting queue; if it is not in the first waiting queue, the DL is read from the cache and discarded.

4. The method for implementing multi-stream MDL parallel non-blocking aggregation according to claim 2, characterized in that, The step of retrieving the DL from the first waiting queue, storing the DL in the second waiting queue before the pre-aggregation is completed, and performing post-aggregation processing after the pre-aggregation is completed further includes: If the pre-aggregation end bit corresponding to the DL is set, the second counter is incremented. If the count values ​​of the first counter and the second counter are equal, it is determined that the aggregated DL data is fully prepared and the MDL is allowed to be sent. If they are not equal, it continues to wait for the next judgment. If the pre-aggregation end bit corresponding to the DL is not set, then the DL is stored in the second waiting queue.

5. The method for implementing multi-stream MDL parallel non-blocking aggregation according to claim 2, characterized in that, The step of storing the DL in the second waiting queue for waiting further includes: For a DL in the second waiting queue, if its corresponding pre-aggregation end bit is set, it is dequeued from the second waiting queue and the second counter is incremented. If the count values ​​of the first counter and the second counter are equal, it is determined that the aggregated DL data is fully prepared and the MDL can be distributed.

6. An apparatus for implementing multi-stream MDL parallel non-blocking aggregation, characterized in that, include: The pre-aggregation unit is used to dequeue the DL to be aggregated into the MDL from the aggregation queue and determine whether the data preparation of the DL is complete. If the data preparation is not complete, the DL is pre-aggregated and stored in the first waiting queue. If the data preparation is complete, the DL is pre-aggregated. Determine if the DL is the last DL. If it is, check the statistics counter to see if all aggregated DL data preparation is complete. If all are complete, allow the MDL to be sent out. If not, set the pre-aggregation end position of the MDL. For a DL entering the first waiting queue, based on the DL's data preparation completion status, the aggregation process after the DL is dequeued and sent in is triggered using mutual exclusion logic. The post-aggregation unit is used to store the DL in the second waiting queue for waiting if the MDL pre-aggregation process associated with the DL has not been completed, and to perform post-aggregation processing on the DL if the MDL pre-aggregation process associated with the DL has been completed. When all the aggregated DL data is ready, the MDL is allowed to be sent out.

7. The apparatus for implementing multi-stream MDL parallel non-blocking aggregation according to claim 6, characterized in that, The prepolymerization unit is further used for: If data preparation is not complete, increment the first counter; if data preparation is complete, increment both the first and second counters. After the last DL undergoes pre-aggregation processing, if the count values ​​of the first counter and the second counter are equal, it is determined that the aggregated DL data preparation is complete. The post-aggregation unit is further used for: After DL is dequeued from the first waiting queue, if pre-aggregation is complete, the second counter is incremented. After the DL is post-aggregated, if the count values ​​of the first counter and the second counter are equal, it is determined that the aggregated DL data preparation is complete.

8. The apparatus for implementing multi-stream MDL parallel non-blocking aggregation according to claim 6, characterized in that, The prepolymerization unit is further used for: The prepared data DL is cached. When the DL arrives at the head of the cache, it is determined whether the DL is in the first waiting queue. If it is in the first waiting queue, the DL is read from the cache and dequeued from the first waiting queue; if it is not in the first waiting queue, the DL is read from the cache and discarded.

9. The apparatus for implementing multi-stream MDL parallel non-blocking aggregation according to claim 7, characterized in that, The post-aggregation unit is further used for: If the pre-aggregation end bit corresponding to the DL is set, the second counter is incremented. If the count values ​​of the first counter and the second counter are equal, it is determined that the aggregated DL data is fully prepared and the MDL is allowed to be sent. If they are not equal, it continues to wait for the next judgment. If the pre-aggregation end bit corresponding to the DL is not set, then the DL is stored in the second waiting queue.

10. The apparatus for implementing multi-stream MDL parallel non-blocking aggregation according to claim 7, characterized in that, The post-aggregation unit is further used for: For a DL in the second waiting queue, if its corresponding pre-aggregation end bit is set, it is dequeued from the second waiting queue and the second counter is incremented. If the count values ​​of the first counter and the second counter are equal, it is determined that the aggregated DL data is fully prepared and the MDL can be distributed.

Images