A multi-host task synchronizer and a multi-host task synchronization method

Abstract

The present application discloses a multi-host task synchronizer and a multi-host task synchronization method, which relate to the field of data processing. In the multi-host task synchronizer, the MPQ, the transport unit and the destination queue storage unit are connected to a communication bus, and the MPQ is in communication connection with the transport unit. The MPQ obtains at least one group of first description information based on the write register content of its own cache space, calls different transport units for different groups of first description information, and asynchronously processes the corresponding group of transport tasks based on the called transport units and the first description information of the corresponding group. When asynchronously processing the source data generated by each host in the corresponding group of transport tasks, the predicted value of the write pointer of the corresponding destination queue on the communication bus is updated according to the size of the source data. The present application uses the same transport unit and a unique bus identity to transport the same group of transport tasks, thereby ensuring atomicity. The transport process does not require the participation of the host, avoids a single host monopolizing the bus, and reduces the host resource occupancy rate.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a multi-host task synchronizer and a multi-host task synchronization method. Background Art

[0002] The existing system-on-chip designs have Figure 1 In the multi-host parallel structure shown, multiple hosts (for example, Figure 1 The first host 150, the second host 151, the third host 152 and the fourth host 153 are shown, and multiple destination queues that can be shared by multiple hosts (for example, Figure 1 The first destination queue 130, the second destination queue 131 and the third destination queue 132 are shown in the figure. The source data generated by the host can be written into the destination queue. When a host updates the target queue, multiple data transmissions may be required to write a complete source data into the destination queue, and the destination queue may need to accept write requests from other hosts at the same time. As a result, the transmission process of multiple data may be interrupted by the write requests of other hosts, resulting in the destruction of the order and integrity of the multiple data, that is, the problem of access atomicity occurs.

[0003] In traditional solutions, mutual exclusion lock structures or complex software algorithms are often used to ensure atomicity. For example, a mutual exclusion lock structure is used to ensure atomicity when multiple central processing units (CPUs) correctly access shared memory. However, the use of mutual exclusion lock structures or complex software algorithms has the problem of high system resource usage. In addition, when performing multi-host synchronization tasks based on bus lock signals, only a single host can have exclusive access to the bus, while other hosts are blocked, resulting in reduced system performance, which needs to be solved urgently. Summary of the invention

[0004] In view of the above problems, the present application provides a multi-host task synchronizer and a multi-host task synchronization method to ensure atomicity while reducing system resource occupancy in the process of writing source data generated by multiple hosts into a destination queue, and solve the problem of a single host exclusively accessing the bus. The specific solution is as follows:

[0005] In a first aspect, the present application provides a multi-host task synchronizer, comprising: a multi-producer queue component, a transport unit, a destination queue storage unit, and a communication bus, wherein the multi-producer queue component, the transport unit, and the destination queue storage unit are connected to the communication bus, the multi-producer queue component is also in communication connection with the transport unit, and the destination queue storage unit stores at least one destination queue;

[0006] The multi-producer queue component is used to obtain at least one set of first description information based on the write register content written into its own cache space, wherein each set of the first description information is used to describe a set of transport tasks, and the set of transport tasks is used to write source data generated by at least one host into the same destination queue, and different sets of the first description information correspond to different destination queues;

[0007] The multi-producer queue component is also used to call different transport units for different groups of the first description information, so as to asynchronously process the corresponding group transport tasks on the communication bus based on the called transport units and the first description information of the corresponding group, and when asynchronously processing the source data generated by each host in the corresponding group transport tasks, update the predicted value of the write pointer of the corresponding destination queue on the communication bus according to the size of the source data generated by the host, wherein the transport unit uses a unique bus identifier when asynchronously processing the corresponding group transport tasks.

[0008] Optionally, for each group of first description information in the at least one group of first description information, the group of first description information is used as target first description information, the transport unit called for the target first description information is used as the target transport unit, and a group of transport tasks described by the target first description information is used as the target transport tasks;

[0009] Then, when the multi-producer queue component asynchronously processes the target transport task on the communication bus based on the target transport unit and the target first description information, it is specifically used to:

[0010] Parsing the first target description information to obtain a storage address of the second target description information, wherein the second target description information is information describing the target destination queue corresponding to the first target description information;

[0011] Acquire the target second description information from the storage address of the target second description information, and parse the target second description information to obtain the address of the write pointer of the target destination queue on the communication bus as the target address;

[0012] Using source data generated by any host that does not carry out source data transportation in the target transportation task as target source data, obtaining a pointer value of a write pointer of the target destination queue on the communication bus from the target address, and judging whether the remaining storage space of the target destination queue is sufficient to store the target source data according to the pointer value;

[0013] If so, a transfer request is generated, and based on the transfer request, the target transfer unit is called to transfer the target source data. After generating the transfer request, the process returns to the step of using the source data generated by any host that has not transferred the source data in the target transfer task as the target source data.

[0014] Optionally, when the multi-producer queue component calls the target transport unit based on the transport request to transport the target source data, it is further configured to:

[0015] When the target transport unit is called based on the transport request, it is determined whether arbitration for using the target transport unit is won. If so, the target source data is transported based on the target transport unit, wherein the transport request carries priority information for the arbitration.

[0016] Optionally, when the multi-producer queue component asynchronously processes the target transport task on the communication bus based on the target transport unit and the target first description information, it is further configured to:

[0017] If the remaining storage space of the target destination queue is insufficient to store the target source data, wait for a preset time length, and when the preset time length is reached, return to execute the step of obtaining the pointer value of the write pointer of the target destination queue on the communication bus from the target address, until the remaining storage space of the target destination queue is sufficient to store the target source data or the preset number of cycles is reached.

[0018] Optionally, when the multi-producer queue component obtains at least one set of first description information based on the write register content written into its own cache space, it is specifically used to:

[0019] Determine whether the own cache space is not empty;

[0020] If so, the at least one set of first description information is obtained based on the write register content written into the own cache space after the own state machine is started.

[0021] Optionally, when the multi-producer queue component obtains the at least one set of first description information based on the write register content written into the own cache space, the multi-producer queue component is specifically configured to:

[0022] Parsing the write register content written into the own cache space to obtain the storage address of the at least one set of first description information;

[0023] The at least one set of first description information is obtained from a storage address of the at least one set of first description information.

[0024] Optionally, the transfer unit is a linear transfer unit in a direct register access DMA.

[0025] A second aspect of the present application provides a multi-host task synchronization method, which is applied to host software and includes:

[0026] Obtain at least one group of transfer tasks, wherein each group of transfer tasks is used to write source data generated by at least one host into the same destination queue, and different groups of transfer tasks correspond to different destination queues;

[0027] Identify multi-producer queue components;

[0028] Generate write register content;

[0029] The write register content is stored in the cache space of the multi-producer queue component, so that the multi-producer queue component obtains the first description information corresponding to at least one group of transfer tasks according to the write register content written into its own cache space, and calls different transfer units for the first description information corresponding to different groups of transfer tasks, so as to asynchronously process the corresponding group of transfer tasks on the communication bus based on the called transfer units and the corresponding first description information, and when asynchronously processing the source data generated by each host in the corresponding group of transfer tasks, updates the predicted value of the write pointer of the corresponding target queue on the communication bus according to the size of the source data generated by the host, wherein the transfer unit uses a unique bus identity when asynchronously processing the corresponding group of transfer tasks, and the first description information corresponding to a group of transfer tasks is used to describe the group of transfer tasks.

[0030] Optionally, before obtaining at least one group of transport tasks, the method further includes:

[0031] First description information corresponding to a plurality of common handling tasks and second description information corresponding to a plurality of common destination queues are pre-configured, wherein the second description information corresponding to each common destination queue is information describing the common destination queue.

[0032] Optionally, the multi-producer queue component is determined according to a preconfigured scheduling strategy and / or a working state of the multi-producer queue component, wherein the scheduling strategy is used to indicate how to schedule the multi-producer queue component.

[0033] By means of the above technical solution, the multi-host task synchronizer provided by the present application includes a multi-producer queue component, a transport unit, a destination queue storage unit and a communication bus. The multi-producer queue component, the transport unit and the destination queue storage unit are connected to the communication bus. The multi-producer queue component is also connected to the transport unit for communication. The destination queue storage unit stores at least one destination queue. Among them, the multi-producer queue component is used to obtain at least one group of first description information based on the content of the write register written into its own cache space, and to call different transport units for different groups of first description information, so as to asynchronously process the corresponding group of transport tasks on the communication bus based on the called transport unit and the first description information of the corresponding group, and when asynchronously processing the source data generated by each host in the corresponding group of transport tasks, the predicted value of the write pointer of the corresponding destination queue on the communication bus is updated according to the size of the source data generated by the host.

[0034] Since different groups of first description information call different transport units, and each transport unit uses a unique bus identifier when asynchronously processing the corresponding group of transport tasks, the source data generated by each host in a group of transport tasks described by each group of first description information can be transmitted based on the same route. Therefore, the source data generated by each host in a group of transport tasks described by each group of first description information can be written into the same destination queue in sequence, and the source data generated by one host will not be disrupted by the source data generated by other hosts when being written into the destination queue, thereby ensuring atomicity.

[0035] After the host of the present application generates a transfer task, the transfer task can be received by the multi-producer queue component in the form of the corresponding first description information. The entire subsequent transfer process does not require the participation of the host, which reduces the occupancy rate of the host resources during the transfer process and avoids a single host monopolizing the data bus.

[0036] In addition, the transport unit in the present application can process the corresponding group transport tasks asynchronously, so that after the source data generated by a host in the corresponding group transport task starts to be transported, it does not need to wait for the transport to be completed before starting to transport the source data generated by the next host, thereby improving the transport efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0038] Figure 1 A schematic diagram of a multi-host parallel structure provided by the prior art;

[0039] Figure 2A schematic diagram of the structure of a multi-host task synchronizer provided in this application;

[0040] Figure 3 A schematic diagram of the corresponding relationship among source data, transport tasks, first description information, destination queues and transport units in a multi-host task synchronization process;

[0041] Figure 4 A flowchart of a multi-host task synchronization method provided by this application;

[0042] Figure 5 A schematic diagram of the interaction between a host software, a multi-host task synchronizer, and multiple hosts. DETAILED DESCRIPTION

[0043] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0044] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0045] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0046] This application provides a multi-host task synchronizer, see Figure 2 , which is a structural diagram of a multi-host task synchronizer.

[0047] like Figure 2 As shown, the multi-host task synchronizer includes: a multi-producer queue component (Multi-Producer-Queue, MPQ) 11, a transport unit (Linear Copy) 12, a destination queue storage unit 13 and a communication bus 14. The multi-producer queue component 11, the transport unit 12 and the destination queue storage unit 13 are connected to the communication bus 14, and the multi-producer queue component 11 is also communicatively connected to the transport unit 12.

[0048] The multi-host task synchronizer may include at least one multi-producer queue component 11 and at least one transport unit 12; the destination queue storage unit 13 stores at least one destination queue, such as Figure 2 Three destination queues are shown, namely a first destination queue 130 , a second destination queue 131 and a third destination queue 132 .

[0049] The transport unit 12 has the ability to process multiple tasks in parallel at the same time. Optionally, the transport unit 12 can be a linear transport unit in a direct register access DMA.

[0050] In the present application, the multi-producer queue component 11, the transport unit 12 and the destination queue storage unit 13 can cooperate with each other to realize the process of multi-host task synchronization on the communication bus 14. Here, "synchronization" refers to keeping the source data in the host and destination queues synchronized.

[0051] In order to ensure atomicity when implementing multi-host task synchronization, the multi-producer queue component 11 can obtain at least one set of first description information based on the write register content written into its own cache space (represented by MDP in this application), wherein each set of first description information is used to describe a group of transport tasks, and a group of transport tasks is used to write source data generated by at least one host into the same destination queue, and the destination queues corresponding to different groups of first description information are different; different transport units are called for different groups of first description information to asynchronously process the corresponding group of transport tasks on the communication bus based on the called transport units and the first description information of the corresponding group, and when asynchronously processing the source data generated by each host in the corresponding group of transport tasks, the predicted value of the write pointer of the corresponding destination queue on the communication bus is updated according to the size of the source data generated by the host, wherein the transport unit uses a unique bus identity when asynchronously processing the corresponding group of transport tasks.

[0052] The above process is explained in detail below.

[0053] In actual application scenarios, there may be multiple hosts, all of which can generate source data. This embodiment can generate corresponding transfer tasks based on the source data generated by each host, so as to write the source data into the destination queue by processing the transfer tasks.

[0054] Since the destination queues to which source data generated by multiple hosts need to be written may be different, the transfer tasks generated by multiple hosts may be grouped based on whether the destination queues are the same to obtain at least one group of transfer tasks. For example, the destination queues to which source data generated by hosts 1 to 10 (for example, host 1 is the first host 150 above, host 2 is the second host 151 above, and so on) need to be written are destination queue 1, destination queue 1, destination queue 1, destination queue 2, destination queue 2, destination queue 3, destination queue 4, destination queue 4, destination queue 4 and destination queue 4 (for example, destination queue 1 is the first host 150 above). The destination queue 130, the destination queue 2 is the second destination queue 131 mentioned above, and so on), then the transfer tasks 1~10 generated by the hosts 1~10 are grouped, and 4 groups of transfer tasks can be obtained, namely: the first group of transfer tasks for writing to the destination queue 1 [transfer task 1, transfer task 2, transfer task 3]; the second group of transfer tasks for writing to the destination queue 2 [transfer task 4, transfer task 5]; the third group of transfer tasks for writing to the destination queue 3 [transfer task 6]; the fourth group of transfer tasks for writing to the destination queue 4 [transfer task 7, transfer task 8, transfer task 9, transfer task 10].

[0055] That is, in the present application, a group of transport tasks is used to write source data generated by at least one host into the same destination queue.

[0056] Furthermore, the present application can describe each group of transport tasks separately to obtain a set of first description information for describing each group of transport tasks. Here, each set of first description information can be used to describe the relevant information of the corresponding group of transport tasks. Since the first description information and the transport tasks correspond one-to-one in the form of groups, and different groups of transport tasks correspond to different destination queues, the destination queues corresponding to the first description information of different groups are also different.

[0057] In a specific embodiment, the first description information may be a first-level descriptor, and the number of first-level descriptors contained in a group of first-level descriptors is the same as the number of transport tasks in a group of transport tasks described by the group of first-level descriptors. Optionally, each first-level descriptor in a group of first-level descriptors may correspond one-to-one to each transport task in a corresponding group of transport tasks. For example, a group of first-level descriptors includes first-level descriptors 1 to 4, and a group of transport tasks described by the group of first-level descriptors includes transport tasks 1 to 4. Then, first-level descriptor 1 is used to describe transport task 1, first-level descriptor 2 is used to describe transport task 2, first-level descriptor 3 is used to describe transport task 3, and first-level descriptor 4 is used to describe transport task 4.

[0058] For example, optionally, each first-level descriptor may include the starting address and source data size of the source data required for the transfer task. Of course, each first-level descriptor may also include more or less information, which is not limited here.

[0059] In the present application, the relevant information of at least one set of the above-mentioned first description information can be written into a register, and the content of the write register can be immediately stored in the cache space of the multi-producer queue component 11, and then the multi-producer queue component 11 can obtain at least one set of first description information based on the content of the write register written into its own cache space.

[0060] In order to ensure atomicity during the synchronization process, the multi-producer queue component 11 can call different transport units for different groups of first description information, and use a unique bus identifier (bus_id) when the transport unit asynchronously processes the corresponding group of transport tasks (that is, different bus identifiers are used for different groups of transport tasks). In this way, each group of transport tasks can use a unique route for data transmission during the data transport process without being interrupted by the data transport process of other groups of transport tasks. At the same time, it ensures that the source data in each transport task contained in a group of transport tasks can be written to the destination queue sequentially, ensuring atomicity.

[0061] See also Figure 3 As shown, a schematic diagram of the correspondence between source data, transport tasks, first description information, destination queues and transport units in a multi-host task synchronization process is shown.

[0062] like Figure 3Two groups of transport tasks are shown, namely the first group of transport tasks and the second group of transport tasks, wherein the first group of transport tasks includes transport tasks 1-3, and the second group of transport tasks includes transport tasks 4-5, transport task 1 is used to write source data 1 generated by host 1 into destination queue 1, transport task 2 is used to write source data 2 generated by host 2 into destination queue 1, transport task 3 is used to write source data 3 generated by host 3 into destination queue 1, transport task 4 is used to write source data 4 generated by host 4 into destination queue 2, and transport task 5 is used to write source data 5 generated by host 5 into destination queue 2. Each group of transport tasks can be described by a group of first description information, that is, the first group of transport tasks corresponds to the first group of first description information, and the second group of transport tasks corresponds to the second group of first description information. In order to better describe each handling task, a first description information can be used to describe each handling task. Therefore, when the first group of handling tasks includes handling tasks 1 to 3, the first group of description information needs to include 3 first description information, specifically, the first description information 1 is used to describe handling task 1, the first description information 2 is used to describe handling task 2, and the first description information 3 is used to describe handling task 3; similarly, when the first group of handling tasks includes handling tasks 4 to 5, the second group of first description information includes 2 first description information, specifically, the first description information 4 is used to describe handling task 4, and the first description information 5 is used to describe handling task 5.

[0063] This application calls different transport units for different groups of first description information, so for Figure 3 The two groups of first description information shown require the use of two transport units. For example, the first group of first description information corresponds to transport unit 1, and the second group of first description information corresponds to transport unit 2.

[0064] It should be noted that Figure 3 This is only an example and is not intended to limit the present application.

[0065] Taking any one of at least one set of first description information as an example, the process of the multi-producer queue component 11 asynchronously processing the corresponding group transport task on the communication bus based on the called transport unit and the first description information of the corresponding group, and updating the predicted value of the write pointer of the corresponding destination queue on the communication bus according to the size of the source data generated by the host when asynchronously processing the source data generated by each host in the corresponding group transport task is introduced. It should be understood that the asynchronous processing process of the corresponding group transport task of other groups of first description information in at least one set of first description information is consistent and can be executed with reference to each other.

[0066] For the convenience of description, any one of the above-mentioned groups of first description information is defined as target first description information, the transport unit called for the target first description information is defined as a target transport unit, a group of transport tasks described by the target first description information is defined as a target transport task, and the destination queue corresponding to the target first description information is defined as a target destination queue, that is, the target destination queue is the destination queue to which the target transport task is to be written.

[0067] Optionally, the process of the multi-producer queue component 11 asynchronously processing the target transport task on the communication bus based on the target transport unit and the target first description information includes: parsing the target first description information to obtain the storage address of the target second description information, wherein the target second description information is information describing the target destination queue corresponding to the target first description information; obtaining the target second description information from the storage address of the target second description information, and parsing the target second description information to obtain the address of the write pointer of the target destination queue on the communication bus as the target address; taking the source data generated by any host that does not perform source data transport in the target transport task as the target source data, obtaining the pointer value of the write pointer of the target destination queue on the communication bus from the target address, and judging whether the remaining storage space of the target destination queue is sufficient to store the target source data according to the pointer value, wherein the target source data refers to the source data generated by any host that does not perform source data transport in the target transport task; if so, generating a transport request, calling the target transport unit based on the transport request to perform transport processing on the target source data, and after generating the transport request, returning to execute the step of taking the source data generated by any host that does not perform source data transport in the target transport task as the target source data.

[0068] Specifically, in the embodiment of the present application, description information for describing the destination queue is also provided. For the purpose of distinction, the description information describing the destination queue is defined as the second description information, that is, the destination queue and the second description information correspond one to one.

[0069] In the embodiment of the present application, the second description information describing the target destination queue is defined as target second description information.

[0070] Optionally, the storage address of the target second description information can be written into the target first description information. Then, when the multi-producer queue component 11 obtains the target first description information, it can parse the target first description information to obtain the storage address of the target second description information from the parsing result, thereby obtaining the target second description information from the storage address of the target second description information.

[0071] Optionally, the target second description information includes the address of the write pointer of the target destination queue on the communication bus. For the convenience of subsequent introduction, the address of the write pointer of the target destination queue on the communication bus is defined as the target address. Then, after the multi-producer queue component 11 obtains the target second description information, it can parse the target second description information to obtain the target address from the parsing result.

[0072] It is understandable that if the source data generated by at least one host in the target handling task is to be written into the target destination queue, it is necessary to determine whether the remaining storage space of the target destination queue is sufficient before writing the data. In an embodiment of the present application, the determination can be made based on the pointer value of the write pointer of the target destination queue on the communication bus read from the target address.

[0073] Since the source data generated by at least one host in the target handling task needs to be written into the target destination queue in sequence, it is necessary to determine whether the remaining storage space of the target destination queue is sufficient for the source data generated by each host in the target handling task. Based on this, the present application first defines the source data generated by any host that does not carry source data in the target handling task as the target source data. For the target source data in the target handling task, the pointer value of the write pointer of the target destination queue on the communication bus is first obtained from the target address, and then the pointer value is used to determine whether the remaining storage space of the target destination queue is sufficient to store the target source data.

[0074] If the remaining storage space of the target destination queue is sufficient to store the target source data, a transfer task is generated, and the target transfer unit is called based on the transfer request to transfer the target source data. Here, "transferring the target source data" specifically refers to reading the target source data from the target source data storage address and then writing it into the target destination queue. Optionally, the target source data storage address can be obtained from the target first description information.

[0075] Optionally, if the remaining storage space of the target destination queue is insufficient to store the target source data, a preset time length may be waited, and when the preset time length is reached, the step of obtaining the pointer value of the write pointer of the target destination queue on the communication bus from the target address is returned to execute until the remaining storage space of the target destination queue is sufficient to store the target source data or the preset number of cycles is reached. That is, if the remaining storage space is insufficient, the data stored in the target destination queue may be waited for to be consumed by the asynchronous consumer end (the asynchronous consumer end may be various, for example, other hosts), and after waiting for the preset time length, the pointer value of the write pointer of the target destination queue on the communication bus is read again. If the data in the target destination queue is consumed, the pointer value read again will change, and the remaining storage space may be judged again based on the pointer value read again. If the remaining storage space is sufficient, the target source data may be written to the target destination queue. If the remaining storage space is still insufficient, the preset time length is waited again, the pointer value is read for the third time, the judgment is made for the third time, and so on. In order to avoid falling into an endless loop due to repeated judgments that the remaining storage space is still insufficient, the number of cycles may be set, and the pointer value is no longer read after reaching the preset number of cycles, and the judgment process is no longer performed.

[0076] In the case where the target transport task includes source data generated by multiple hosts, since the target transport unit has the ability to asynchronously process multiple tasks at the same time, in order to be able to asynchronously process the source data generated by each host in the target transport task, when processing the target source data, the predicted value of the write pointer of the target destination queue on the communication bus can be updated according to the size of the target source data. Optionally, the predicted value can be updated according to the size of the target source data when reading the target source data.

[0077] In the embodiment of the present application, after the above transfer request is generated, the next target source data can be determined (the above prediction value can reflect the real starting address of the next target source data written into the target destination queue), that is, the source data generated by any host that has not carried out source data transfer in the target transfer task is used as the target source data, and the target source data determined this time is transferred using the same process as the above process. And so on, until there are no more hosts and source data that have not carried out source data transfer in the target transfer task.

[0078] It is worth noting that for the target transport unit, after receiving each transport request, it can read the target source data and write it into the target destination queue based on the transport request. The subsequent host does not need to pay attention to the specific processing progress. When the target source data is written, a notification signal can be returned so that the host can know whether the target source data has been transported. That is, when the target transport unit receives the first transport request, it can transport the source data generated by the first host based on the first transport request, and return a notification signal to the first host after the transport is completed; when the target transport unit receives the second transport request, it can immediately transport the source data generated by the second host based on the second transport request in an asynchronous manner, and return a notification signal to the second host after the transport is completed; and so on.

[0079] It should also be noted that although the target transport unit can asynchronously process the source data generated by each host in the same group of transport tasks, since the identity of the target transport unit is unique and uses a unique bus identity, the source data generated by each host in the same group of transport tasks can only be transmitted sequentially on the communication bus, and thus the source data generated by each host can be written into the target destination queue in sequence, and the process of writing the source data of one host into the target destination queue will not be interrupted by the source data of other hosts, thereby ensuring atomicity.

[0080] Considering that in actual application scenarios, there may be multiple multi-producer queue components 11, and each of the multiple multi-producer queue components 11 may receive at least one set of first description information, and call the transport unit according to the received first description information, it is possible that multiple multi-producer queue components 11 call the same transport unit at the same time, for example, multi-producer queue component a and multi-producer queue component b simultaneously initiate transport requests to the same transport unit. In order to avoid transport errors at this time, arbitration can be performed through an arbitrator, where arbitration refers to arbitration for applying for transport unit resources.

[0081] Optionally, the basis for the arbitrator's arbitration can be the priority information carried by the transport request. Optionally, the priority information includes 4 categories, namely, highest, second highest, second lowest, and lowest. It is possible that multiple transport requests carry the same priority information. In this case, arbitration is performed according to the round-robin rule. After winning the arbitration, the same transport unit called by multiple transport requests will latch all transport requests and return a grant signal to the multi-producer queue component 11, so that the multi-producer queue component 11 can determine whether to win the arbitration for using the target transport unit. The multi-producer queue component 11 that wins the arbitration can perform transport processing on the source data based on the transport unit won.

[0082] Taking any of the target source data mentioned above as an example, when the multi-producer queue component 11 calls the target transport unit based on the transport request, it determines whether it wins the arbitration for using the target transport unit. If so, it performs transport processing on the target source data based on the target transport unit.

[0083] This application uses an arbitration mechanism to enable the transport unit to carry out data transport in an orderly manner, thereby avoiding errors in the data transport process and improving transport efficiency.

[0084] In some embodiments of the present application, the process of the multi-producer queue 11 obtaining at least one set of first description information based on the write register content written into its own cache space is introduced.

[0085] Optionally, the multi-producer queue component 11 can first determine whether its own cache space is not empty. If so, it means that the write register content has been written into its own cache space. Then, its own state machine can be started. After its own state machine is started, at least one set of first description information is obtained based on the write register content written into its own cache space.

[0086] Optionally, this embodiment can write the storage address of at least one set of first description information into the write register content, then the present application can parse the write register content written into its own cache space to obtain the storage address of at least one set of first description information, and then obtain at least one set of first description information from the storage address of at least one set of first description information.

[0087] In order to enable those skilled in the art to better understand the working mechanism of the multi-producer queue component of the present application, the working mechanism is summarized and introduced as follows.

[0088] If MPQ obtains multiple groups of first-level descriptors, each group of first-level descriptors can be processed in parallel, wherein the processing process of each group of first-level descriptors can refer to the working process of the following multi-producer queue component.

[0089] First, each symbol is introduced as follows. The first-level descriptor M1 is used to describe a group of transfer tasks P. The transfer task P is used to transfer the source data X=(x1, x2, ..., xn) generated by multiple hosts (host 1~host n) to the destination queue Q. The second-level descriptor M2 is used to describe the destination queue Q. The multi-producer queue component MPQ calls the transfer unit B to asynchronously process the transfer task P.

[0090] The multi-producer queue component first determines whether its own cache space (i.e., MPQ buffer) is not empty. If not, it continues to wait until the MPQ buffer is not empty, indicating that the write register content is stored in the MPQ buffer.

[0091] The state machine of MPQ is started, and MPQ parses the write register content MDP to obtain the storage address of the first-level descriptor M1. The first-level descriptor M1 can be obtained from the storage address of the first-level descriptor M1.

[0092] MPQ parses the primary descriptor M1, obtains the storage address of the secondary descriptor M2, and obtains the secondary descriptor M2 from the storage address of the secondary descriptor M2.

[0093] The secondary descriptor M2 here is a specific form of the second description information mentioned above and is not intended to limit the second description information.

[0094] By parsing the secondary descriptor M2, the address of the write pointer of the destination queue Q on the communication bus can be obtained.

[0095] The untransferred source data in X is used as the target source data, and MPQ obtains the pointer value of the write pointer of the destination queue Q on the communication bus from the address of the write pointer of the destination queue Q on the communication bus.

[0096] The pointer value is used to determine whether the remaining storage space of the destination queue Q is sufficient to store the target source data. If so, a DMA request (i.e., a transfer request) is generated, and arbitration for the transfer unit B resources is initiated based on the DMA request.

[0097] Determine whether the arbitration is won. If not, regenerate the DMA request until the arbitration is won.

[0098] On the one hand, the target source data is transported based on the transport unit B; on the other hand, the predicted value of the write pointer of the destination queue Q on the communication bus is updated according to the size of the target source data, and the step of using the untransported source data in X as the target source data is returned. When there is no source data to be transported in X, the state machine returns to idle.

[0099] The embodiments of the present application have the following advantages: First, by using a two-level descriptor to address multiple times, an MPQ can continuously process transport tasks pointing to different destination queues, thereby increasing the hardware resource reuse rate and improving the system flexibility; Second, by caching the register contents in the MPQ buffer, a large number of multi-host synchronization tasks can be processed without intervals or bubbles (inflight), thereby improving the efficiency of multi-host synchronization tasks; Third, the status of the destination queue is automatically updated by post-task processing controlled by the MPQ state machine, making the software work more convenient and efficient.

[0100] Corresponding to the multi-host task synchronizer provided above, the present application also provides a multi-host task synchronization method, which is applied to host software that can interact with the multi-host task synchronizer.

[0101] See also Figure 4 , is a flowchart of a multi-host task synchronization method provided by this application. Figure 4 As shown, the method may include:

[0102] Step S401: Obtain at least one set of transport tasks.

[0103] Each group of transport tasks is used to write source data generated by at least one host into the same destination queue. Different groups of transport tasks correspond to different destination queues, that is, different groups of transport tasks write into different destination queues.

[0104] In an embodiment of the present application, when a multi-host synchronization task is required, the host will first store the source data generated by itself in an independent storage space, which may be local or not. However, the storage space stored by each host is generally relatively fixed, and the destination queue to which the source data generated by each host is to be written is often also fixed, which makes most of the information in the first description information and the second description information relatively fixed. Therefore, in order to improve the efficiency of multi-host synchronization tasks, the first description information corresponding to multiple common handling tasks and the second description information corresponding to multiple common destination queues can be pre-configured. Here, the second description information corresponding to each common destination queue is the information describing the common destination queue.

[0105] For example, the first description information M may include the starting address of the source data a generated by the host a in the storage space and the starting address of the destination queue b to which the source data a is to be written. Since these two starting addresses are relatively fixed, the first description information M may be pre-set based on these two starting addresses.

[0106] It is understandable that for actual application scenarios, there may be a situation where the pre-configured first description information and / or second description information does not match the actual handling task. In this case, after obtaining at least one set of handling tasks, part of the pre-configured first description information and / or second description information can be modified according to the obtained handling tasks.

[0107] The present application hardens and solidifies the software behavior of the system by level by using two description information, places a large amount of operation in preprocessing before the synchronization work starts, reduces the frequency of reading and writing when the software is working, and improves the system performance.

[0108] Step S402: Determine a multi-producer queue component.

[0109] After at least one group of transport tasks is acquired in step S401, the present application may determine a multi-producer queue component required to process the at least one group of transport tasks.

[0110] Optionally, this step may determine one multi-producer queue component or multiple multi-producer queue components for processing the at least one group of transport tasks mentioned above.

[0111] For example, assuming that at least one group of handling tasks includes two groups of handling tasks, namely, a first group of handling tasks consisting of handling tasks 1 to 3 and a second group of handling tasks consisting of handling tasks 4 to 5, then this step can determine two multi-producer queue components, each of which handles a group of handling tasks; this step can also determine five multi-producer queue components, each of which handles one handling task; this step can also determine one multi-producer queue component, which handles the above two groups of handling tasks.

[0112] It should be noted that in the above example, no matter how many multi-producer queue components are finally determined, when the multi-producer queue component calls the transport unit, the same transport unit needs to be called for the same group of transport tasks, that is, the same transport unit needs to be called for transport tasks 1 to 3, and the same transport unit needs to be called for transport tasks 4 to 5, so as to ensure atomicity.

[0113] In a possible implementation, the multi-producer queue component in this step may be determined based on a preconfigured scheduling strategy and / or the working state of the multi-producer queue component, where the scheduling strategy is used to indicate how to schedule the multi-producer queue component.

[0114] Optionally, the scheduling strategy includes: selecting different multi-producer queue components for different types of handling tasks. For example, among a large number of synchronous tasks, there are small-sized, short-path type A tasks and large-sized, long-path type B tasks. Since these two types of tasks may consume and occupy different system resources, have different frequencies of occurrence, and have different urgency of task processing, different MPQ hardware can be used to alleviate the system performance bottleneck caused by insufficient MPQ resources.

[0115] Step S403: Generate write register content.

[0116] In the present application, write register content can be generated based on at least one group of acquired transport tasks, so that the subsequent multi-producer queue component can obtain first description information corresponding to at least one group of transport tasks based on the write register content.

[0117] Step S404, storing the write register content into the cache space of the multi-producer queue component, so that the multi-producer queue component obtains the first description information corresponding to at least one group of transfer tasks according to the write register content written into its own cache space, and calls different transfer units for the first description information corresponding to different groups of transfer tasks, so as to asynchronously process the corresponding group of transfer tasks on the communication bus based on the called transfer units and the corresponding first description information, and when asynchronously processing the source data generated by each host in the corresponding group of transfer tasks, update the predicted value of the write pointer of the corresponding target queue on the communication bus according to the size of the source data generated by the host, wherein the transfer unit uses a unique bus identity when asynchronously processing the corresponding group of transfer tasks, and the first description information corresponding to a group of transfer tasks is used to describe the group of transfer tasks.

[0118] In an optional embodiment, before storing the write register content in the cache space of the multi-producer queue component, it is also possible to first determine whether the remaining space in the cache space of the multi-producer queue component is sufficient to store the write register content. If so, the write register content is stored in the cache space of the multi-producer queue component.

[0119] See also Figure 5 , which is a schematic diagram of the interaction between a host software, a multi-host task synchronizer, and multiple hosts.

[0120] like Figure 5 As shown, the host software may pre-configure a plurality of first description information (eg Figure 5 The first description information 1, the first description information 2 and the first description information 3) and a plurality of second description information (eg Figure 5 The second description information 1, the second description information 2 and the second description information 3 are shown. Each first description information is information describing a transport task, and each second description information is information describing a destination queue. The pre-configured first description information and the second description information can be put together and managed uniformly by the host software. Figure 5 The host software can also manage the individual destination queues (e.g. Figure 5 1 shows the read and write pointers of the first destination queue 130, the second destination queue 131 and the third destination queue 132, for example Figure 5 Read pointer 1 and write pointer 1 are read-write pointers in the first destination queue 130 , read pointer 2 and write pointer 2 are read-write pointers in the second destination queue 121 , and read pointer 3 and write pointer 3 are read-write pointers in the third destination queue 132 .

[0121] Host software can perform Figure 5It shows whether the first host 150, the second host 151 and the third host 152 generate source data for monitoring. When the host generates source data, that is, a transfer task is generated, the corresponding first description information and / or part of the second description information can be modified according to the transfer task. After that, the host software can determine the multi-producer queue component 11 (i.e., MPQ) and determine whether the cache space of the determined MPQ is sufficient. If so, the write register content will be generated and stored in the cache space of the MPQ, so that the MPQ can transfer data according to the working process described above.

[0122] For the same group of transport tasks, the present application can call the same transport unit 12 to use a unique bus identity to ensure atomicity. By writing the write register content in the cache space of the multi-producer queue component MPQ, each host can generate a transport task and send it to the MPQ hardware without having to pay attention to the subsequent transport progress. Writing to the MPQ cache space allows the subsequent transport unit to asynchronously process a group of transport tasks, realizing the processing of a large number of multi-host synchronization tasks without intervals and bubbles, solving the problem of a single host exclusively accessing the bus, improving synchronization efficiency, reducing host occupancy, and improving system performance.

[0123] It should be noted that the parts not described in detail in the multi-host task synchronization method provided in this application can be referred to the introduction of the multi-host task synchronizer in the previous text, and the two can be referenced to each other and will not be repeated here.

[0124] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0125] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0126] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0127] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A multi-host task synchronizer, characterized in that: include: A multi-producer queue component, a transport unit, a destination queue storage unit and a communication bus, wherein the multi-producer queue component, the transport unit and the destination queue storage unit are connected to the communication bus, the multi-producer queue component is also in communication connection with the transport unit, and the destination queue storage unit stores at least one destination queue; The multi-producer queue component is used to obtain at least one set of first description information based on the write register content written into its own cache space, wherein each set of the first description information is used to describe a set of transport tasks, and the set of transport tasks is used to write source data generated by at least one host into the same destination queue, and different sets of the first description information correspond to different destination queues; The multi-producer queue component is further used to call different transport units for different groups of the first description information, so as to asynchronously process the corresponding group transport task on the communication bus based on the called transport unit and the first description information of the corresponding group, and when asynchronously processing the source data generated by each host in the corresponding group transport task, update the predicted value of the write pointer of the corresponding destination queue on the communication bus according to the size of the source data generated by the host, wherein the transport unit uses a unique bus identity when asynchronously processing the corresponding group transport task; For each set of first description information in the at least one set of first description information, the set of first description information is used as target first description information, the transport unit called for the target first description information is used as the target transport unit, and a set of transport tasks described by the target first description information is used as the target transport tasks; Then, when the multi-producer queue component asynchronously processes the target transport task on the communication bus based on the target transport unit and the target first description information, it is specifically used to: Parsing the first target description information to obtain a storage address of the second target description information, wherein the second target description information is information describing the target destination queue corresponding to the first target description information; Acquire the target second description information from the storage address of the target second description information, and parse the target second description information to obtain the address of the write pointer of the target destination queue on the communication bus as the target address; Using source data generated by any host that does not carry out source data transportation in the target transportation task as target source data, obtaining a pointer value of a write pointer of the target destination queue on the communication bus from the target address, and judging whether the remaining storage space of the target destination queue is sufficient to store the target source data according to the pointer value; If so, a transfer request is generated, and based on the transfer request, the target transfer unit is called to transfer the target source data. After generating the transfer request, the process returns to the step of using the source data generated by any host that has not transferred the source data in the target transfer task as the target source data.

2. The multi-host task synchronizer according to claim 1, characterized in that: When the multi-producer queue component calls the target transport unit based on the transport request to transport the target source data, the multi-producer queue component is further used to: When the target transport unit is called based on the transport request, it is determined whether arbitration for using the target transport unit is won. If so, the target source data is transported based on the target transport unit, wherein the transport request carries priority information for the arbitration.

3. The multi-host task synchronizer according to claim 1, characterized in that: When the multi-producer queue component asynchronously processes the target transport task on the communication bus based on the target transport unit and the target first description information, the multi-producer queue component is further configured to: If the remaining storage space of the target destination queue is insufficient to store the target source data, wait for a preset time length, and when the preset time length is reached, return to execute the step of obtaining the pointer value of the write pointer of the target destination queue on the communication bus from the target address, until the remaining storage space of the target destination queue is sufficient to store the target source data or the preset number of cycles is reached.

4. The multi-host task synchronizer according to claim 1, characterized in that: The multi-producer queue component, when acquiring at least one set of first description information based on the write register content written into its own cache space, is specifically used to: Determine whether the own cache space is not empty; If so, the at least one set of first description information is obtained based on the write register content written into the own cache space after the own state machine is started.

5. The multi-host task synchronizer according to claim 4, characterized in that: The multi-producer queue component, when acquiring the at least one set of first description information based on the write register content written into the own cache space, is specifically used to: Parsing the write register content written into the own cache space to obtain the storage address of the at least one set of first description information; The at least one set of first description information is obtained from a storage address of the at least one set of first description information.

6. The multi-host task synchronizer according to any one of claims 1 to 5, characterized in that: The transfer unit is a linear transfer unit in direct register access DMA.

7. A multi-host task synchronization method, characterized in that: Applies to host software, including: Obtain at least one group of transfer tasks, wherein each group of transfer tasks is used to write source data generated by at least one host into the same destination queue, and different groups of transfer tasks correspond to different destination queues; Identify multi-producer queue components; Generate write register content; The write register content is stored in the cache space of the multi-producer queue component, so that the multi-producer queue component obtains the first description information corresponding to at least one group of transport tasks according to the write register content written into its own cache space, and calls different transport units for the first description information corresponding to different groups of transport tasks, so as to asynchronously process the corresponding group of transport tasks on the communication bus based on the called transport units and the corresponding first description information, and when asynchronously processing the source data generated by each host in the corresponding group of transport tasks, the predicted value of the write pointer of the corresponding target queue on the communication bus is updated according to the size of the source data generated by the host, wherein the transport unit uses a unique bus identity when asynchronously processing the corresponding group of transport tasks, and the first description information corresponding to a group of transport tasks is used to describe the group of transport tasks; For each set of first description information in at least one set of first description information, the set of first description information is used as target first description information, the transport unit called for the target first description information is used as the target transport unit, and a set of transport tasks described by the target first description information is used as the target transport tasks; Then, when the multi-producer queue component asynchronously processes the target transport task on the communication bus based on the target transport unit and the target first description information, it includes: Parsing the first target description information to obtain a storage address of the second target description information, wherein the second target description information is information describing the target destination queue corresponding to the first target description information; Acquire the target second description information from the storage address of the target second description information, and parse the target second description information to obtain the address of the write pointer of the target destination queue on the communication bus as the target address; Using source data generated by any host that does not carry out source data transportation in the target transportation task as target source data, obtaining a pointer value of a write pointer of the target destination queue on the communication bus from the target address, and judging whether the remaining storage space of the target destination queue is sufficient to store the target source data according to the pointer value; If so, a transfer request is generated, and based on the transfer request, the target transfer unit is called to transfer the target source data. After generating the transfer request, the process returns to the step of using the source data generated by any host that has not transferred the source data in the target transfer task as the target source data.

8. The multi-host task synchronization method according to claim 7, characterized in that: Before obtaining at least one group of transport tasks, the method further includes: First description information corresponding to a plurality of common handling tasks and second description information corresponding to a plurality of common destination queues are pre-configured, wherein the second description information corresponding to each common destination queue is information describing the common destination queue.

9. The multi-host task synchronization method according to claim 7, characterized in that: The multi-producer queue component is determined according to a pre-configured scheduling strategy and / or a working state of the multi-producer queue component, wherein the scheduling strategy is used to indicate how to schedule the multi-producer queue component.

Images