Inter-chip set communication method and system for optimizing communication performance
By adding a collective communication management device to the downstream chip, the sequential management between the identification bit slice and the application data slice is realized, and the problems of waste of communication time and increase management costs in the prior art are solved, and the efficiency and performance of inter-chip collective communication are improved.
Patent Information
- Application Number
- CN202411825695.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-12
- Publication Date
- 2025-04-22
AI Technical Summary
When the existing inter-chip collective communication method ensures data accuracy, the upstream chip needs to wait for the downstream chip to return a successful write response, resulting in wasted communication time and increased management costs, which affects communication performance.
Add a collective communication management device to the downstream chip, including a transaction sorting FIFO memory and a slice-registered FIFO memory. Through ID rearrangement number and response information management, it ensures that the identification bit slice arrives later than the application data slice, and allows the upstream chip to continuously issue transaction slices.
It reduces the software management consumption and cost of upstream chips, optimizes the efficiency of collective communication between chips, reduces communication waiting time, and improves the application speed of downstream chips.
Smart Images
Figure CN119862149A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of inter-chip collective communication, and in particular to an inter-chip collective communication method and system for optimizing communication performance. Background Art
[0002] Collective communication between different chips is a common algorithm in parallel computing and is widely used in various scenarios. With the development of large models, distributed training and reasoning have become the current mainstream mode, often requiring thousands or even tens of thousands of GPU cards for ultra-large-scale parallel training. This is a typical inter-chip collective communication scenario, and the performance of inter-chip collective communication has become the key to performance assurance in these application scenarios.
[0003] The upstream and downstream chips in the collective communication process the original transaction by splitting it into smaller slices. The upstream chip splits the application data of an original transaction into multiple application data slices and sends them to the downstream chip to write into the on-chip memory. The upstream chip also generates an identification bit slice for an original transaction and sends it to the downstream chip to write into the on-chip memory. The identification bit slice of an original transaction is used to indicate that the application data of the original transaction has been written into the on-chip memory of the downstream chip. Therefore, the order between the multiple application data slices split from an original transaction and the identification bit slice of the original transaction must be strictly guaranteed. The landing of the identification bit slice of the original transaction must be later than the landing of all the application data slices of the original transaction. Only in this way will there be no data errors when the downstream chip processes the slice data.
[0004] In order to meet the above order to ensure the accuracy of data processing by the downstream chip, the existing inter-chip collective communication method is: the upstream chip sends multiple application data slices obtained by splitting an original transaction in sequence, and the downstream chip returns a response to the upstream chip after writing each application data slice. The upstream chip waits until it receives the response of all application data slices of an original transaction before sending the identification bit slice of the original transaction to the downstream chip. Although this inter-chip collective communication method ensures data accuracy, the software of the upstream chip needs to collect the response of the application data slice and determine whether to send the identification bit slice, which requires a lot of management costs. In addition, the identification bit slice can only be sent after the application data slice response is collected, which not only wastes communication time but also affects the downstream pick-up time. These will affect the performance of inter-chip collective communication. Summary of the invention
[0005] In view of the above problems and technical requirements, this application proposes an inter-chip collective communication method for optimizing communication performance. The technical solution of this application is as follows:
[0006] An inter-chip collective communication method for optimizing communication performance, the downstream chip includes a collective communication management device, the collective communication management device includes a transaction sorting FIFO memory and a slice register FIFO memory, and the inter-chip collective communication method performed by the downstream chip using the collective communication management device includes:
[0007] Receive multiple transaction slices of an original transaction continuously sent by the upstream chip, where the multiple transaction slices sent in sequence for each original transaction include multiple application data slices and identification bit slices obtained by splitting;
[0008] The source ID information of each received transaction slice is sequentially written into the transaction sorting FIFO memory and the write pointer value is used as the rearrangement ID information of the transaction slice. Each received application data slice carries its rearrangement ID information and is sent to the on-chip memory for data writing. The received identification bit slice and its rearrangement ID information are correspondingly stored in the slice storage FIFO memory.
[0009] When detecting the response information carrying its own rearrangement ID information returned by each application data slice after writing to the on-chip memory is completed, the response identifier of the corresponding source ID information in the transaction sorting FIFO memory is set to a valid state according to the rearrangement ID information carried in the response information;
[0010] Based on the read pointer, the source ID information corresponding to the response identifier in the valid state is read from the transaction sorting FIFO memory in sequence, and when it is determined based on the read pointer that the source ID information of the identification bit slice is read, the identification bit slice and its rearranged ID information stored in the slice FIFO memory are sent to the on-chip memory.
[0011] A further technical solution is that the source ID information corresponding to the response identifier being in a valid state is sequentially read from the transaction sorting FIFO memory based on the read pointer, including:
[0012] When it is detected that the response identifier of the source ID information corresponding to the current read pointer is in a valid state, the read pointer is updated by one, otherwise the current read pointer is kept unchanged;
[0013] When it is detected that the value of the current read pointer is consistent with the rearranged ID information of the identification bit slice in the slice storage FIFO memory, it is determined to send the identification bit slice and its rearranged ID information to the on-chip memory.
[0014] A further technical solution is that the inter-chip collective communication method further includes:
[0015] The received response information is stored in the entry of the source ID information corresponding to the rearrangement ID information carried by the response information in the transaction sorting FIFO memory; when it is detected that the response identifier of the source ID information corresponding to the current read pointer is in a valid state, the rearrangement ID information carried by the response information stored in the entry of the source ID information corresponding to the current read pointer is replaced with the source ID information to generate a successful write response and return it to the upstream chip, and the response identifier of the source ID information is reset to an invalid state, and the read pointer is increased by one.
[0016] A further technical solution is that the collective communication management device further includes an overflow timer, and the inter-chip collective communication method further includes:
[0017] When sending any slice to the on-chip memory, the overflow timer is started;
[0018] When a response message carrying its own rearrangement ID information is received after the slice sent down is written into the on-chip memory before the overflow timer reaches the overflow time, the response identifier of the corresponding source ID information in the transaction sorting FIFO memory is set to a valid state according to the rearrangement ID information carried in the response message, and the rearrangement ID information carried in the response message is replaced with the source ID information of the slice to generate a write success response and return it to the upstream chip;
[0019] When the overflow timer reaches the overflow time and the response information carrying its own rearrangement ID information returned by the sent slice has not been received, the response flag of the corresponding source ID information in the transaction sorting FIFO memory is set to a valid state according to the slice's rearrangement ID information, and a write failure response carrying the slice's source ID information is generated and returned to the upstream chip.
[0020] A further technical solution is that the inter-chip collective communication method further includes:
[0021] Determine, according to the connected configuration register, an application data address range composed of the storage addresses of all application data slices sent by the upstream chip and an identification bit address range composed of the storage addresses of all identification bit slices sent by the upstream chip;
[0022] For any transaction slice received, when it is detected that the storage address of the transaction slice is within the application data address range, the type of the transaction slice is determined to be an application data slice; when it is detected that the storage address of the transaction slice is within the identification bit address range, the type of the transaction slice is determined to be an identification bit slice.
[0023] A further technical solution is that the order in which the response information of each detected application data slice is returned is consistent or inconsistent with the writing order of the source ID information of each application data slice in the transaction sorting FIFO memory.
[0024] A further technical solution is that the inter-chip collective communication method further includes:
[0025] Receive multiple transaction slices of multiple original transactions continuously sent by the upstream chip, write the source ID information of the transaction slices belonging to the multiple original transactions into the transaction sorting FIFO memory in sequence, and store the identification bit slices of each original transaction and their corresponding rearrangement ID information in the slice storage FIFO memory.
[0026] Its further technical solution is to receive the response information returned by the application data slices of other original transactions after detecting the response information returned by all application data slices of an original transaction;
[0027] Alternatively, before the response information returned by all application data slices of an original transaction is detected, the response information returned by the application data slices of other original transactions is received.
[0028] A further technical solution is that the downstream chip includes a plurality of collective communication management devices, and the inter-chip collective communication method further includes:
[0029] Receive transaction slices of original transactions continuously sent by multiple different upstream chips, and use different collective communication management devices to send the transaction slices of original transactions sent by different upstream chips to the on-chip memory.
[0030] An inter-chip collective communication system for optimizing communication performance, the inter-chip collective communication system comprising an upstream chip and a downstream chip, the downstream chip comprising a collective communication management device, the collective communication management device comprising a transaction sorting FIFO memory and a slice register FIFO memory;
[0031] The upstream chip is used to continuously send multiple transaction slices of an original transaction, and the multiple transaction slices sent in sequence for each original transaction include multiple application data slices and identification bit slices obtained by splitting;
[0032] The downstream chip is used to use the collective communication management device to send multiple transaction slices of each original transaction to the on-chip memory according to the inter-chip collective communication method of the first part.
[0033] The beneficial technical effects of this application are:
[0034] The present application discloses an inter-chip collective communication method and system for optimizing communication performance. The inter-chip collective communication system adds a collective communication management device in the downstream chip. The collective communication management device performs ID reordering and response information management on the received transaction slices, which can ensure that the identification bit slice of each original transaction is landed later than the previous application data slice. In this way, the upstream chip can send the identification bit slice without waiting for the downstream chip to return the write success response one by one, and can realize the continuous sending of transaction slices. It is only necessary to ensure the sending order of application data slices and identification bit slices, which reduces the software management consumption and cost of the upstream chip to collect downstream responses and then send identification bit slices. And the intra-chip sending mechanism and response management mechanism of the collective communication management device in the downstream chip for the transaction slice are automatically performed based on hardware using the write pointer and read pointer of the FIFO memory, so the software management cost of the downstream chip is not greatly increased, thereby optimizing the efficiency of inter-chip collective communication. In addition, after the downstream chip receives all the previous data slice responses within the chip, it can quickly send the identification bit slices to the on-chip memory within the chip without waiting for the upstream chip to receive all the previous responses. Therefore, the downstream chip's identification is set to valid much faster than the upstream one. Therefore, the downstream chip can also quickly start the application, saving communication waiting time and further optimizing the efficiency of inter-chip collective communication.
[0035] The collective communication management device implements the inter-chip collective communication method, which can be applied to the out-of-order return of the response information of the application data slice, even if the response information of the application data slice mixed with other original transactions is returned out of order, and in all cases, the response can be returned to the upstream chip in order, so that the upstream chip can continuously send transaction slices of multiple original transactions. In addition, it can be extended to the case where multiple collective communication management devices are used to support the independent order of multiple upstream chips, with a wide range of applications and high versatility. The collective communication management device also has a timeout judgment anti-hanging mechanism, which can avoid the occurrence of upstream and downstream collective communication hangs caused by transaction slice hangs. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 It is a schematic diagram of the system structure of an inter-chip collective communication system according to an embodiment of the present application.
[0037] Figure 2 It is a flowchart of a method in which a collective communication management device sends each received transaction slice to an on-chip memory.
[0038] Figure 3 It is a flow chart of updating the transaction sorting FIFO memory after the collective communication management device receives the response information. DETAILED DESCRIPTION
[0039] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.
[0040] This application discloses an inter-chip collective communication system that optimizes communication performance. Please refer to Figure 1 As shown in the schematic diagram of the system structure, the inter-chip collective communication system includes an upstream chip and a downstream chip, and the downstream chip includes a collective communication management device.
[0041] In the inter-chip collective communication system, the upstream chip sends the transaction slices of the original transaction to the downstream chip to realize the inter-chip collective communication. The transaction slices of each original transaction sent by the upstream chip to the downstream chip include multiple application data slices obtained by splitting the application data of the original transaction and the identification bit slices of the original transaction. However, unlike the traditional practice in which the upstream chip first sends each application data slice to the downstream chip, and then sends the identification bit slice to the downstream chip after receiving the write response of each application data slice returned by the downstream chip, in this application, the upstream chip continuously sends multiple transaction slices of an original transaction, and the multiple transaction slices sent in sequence for each original transaction include multiple application data slices and identification bit slices obtained by splitting, that is, the upstream chip does not need to wait for the write response of all application data slices to be collected before sending the identification bit slice, but sends the identification bit slice together with each application data slice continuously, and only needs to ensure the sending order of each application data slice and the identification bit slice. The upstream chip does not need to manage the write response of each application data slice returned by the downstream chip, reducing the management loss and cost caused by this.
[0042] After the working mechanism of the upstream chip is changed to continuously send transaction slices, the application manages multiple transaction slices of each original transaction in the downstream chip through an added collective communication management device to ensure that the order requirement that the identification bit slice must be landed after multiple application data slices can still be met. The downstream chip executes the inter-chip collective communication method of the present application through the added collective communication management device, and can send the multiple transaction slices of each original transaction continuously sent by the upstream chip to the on-chip memory, and ensure that the identification bit slice is landed after multiple application data slices.
[0043] like Figure 1 As shown, the collective communication management device newly added in the downstream chip of the present application includes a transaction sorting FIFO memory and a slice storage FIFO memory. The inter-chip collective communication method implemented by the downstream chip using the collective communication management device includes the following process, please refer to Figure 2 The flowchart shown:
[0044] Step 210, receiving multiple transaction slices of an original transaction continuously sent by the upstream chip.
[0045] Step 220, write the source ID information of each received transaction slice into the transaction sorting FIFO memory in the collective communication management device in sequence, and use the value of the write pointer as the rearranged ID information of the transaction slice.
[0046] The source ID information of each transaction slice is used to uniquely identify a transaction slice. The source ID information of each transaction slice is written into the transaction sorting FIFO memory in the order of each received transaction slice. The write pointer is updated by one for each stored source ID information of a transaction slice, so that the ID reordering of the transaction slice can be completed. The source ID information stored in each entry in the transaction sorting FIFO memory also has a corresponding response identifier, which is invalid by default.
[0047] Step 230: For each received transaction slice, in addition to storing the source ID information of the transaction slice in the transaction sorting FIFO memory, the following operations are performed on the transaction slice:
[0048] For each received application data slice, the application data slice is directly sent to the on-chip memory for data writing. Since the source ID information of each application data slice has been replaced with the rearrangement ID information, each application data slice will actually be sent to the on-chip memory with its rearrangement ID information.
[0049] For each received identification bit slice, the identification bit slice and its rearranged ID information are first stored in the slice storage FIFO memory in the collective communication management device. Therefore, the transaction sorting FIFO memory needs to set the storage depth according to the memory access delay of the downstream chip to ensure that the outstanding capability can meet the needs of the local network delay of the downstream chip, thereby ensuring system performance.
[0050] Since different operations need to be performed on application data slices and identification bit slices, for each transaction slice received, the type of transaction slice must first be distinguished. In one embodiment, the type of transaction slice is distinguished by storage address. The collective communication is unified by software to plan the upstream and downstream address space, so as long as the downstream chip can know the type of slice sent by the upstream and then complete the corresponding sequence function. In this embodiment, a set of configuration registers is used to achieve the distinction, and the software needs to be configured according to the address situation of the entire network, giving the application data address range composed of the storage addresses of all application data slices sent by the upstream chip and the identification bit address range composed of the storage addresses of all identification bit slices sent by the upstream chip.
[0051] The downstream chip can determine the application data address range and the identification bit address range according to the connected configuration register. For any received transaction slice, when it is detected that the storage address of the transaction slice is within the application data address range, the type of the transaction slice is determined to be an application data slice, and when it is detected that the storage address of the transaction slice is within the identification bit address range, the type of the transaction slice is determined to be an identification bit slice.
[0052] Step 240, each application data slice sent to the on-chip memory will return a response message carrying its own rearrangement ID information after writing is completed. When the collective communication management device detects the response message, it can determine which application data slice the response message is returned from based on the rearrangement ID information carried in the response message. Since the source ID information of each application data slice corresponds to the rearrangement ID information one-to-one, the corresponding source ID information in the corresponding transaction sorting FIFO memory can be located based on the rearrangement ID information of the application data slice carried in the response message. Then, the response identifier of the source ID information corresponding to the rearrangement ID information carried by the response message in the transaction sorting FIFO memory is set to a valid state. In addition, the received response information will be stored in the entry of the source ID information corresponding to the rearrangement ID information carried by the response message in the transaction sorting FIFO memory. Please refer to Figure 3 Flowchart shown.
[0053] Step 250, based on the read pointer, sequentially reads source ID information corresponding to the response identifier in a valid state from the transaction sorting FIFO memory, including:
[0054] When it is detected that the response identifier of the source ID information corresponding to the current read pointer is in an invalid state, the current read pointer is kept unchanged.
[0055] When the response identifier of the source ID information corresponding to the current read pointer is detected to be in a valid state, the read pointer is updated by one. In addition, the rearranged ID information carried by the response information stored in the entry of the source ID information is replaced with the source ID information to generate a write success response containing the source ID information of the application data slice and return it to the upstream chip, and reset the response identifier of the source ID information to an invalid state. The write success response returned to the upstream chip is used to indicate that the application data slice with the source ID information has been successfully written into the on-chip memory of the downstream chip.
[0056] In the update process of the read pointer, when it is determined based on the read pointer that the identification bit slice re-arrangement ID information is read, the identification bit slice and its re-arrangement ID information in the slice storage FIFO memory are sent to the on-chip memory. Specifically, as recorded in step 230, the received identification bit slice has been stored corresponding to the re-arrangement ID information, and the re-arrangement ID information of the identification bit slice is the value of the write pointer when the identification bit slice is written into the transaction sorting FIFO memory. Therefore, when it is detected that the value of the current read pointer is consistent with the re-arrangement ID information of the identification bit slice in the slice storage FIFO memory, it can be determined that the re-arrangement ID information of the identification bit slice is read. At this time, it indicates that the response identifier of the source ID information of all application data slices preceding the identification bit slice has been set to an over-valid state, that is, it indicates that all application data slices preceding the identification bit slice have been successfully written into the on-chip memory and received the corresponding response information. Therefore, the identification bit slice can be sent to the on-chip memory, thereby ensuring that the identification bit slice is landed later than the application data slice. Moreover, at this time, the downstream chip does not need to wait for the upstream chip to receive the response information of all application data slices before sending the identification slices. Instead, the downstream chip quickly sends the identification slices within the chip when the response information of all application data slices is received within the chip. Compared with the traditional mechanism sent by the upstream chip, the downstream chip can start the application more quickly, thereby improving communication efficiency and reducing inter-chip link consumption.
[0057] Since the present application performs ID reordering upon receiving the transaction slices through step 220, and reorders the returned response information through steps 240 and 250, even if the application data slices preceding the identification bit slices are returned out of order, the order preservation between the identification bit slices and the application data slices will not be affected. That is, when receiving the response information returned by each application data slice, the order of the response information returned by each application data slice detected may be consistent with or inconsistent with the order in which the source ID information of each application data slice is written in the transaction sorting FIFO memory.
[0058] For example, in one instance, see Figure 1 As shown, the transaction slices of an original transaction continuously sent down by the upstream chip include 3 application data slices (with source ID information 1 to source ID information 3 in sequence) and 1 identification bit slice (with source ID information 4). The downstream chip first receives the application data slice 1 with source ID information 1, and then stores the source ID information 1 of the application data slice 1 in the transaction sorting FIFO memory. At this time, the value of the write pointer is used as the rearrangement ID information 1 of the application data slice 1, and the write pointer is automatically increased by one after writing. In addition, the application data slice 1 and its rearrangement ID information 1 are directly sent down to the on-chip memory.
[0059] Then the downstream chip receives the application data slice 2 with the source ID information 2. Similarly, the source ID information 2 is stored in the transaction sorting FIFO memory, the write pointer is automatically increased by one, and the application data slice 2 and its rearranged ID information 2 are directly sent to the on-chip memory.
[0060] Then the downstream chip receives the application data slice 3 with the source ID information 3. Similarly, the source ID information 3 is stored in the transaction sorting FIFO memory, the write pointer is automatically increased by one, and the application data slice 3 and its rearranged ID information 3 are directly sent to the on-chip memory.
[0061] Then the downstream chip receives the identification bit slice 1 with the source ID information 4, stores the source ID information 4 in the transaction sorting FIFO memory, automatically increases the write pointer by one, and stores the identification bit slice 1 and its rearranged ID information 4 in the slice storage FIFO memory.
[0062] The order in which the response information of application data slices 1 to 3 sent by the collective communication management device to the on-chip memory is returned is not certain. The response information may be returned in the order in which the application data slices 1 to 3 are written, or it may be returned in a disordered order. Assuming that the response information is returned in the order of application data slice 3, application data slice 1, and application data slice 2, the collective communication management device processes the response information as follows:
[0063] The collective communication management device first receives the response information carrying the rearrangement ID information 3, then stores the response information in the entry of the source ID information 3 corresponding to the rearrangement ID information 3, and sets the response flag of the source ID information 3 to a valid state. However, at this time, the read pointer is still at the entry of the source ID information 1, and since the response flag of the source ID information 1 is still in an invalid state, the read pointer remains unchanged.
[0064] Then the collective communication management device receives the response information carrying the rearrangement ID information 1, stores the response information in the entry of the source ID information 1 corresponding to the rearrangement ID information 1, and sets the response identifier of the source ID information 1 to a valid state. At this time, the response identifier of the source ID information 1 corresponding to the current read pointer is in a valid state, so the rearrangement ID information 1 carried by the response information is replaced with the source ID information 1 to generate a write success response and return it to the upstream chip, and reset the response identifier of the source ID information 1 to an invalid state, and update the read pointer to automatically add 1 to the entry of the source ID information 2. Since the response identifier of the source ID information 2 is still in an invalid state, no further operation is performed.
[0065] When the collective communication management device receives the response information carrying the rearrangement ID information 2, the response information is stored in the entry of the source ID information 2 corresponding to the rearrangement ID information 2, and the response identifier of the source ID information 2 is set to a valid state. At this time, the response identifier of the source ID information 2 corresponding to the current read pointer is in a valid state, so the rearrangement ID information 2 carried by the response information is replaced with the source ID information 2 to generate a write success response and return it to the upstream chip, and the response identifier of the source ID information 2 is reset to an invalid state, and the read pointer is updated to automatically add 1 to the entry of the source ID information 3. Since the response identifier of the source ID information 3 is already in a valid state, the rearrangement ID information 3 carried by the response information stored at the entry of the source ID information 3 will continue to be replaced with the source ID information 3 to generate a write success response and return it to the upstream chip, and the response identifier of the source ID information 3 is reset to an invalid state, and the read pointer is updated to automatically add 1 to the entry of the source ID information 4. At this time, it is detected that the value of the current read pointer is consistent with the rearrangement ID information 4 of the identification bit slice, so that the identification bit slice 1 and its rearrangement ID information 4 in the slice storage FIFO memory are sent to the on-chip memory. From the above examples, it can be seen that although the response information of the application data slice preceding the identification bit slice is returned out of order, the order preservation requirement between the identification bit slice and the preceding application data slice can still be met. And from the examples, it can also be seen that the order of the write success response returned to the upstream chip is consistent with the order in which the application data slices are sent, that is, even if the application data slice returns the response information out of order inside the downstream chip, the write success response can still be returned to the upstream chip in order to achieve response reordering.
[0066] In another embodiment, the collective communication management device further includes an overflow timer, such as Figure 1 As shown, the inter-chip collective communication method also includes:
[0067] (1) When any slice is sent to the on-chip memory, the overflow timer is started.
[0068] (2) When a response message carrying the rearrangement ID information of the slice is received after the writing to the on-chip memory is completed before the overflow timer reaches the overflow time, the corresponding source ID information can be located according to the rearrangement ID information of the slice in accordance with the above method, and the response flag of the source ID information in the transaction sorting FIFO memory is set to a valid state, and the response message carrying the rearrangement ID information is replaced with the source ID information of the slice to generate a write success response and return it to the upstream chip, and the write success response indicates that the writing of the slice in the on-chip memory of the downstream chip is successful.
[0069] (3) When the overflow timer reaches the overflow time and the response information carrying the rearrangement ID information of the slice sent down is not received, the response flag of the corresponding source ID information in the transaction sorting FIFO memory is set to a valid state according to the rearrangement ID information of the slice, and a write failure response carrying the source ID information of the slice is generated and returned to the upstream chip. The write success response indicates that the write of the slice in the on-chip memory of the downstream chip has failed. That is, once a slice is sent down to the storage memory, the slice corresponding to the read pointer should return a response within the overflow time, otherwise the overflow timer will overflow and reset, and the read pointer will increase by one, so as to avoid the occurrence of upstream and downstream collective communication hang caused by the slice transaction hang.
[0070] In addition, in the inter-chip collective communication scenario, in order to improve the communication performance, the upstream chip will continuously send multiple transaction slices of multiple original transactions, and the identification bit slice of each original transaction will be sent after the application data slice of the original transaction. Correspondingly, the collective communication management device will receive multiple transaction slices of multiple original transactions continuously sent by the upstream chip, and then write the source ID information of the transaction slices belonging to the multiple original transactions into the transaction sorting FIFO memory in sequence, and store the identification bit slices of each original transaction and their corresponding rearrangement ID information in the slice storage FIFO memory. Due to the existence of multiple original transactions, the source ID information of each identification bit slice in the transaction sorting FIFO memory will be preceded by the source ID information of multiple application data slices belonging to the same original transaction, and the identification bit slice will also be followed by multiple source ID information of application data slices belonging to other original transactions, and the identification bit slices of multiple original transactions and their corresponding rearrangement ID information will be stored in sequence in the slice storage FIFO memory.
[0071] For example, Figure 1 In the example, the collective communication management device receives the application data slice with source ID information 1, the application data slice with source ID information 2, the identification bit slice 1 with source ID information 3, and the application data slice with source ID information 4 in sequence, and stores the source ID information of each transaction slice in the transaction sorting FIFO memory in sequence. Then the identification bit slice 1 and its preceding application data slices with source ID information 1 and source ID information 2 belong to the same original transaction, and the application data slice with source ID information 4 subsequent to the identification bit slice 1 belongs to another original transaction. In addition, the identification bit slices of each original transaction and their rearranged ID information are stored in sequence in the slice storage FIFO memory. The method of the present application can also be applied to this scenario to ensure that the identification bit slice of each original transaction is landed later than all its preceding application data slices.
[0072] The method of the present application can also be applied to the scenario where multiple original transactions are continuously issued. In this scenario, the collective communication management device will receive the response information returned by the application data slices of other original transactions after detecting the response information returned by all the application data slices of an original transaction. Alternatively, the collective communication management device receives the response information returned by the application data slices of other original transactions before detecting the response information returned by all the application data slices of an original transaction. That is, even if the response information of different original transactions is mixed and returned, thanks to the ID reordering and response reordering mechanism of the collective communication management device, the order preservation between the identification bit slice and the application data slice of each original transaction can be achieved.
[0073] In another embodiment, the downstream chip includes multiple collective communication management devices, then the inter-chip collective communication system includes multiple upstream chips and one downstream chip, each upstream chip sends transaction slices of the original transaction to the downstream chip respectively, and the corresponding downstream chip will receive the transaction slices of the original transaction continuously sent by multiple different upstream chips, and then use each collective communication management device to send the transaction slices of the original transaction sent by each upstream chip to the on-chip memory according to the above method, and use different collective communication management devices to send the transaction slices of the original transaction sent by different upstream chips to the on-chip memory.
[0074] The above are only preferred embodiments of the present application, and the present application is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included in the protection scope of the present application.
Claims
1. A method for inter-chip collective communication for optimizing communication performance, characterized in that: The downstream chip includes a collective communication management device, the collective communication management device includes a transaction sorting FIFO memory and a slice register FIFO memory, and the inter-chip collective communication method performed by the downstream chip using the collective communication management device includes: Receive multiple transaction slices of an original transaction continuously sent by the upstream chip, where the multiple transaction slices sent in sequence for each original transaction include multiple application data slices and identification bit slices obtained by splitting; The source ID information of each received transaction slice is sequentially written into the transaction sorting FIFO memory and the value of the write pointer is used as the rearrangement ID information of the transaction slice. Each received application data slice carries its rearrangement ID information and is sent to the on-chip memory for data writing. The received identification bit slice and its rearrangement ID information are correspondingly stored in the slice storage FIFO memory; When detecting the response information carrying its own rearrangement ID information returned by each application data slice after writing to the on-chip memory is completed, the response identifier of the corresponding source ID information in the transaction sorting FIFO memory is set to a valid state according to the rearrangement ID information carried by the response information; Based on the read pointer, the source ID information corresponding to the response identifier in the valid state is read from the transaction sorting FIFO memory in sequence, and when it is determined based on the read pointer that the source ID information of the identification bit slice is read, the identification bit slice and its rearranged ID information stored in the slice FIFO memory are sent to the on-chip memory.
2. The inter-chip collective communication method according to claim 1, characterized in that: Reading the source ID information corresponding to the response identifier being in a valid state from the transaction sorting FIFO memory in sequence based on the read pointer includes: When it is detected that the response identifier of the source ID information corresponding to the current read pointer is in a valid state, the read pointer is updated by one, otherwise the current read pointer is kept unchanged; When it is detected that the value of the current read pointer is consistent with the rearranged ID information of the identification bit slice in the slice storage FIFO memory, it is determined to send the identification bit slice and its rearranged ID information to the on-chip memory.
3. The inter-chip collective communication method according to claim 2, characterized in that: The inter-chip collective communication method further includes: The received response information is stored in the entry of the source ID information corresponding to the rearrangement ID information carried by the response information in the transaction sorting FIFO memory; when it is detected that the response identifier of the source ID information corresponding to the current read pointer is in a valid state, the rearrangement ID information carried by the response information stored in the entry of the source ID information corresponding to the current read pointer is replaced with the source ID information to generate a successful write response and return it to the upstream chip, and the response identifier of the source ID information is reset to an invalid state, and the read pointer is increased by one.
4. The inter-chip collective communication method according to claim 3, characterized in that: The collective communication management device further includes an overflow timer, and the inter-chip collective communication method further includes: When any slice is sent to the on-chip memory, the overflow timer is started; When receiving a response message carrying its own rearrangement ID information returned by the slice after writing into the on-chip memory is completed before the overflow timer reaches the overflow duration, setting the response identifier of the corresponding source ID information in the transaction sorting FIFO memory to a valid state according to the rearrangement ID information carried in the response message, and replacing the rearrangement ID information carried in the response message with the source ID information of the slice to generate a write success response and return it to the upstream chip; When the overflow timer reaches the overflow time and has not received the response information carrying its own rearrangement ID information returned by the slice, the response flag of the corresponding source ID information in the transaction sorting FIFO memory is set to a valid state according to the rearrangement ID information of the slice, and a write failure response carrying the source ID information of the slice is generated and returned to the upstream chip.
5. The inter-chip collective communication method according to claim 1, characterized in that: The inter-chip collective communication method further includes: Determine, according to the connected configuration register, an application data address range composed of the storage addresses of all application data slices sent by the upstream chip and an identification bit address range composed of the storage addresses of all identification bit slices sent by the upstream chip; For any transaction slice received, when it is detected that the storage address of the transaction slice is within the application data address range, the type of the transaction slice is determined to be an application data slice; when it is detected that the storage address of the transaction slice is within the identification bit address range, the type of the transaction slice is determined to be an identification bit slice.
6. The inter-chip collective communication method according to claim 1, characterized in that: The order in which the response information of each detected application data slice is returned is consistent with or inconsistent with the writing order of the source ID information of each application data slice in the transaction sorting FIFO memory.
7. The inter-chip collective communication method according to claim 1, characterized in that: The inter-chip collective communication method further includes: Receive multiple transaction slices of multiple original transactions continuously sent by the upstream chip, write the source ID information of the transaction slices belonging to the multiple original transactions into the transaction sorting FIFO memory in sequence, and store the identification bit slices of each original transaction and their corresponding rearrangement ID information in the slice storage FIFO memory.
8. The inter-chip collective communication method according to claim 7, characterized in that: After detecting the response information returned by all application data slices of an original transaction, the response information returned by the application data slices of other original transactions is received; Alternatively, before the response information returned by all application data slices of an original transaction is detected, the response information returned by the application data slices of other original transactions is received.
9. The inter-chip collective communication method according to claim 1, characterized in that: The downstream chip includes a plurality of collective communication management devices, and the inter-chip collective communication method further includes: Receive transaction slices of original transactions continuously sent by multiple different upstream chips, and use different collective communication management devices to send the transaction slices of original transactions sent by different upstream chips to the on-chip memory.
10. An inter-chip collective communication system for optimizing communication performance, characterized in that: The inter-chip collective communication system includes an upstream chip and a downstream chip, the downstream chip includes a collective communication management device, and the collective communication management device includes a transaction sorting FIFO memory and a slice registration FIFO memory; The upstream chip is used to continuously issue multiple transaction slices of an original transaction, wherein the multiple transaction slices issued in sequence for each original transaction include multiple application data slices and identification bit slices obtained by splitting; The downstream chip is used to send multiple transaction slices of each original transaction to the on-chip memory using the collective communication management device according to the inter-chip collective communication method described in any one of claims 1-9.