Method, system and terminal for out-of-order reorganization and accurate retransmission request of large packet data
By using the on-chip cache and off-chip cache and index information table in the hardware, the out-of-sequence reorganization and precise retransmission requests of large packet data are realized, and the existing technology does not support out-of-sequence reorganization and retransmission requests are solved. The performance degradation problem improves network performance and connectivity support capabilities.
Patent Information
- Application Number
- CN202210306418.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-27
- Publication Date
- 2025-02-07
- Estimated Expiration
- 2042-03-27
AI Technical Summary
The existing hardware supports reliable transmission mechanisms that do not support out-of-order reorganization of network packets. The use of Go-Bank-N retransmission mechanism when frame drops, resulting in performance degradation, which can only support more than one thousand connections, and timer delay Increase network delay.
By using the on-chip cache and off-chip cache and index information table to achieve out-of-order reorganization and precise retransmission requests of large packet data, it supports reliable transmission of data established by any two units in the network, and when frame drops Make accurate retransmission requests.
Improve network performance, accelerate big data processing, reduce network processing delay, support out-of-order reorganization and precise retransmission requests of thousands of connections, avoiding performance degradation caused by retransmission requests.
Smart Images
Figure CN114817082B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of hardware reliable transmission, and in particular relates to a method, system and terminal for out-of-order reorganization and precise retransmission request of large-packet data. Background Art
[0002] At present, in order to ensure the reliable transmission of data through hardware, it is necessary to consume more on-chip memory resources to save the data transmission status. Therefore, the existing hardware implementation technology usually adopts the Go-Bank-N retransmission mechanism, that is, after detecting frame loss, all the data after the data packet of the lost frame will be retransmitted. This will cause serious performance degradation when the frame is lost, and the throughput will be close to 0 when the packet loss rate is 1%.
[0003] The existing hardware supports reliable transmission mechanisms but does not support the reorganization of out-of-order network packets. Due to the influence of the paths and nodes that packets pass through in the network and the forwarding units, the data in the network is likely to arrive out of order. Some existing technologies retransmit out-of-order packets, which will further deteriorate performance and latency. Due to limited hardware resources, only about a thousand connections are supported.
[0004] At the same time, the existing hardware-guaranteed reliable transmission mechanism only detects data packet loss through the cooperation of timers and confirmation response mechanisms, which is not flexible enough in the network. In addition, due to the delay of the timer, the network delay will be further increased when packets are lost. Hardware-based reliable transmission is generally aimed at time-sensitive networks.
[0005] In addition, the existing hardware-guaranteed reliable transmission mechanism is limited by on-chip memory resources. When frame loss is detected, the Go-Bank-N retransmission mechanism is used, which will cause serious performance degradation when frame loss occurs. The hardware-guaranteed reliable transmission mechanism does not support the out-of-order reorganization of network data packets. Affected by the network path and the network node forwarding unit, network data may arrive out of order, but the application uses the data sequentially, so out-of-order packet reorganization must be performed on the hardware. Some existing mechanisms retransmit out-of-order data packets, which will lead to deterioration of performance and delay. The existing hardware-guaranteed reliable transmission mechanism is limited by hardware resources and can only support about one thousand connections. Therefore, it is urgent to design a new out-of-order reorganization and precise retransmission request method and system to make up for the shortcomings of the existing technology.
[0006] Through the above analysis, the problems and defects of the prior art are as follows:
[0007] (1) The existing hardware-supported reliable transmission mechanism does not support the reorganization of out-of-order network data packets. It only detects data packet loss through the coordination of timers and confirmation response mechanisms, which is not flexible enough in the network.
[0008] (2) Due to the delay of the timer, the existing hardware-guaranteed reliable transmission mechanism will further increase the network delay when packet loss occurs, and hardware-based reliable transmission is generally aimed at time-sensitive networks.
[0009] (3) The existing hardware-guaranteed reliable transmission mechanism is limited by the on-chip memory resources. When frame loss is detected, the Go-Bank-N retransmission mechanism is used, which will cause serious performance degradation when frame loss occurs.
[0010] (4) The existing mechanism is to retransmit out-of-order data packets, which will lead to deterioration of performance and latency. The existing hardware-guaranteed reliable transmission mechanism is limited by hardware resources and can only support about one thousand connections. Summary of the invention
[0011] In view of the problems existing in the prior art, the present invention provides a method, system and terminal for out-of-order reorganization and precise retransmission request of large packet data, and more particularly, relates to a method, system, device and terminal for out-of-order reorganization and precise retransmission request of large packet data based on hardware guarantee.
[0012] The present invention is implemented as follows: a method for disorderly reorganization and precise retransmission request of large packet data, the method comprising: utilizing an on-chip cache for storing data frames, an index information table for storing data storage addresses and information, and storing the data frames in an off-chip cache when the on-chip storage is full; utilizing the cooperation of on-chip and off-chip caches and index information tables to realize frame loss detection and precise retransmission request of data frames, supporting reliable transmission of data connected between any two devices in the network, supporting frame loss detection of data packets and sending retransmission requests to the opposite end.
[0013] Furthermore, the method for reorganizing out of order and accurately retransmitting large data packets includes the following steps:
[0014] Step 1: Apply for the index information table address through QP and the first SN number, record the QP and the first SN and the last SN number of the transmitted data packet, and record whether the storage bit corresponding to the SN code table under the current QP is valid; it is used to determine the storage location of the current frame and update the information of the index information table so that the subsequent SN numbers of this frame can accurately store data according to the range of the first SN and the last SN.
[0015] Step 2: query the index information table through QP and non-first SN, store the frame at the mapped storage address, and record whether the storage bit corresponding to the SN code table under the current QP is valid; it is used to correctly store the frame according to the offset address obtained by adding the base address corresponding to the index information table to the difference between the first SN and the current SN, and update the storage indication table to facilitate frame loss detection and accurate retransmission request control.
[0016] Step 3: No index information table is found through the QP and SN number, indicating that the frame is stored in the chip cache. The address of the off-chip cache is mapped with the QP and SN number for storage. At the same time, the storage bit corresponding to the SN code table under the current QP is recorded as valid; it is used to combine the off-chip cache to ensure that the data frame will not be lost. When the on-chip space is full, it has to reduce performance guarantee and store it off-chip.
[0017] Furthermore, for the reception of large packets, there is a configurable on-chip cache that supports the number of out-of-order reassembled data packets and an off-chip cache that is uniquely mapped by QP and SN. The interval division of the on-chip cache is dynamically configured and adjusted according to the storage resources of the current hardware. When a new data frame is received, it is determined whether to allocate a new index information table or to query the existing index table information based on the received QP and whether the current frame is the first SN. The address of the index information table is the applied row number, and the row number is the division label of the on-chip storage interval, and the information is the QP, first SN and last SN of the currently applied frame.
[0018] If the current frame does not find the index information table and is not the first SN frame, it is directly stored outside the chip. When the upper-layer application moves the data frame, it queries the index table information based on QP and the first SN. If it is not found, the data frame is marked as stored outside the chip.
[0019] If the current frame is the first SN frame, the QP of the current frame, the first SN and the last SN calculated according to the size of the data packet are written to the address of the applied index information table, and the index information table is set to be valid, and the bit position indicating that the frame of the current SN has been received is set to be valid; the address of the SN code table managed by each QP is the QP of the current stored frame, and each bit in the table entry corresponds to an SN. When the bit position is valid, it indicates that a data frame is written to the cache; when the data is submitted to the upper layer, the bit corresponding to the moved data frame will be cleared.
[0020] If the current frame is not the first SN frame and a valid index information table is found based on the QP and SN, the frame will be accurately stored at the offset address calculated by adding the base address mapped by the index information table to the SN and the first SN recorded in the index information table. At the same time, the storage bit corresponding to the SN code table under the current QP is recorded to be valid. The on-chip storage address is uniquely determined based on the index information table and the offset address determined by the current SN and the first SN.
[0021] Furthermore, the method for out-of-order reorganization and accurate retransmission request of large packet data also includes:
[0022] Each time a data frame is received, the storage locations corresponding to the first SN and the last SN corresponding to the currently received frame are scanned;
[0023] Before the receiving counter times out, if the storage bits of the first SN and the last SN of the current frame are all valid, a time write event is initiated to the upper layer application to pass the collected data to the upper layer;
[0024] After the receiving counter times out, if the storage bits of the first SN and the last SN of the current frame are still not all received, a retransmission request will be made for the SN number with invalid bit position.
[0025] Furthermore, the implementation mechanism for accurate retransmission request is to update a storage indication table and an index information table for each stored data frame. The address of the storage indication table is QP. Each queue maintains a bit code table of SN numbers. When the SN number corresponding to the current QP is stored, it is set to valid, indicating that the data has been written into the cache; when the data packet is collected and transmitted to the upper layer, it is set to invalid, indicating that the data has been read; the address of the index information table is the identification number of the free storage space applied for the current frame, each identification number corresponds to a fixed storage space range, and the content of the index information table is the base address corresponding to the identification number currently applied, as well as the first SN and last SN number corresponding to the current frame. When a new data frame is received, the storage bit of the SN code table under the corresponding QP address is scanned according to the first SN and last SN stored in the index information table. If the bits within the first and last SN range are all valid, the timeout counter is cleared. Otherwise, after the reception counter times out, the storage bit of the SN code table under the corresponding QP address is scanned according to the first SN and last SN stored in the index information table, and an accurate retransmission request is made for the SN number with invalid storage bit identification.
[0026] Furthermore, the method for out-of-order reorganization and accurate retransmission request of large packet data also includes:
[0027] Each time a new data frame is received, the bit vectors of the storage unit idle indication are scanned to see if they are all valid. If it is found that the storage space corresponding to a bit is invalid, the row number and / or base address corresponding to the bit is returned as the storage base address of the current frame.
[0028] If all the indication bits of the current storage space mapping are found to be valid after scanning, indicating that the current storage space has been fully occupied, the current frame will be stored off-chip;
[0029] If the storage bits of the first SN and the last SN of the scanned frame are valid after the current frame is received, a write event is sent to the upper layer to move the collected data to the upper layer, and after the move is completed, the index table information and the storage indication table information mapped according to the QP and the first SN of the current frame are cleared; at the same time, the corresponding bit position of the storage unit idle indication bit vector is invalid, indicating that the current storage space has been released.
[0030] Furthermore, the updating and releasing of the on-chip storage space is based on the on-chip cache dividing the address interval according to the PMTU in the pre-configured network and the maximum SN number consumed by a data request packet; each divided storage interval corresponds to a storage bit, which is used to identify whether the current storage space is occupied; when a new data frame is stored, the update interval storage bit is valid, indicating that this interval is occupied, and after the migration is completed, the index table information mapped according to the QP and the first SN of the current frame and the storage indication table information are cleared; at the same time, the corresponding bit position of the storage unit idle indication bit vector is invalid, indicating that the current storage space has been released.
[0031] Another object of the present invention is to provide a large packet data reordering and precise retransmission request system using the large packet data reordering and precise retransmission request method, the large packet data reordering and precise retransmission request system comprising an on-chip and off-chip joint storage structure and a storage indication table and an index information table, and also comprising:
[0032] The index information table address application module is used to apply for the index information table address through the QP and the first SN number, record the QP and the first SN and the last SN number of the transmitted data packet, and record whether the storage bit corresponding to the SN code table under the current QP is valid;
[0033] An index information table query module is used to query the index information table through QP and non-first SN, store the frame at the mapped storage address, and record whether the storage bit corresponding to the SN code table under the current QP is valid;
[0034] The data frame storage module is used to store the frame after mapping the address of the off-chip cache using the QP and SN number and recording whether the storage bit corresponding to the SN code table under the current QP is valid.
[0035] Another object of the present invention is to provide a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and when the computer program is executed by the processor, the processor performs the following steps:
[0036] By utilizing the on-chip cache for storing data frames and the index information table for storing data storage addresses and information, when the on-chip storage is full, the data frames are stored in the off-chip cache; the on-chip and off-chip caches are used in conjunction with the index information table to implement frame loss detection and accurate retransmission requests for data frames, support reliable transmission of data connected between any two devices in the network, support frame loss detection of data packets and sending retransmission requests to the other end.
[0037] Another object of the present invention is to provide an information data processing terminal, which is used to implement the large packet data out-of-order reorganization and accurate retransmission request system.
[0038] In combination with the above technical solutions and the technical problems solved, please analyze the advantages and positive effects of the technical solutions to be protected by the present invention from the following aspects:
[0039] First, in view of the technical problems existing in the above-mentioned prior art and the difficulty of solving the problems, the technical solutions to be protected by the present invention and the results and data during the research and development process are closely combined to analyze in detail and deeply how the technical solutions of the present invention solve the technical problems, and some creative technical effects brought about after solving the problems. The specific description is as follows:
[0040] In the network, a large number of data packets often need to be transmitted. Due to the delay introduced by the data packets passing through different paths, nodes and different forwarding units in the network, the data packets arrive out of order. At this time, it is necessary to reorganize the out-of-order packets and control the retransmission request after detecting frame loss.
[0041] The present invention can support the disordered reorganization of thousands of connected large packets through the joint scheduling of on-chip and off-chip, accelerate the performance in the processing of big data, and reduce the network processing delay. Each connection has a unique queue identifier QP, and each queue maintains its own sequence number SN. When the data arrives, it can first detect whether the current on-chip storage module has any remaining space. If so, the data can be accurately indexed to the storage address according to QP and SN, and the storage identifier is updated. When updating the index information table, the SN range of the current transmission data will be determined according to the size of the current transmission data and the first SN. When receiving a new data frame, it is first determined whether it belongs to this range. If it belongs to this range, the data is accurately stored at the index address, otherwise a new index address is opened for storage, thereby supporting the same data packet to arrive in disorder, and different data packets can support outstanding. If there is no space on the chip, off-chip storage is performed. The off-chip storage is based on the storage address uniquely mapped by QP and SN, and accurate storage is also achieved. However, since off-chip storage and reading require more clock cycles, the latency performance will be reduced.
[0042] The present invention proposes a new frame loss detection and retransmission request mechanism. When a frame is lost, an accurate retransmission request is made, so as not to cause serious performance degradation. The implementation mechanism of frame loss detection is that each time a data frame is stored, a storage indication table and an index information table will be updated. The address of the storage indication table is QP. Each queue maintains a bit code table of SN numbers. When the SN number corresponding to the current QP is stored, it is set to valid, indicating that the data has been written into the cache. When the data packet is collected and transmitted to the upper layer, it is set to invalid, indicating that the data has been read. The address of the index information table is the identification number of the free storage space applied for the current frame. Each identification number corresponds to a fixed storage space range. The content of the index information table is the base address corresponding to the identification number currently applied, as well as the first SN and the last SN number corresponding to the current frame. Each time a new data frame is received, the storage bits of the SN code table under the corresponding QP address are scanned according to the first SN and the last SN stored in the index information table. If the bits within the range of the first and last SNs are valid, the timeout counter is cleared. Otherwise, after the receiving counter times out, the storage bits of the SN code table under the corresponding QP address are scanned according to the first SN and the last SN stored in the index information table, and an accurate retransmission request is made for the SN number whose storage bit mark is invalid, so as to avoid performance degradation caused by retransmission request of the entire data packet.
[0043] Second, considering the technical solution as a whole or from the perspective of the product, the technical effects and advantages of the technical solution to be protected by the present invention are described in detail as follows:
[0044] The present invention accelerates the performance of big data processing and reduces network processing delay through joint scheduling inside and outside the chip. The present invention proposes a new frame loss detection and retransmission mechanism, which can achieve accurate retransmission when frames are lost without causing serious performance degradation. The present invention improves the performance of large packet data by combining on-chip and off-chip caches to effectively solve the performance degradation caused by delays and other retransmission mechanisms.
[0045] The present invention mainly solves the existing hardware guarantee of large packet disorder reorganization and accurate retransmission request in reliable transmission. The architecture of this optimization method includes: an on-chip cache for storing data frames, an index information table for storing data storage addresses and information, and an off-chip cache for storing it when the on-chip storage is full. Among them, the on-chip and off-chip caches cooperate with the index information table to realize frame loss detection and accurate retransmission request of data frames, without the need to retransmit the entire lost data frame, thereby improving the network performance under retransmission.
[0046] Compared with the prior art, the advantages of the present invention further include:
[0047] (1) It supports the reordering of large packets and performs outstanding processing on different QPs, without having to wait for the current QP to be fully received before processing the next QP.
[0048] (2) Upper-level applications can dynamically divide storage intervals and perform address mapping based on the size of on-chip resources and enabled QPs.
[0049] (3) The ability to accurately send retransmission requests through the index information table can improve network performance.
[0050] Third, as auxiliary evidence of the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:
[0051] The expected benefits and commercial value of the technical solution of the present invention after transformation are as follows: The present invention is a key patent in the field of out-of-order retransmission of large packets of data and precise retransmission requests on reliable connections. Since the paths and nodes through which data pass in the network are different, hardware is usually required to provide out-of-order reorganization. Traditional hardware is difficult to support out-of-order reorganization of large packets of data or needs to resend the entire data to ensure data reliability. The present invention divides the cache unit into configurable partitions according to the PMTU in the current network through joint storage on and off the chip, and performs out-of-order reorganization of large packets and precise retransmission requests through the storage information of the index information table. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0053] Figure 1 It is a flow chart of a method for reorganizing out-of-order large packet data and accurately retransmitting a request provided by an embodiment of the present invention;
[0054] Figure 2 It is a schematic diagram of a method for reorganizing out-of-order large data packets and accurately requesting retransmission provided by an embodiment of the present invention;
[0055] Figure 3 It is a structural block diagram of a system for reorganizing out-of-order large-packet data and accurately retransmitting a request provided by an embodiment of the present invention;
[0056] Figure 4 Schematic diagram of the division of the on-chip storage area provided by an embodiment of the present invention;
[0057] Figure 5 is an RDMA index information application diagram provided by an embodiment of the present invention;
[0058] Figure 6 is an index information search graph of RDMA provided by an embodiment of the present invention;
[0059] Figure 7 It is a diagram of RDMA data migration and information clearing provided by an embodiment of the present invention;
[0060] Figure 8 It is an overall simulation diagram of multiple data streams provided by an embodiment of the present invention.
[0061] In the figure: 1. Index information table address application module; 2. Index information table query module; 3. Data frame storage module. DETAILED DESCRIPTION
[0062] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0063] In view of the problems existing in the prior art, the present invention provides a method, system and terminal for out-of-order reorganization and precise retransmission request of large packet data. The present invention is described in detail below with reference to the accompanying drawings.
[0064] 1. Explanatory Examples In order to enable those skilled in the art to fully understand how to implement the present invention, this section provides an illustrative example that expands and describes the technical solution of the claims.
[0065] Explanation of terms: PMTU: Maximum transmission unit of a link; QP: A connection pair between a sender and a receiver; SN: A sequence number corresponding to each queue.
[0066] like Figure 1 As shown, the method for reorganizing out-of-order large packet data and accurately retransmitting a request provided by the embodiment of the present invention includes the following steps:
[0067] S101, applying for the address of the index information table through the QP and the first SN number, recording the QP and the first SN and the last SN number of the transmitted data packet, and recording that the storage bit corresponding to the SN code table under the current QP is valid;
[0068] S102, query the index information table through the QP and the non-first SN, store the frame in the mapped storage address, and record the validity of the storage bit corresponding to the SN code table under the current QP;
[0069] S103, if no index information table is found through the QP and SN number, it indicates that the frame is stored in the slice cache, and the address of the off-chip cache is mapped out by using the QP and SN number for storage, and at the same time, it is recorded that the storage bit corresponding to the SN code table under the current QP is valid.
[0070] The schematic diagram of the method for reorganizing large data packets out of order and accurately retransmitting requests provided by the embodiment of the present invention is shown in Figure 2 .
[0071] like Figure 3 As shown, the out-of-order reorganization and accurate retransmission request system for large packet data provided by the embodiment of the present invention includes an on-chip and off-chip joint storage structure, a storage indication table and an index information table, and also includes:
[0072] The index information table address application module 1 is used to apply for the index information table address through the QP and the first SN number, record the QP and the first SN and the last SN number of the transmission data packet, and record whether the storage bit corresponding to the SN code table under the current QP is valid;
[0073] Index information table query module 2, used to query the index information table through QP and non-first SN, store the frame in the mapped storage address, and record the validity of the storage bit corresponding to the SN code table under the current QP;
[0074] The data frame storage module 3 is used to store the frame after mapping the address of the off-chip cache with the QP and SN number and recording whether the storage bit corresponding to the SN code table under the current QP is valid.
[0075] Example 1
[0076] The method for implementing the hardware-secured large packet data out-of-order reorganization and retransmission request provided in the embodiment of the present invention is a large packet out-of-order reorganization and precise retransmission request for data in the network based on FPGA or specially developed custom chips, supports reliable transmission of data connecting any two devices in the network, supports frame loss detection of data packets and sending retransmission requests to the other end.
[0077] The embodiment of the present invention provides a transmission system for hardware-guaranteed disorderly reorganization of large-packet data and accurate retransmission request, which includes an on-chip and off-chip joint storage structure, a storage indication table, and an index information table.
[0078] The method for implementing the out-of-order reorganization and retransmission request of large data packets provided by the embodiment of the present invention with hardware guarantee includes the following steps:
[0079] Apply for the index information table address through the QP and the first SN number, and record the QP, the first SN and the last SN number of the data packet transmitted this time, and record the validity of the storage bit corresponding to the SN code table under the current QP;
[0080] The index information table is queried through QP and non-first SN, the frame is stored in the mapped storage address, and the storage bit corresponding to the SN code table under the current QP is recorded as valid;
[0081] If the index information table is not found through the QP and SN number, it means that the frame is stored in the slice cache. It is necessary to use the QP and SN number to map the address of the off-chip cache and store it. At the same time, it is recorded that the storage bit corresponding to the SN code table under the current QP is valid.
[0082] For receiving large packets, there is a configurable on-chip buffer that supports the number of out-of-order reassembly packets and an off-chip buffer that is uniquely mapped by QP and SN. Figure 4 As shown in Figure 1, the interval division of the on-chip cache can be dynamically configured and adjusted according to the storage resources of the current hardware. When a new data frame is received, it will decide whether to allocate a new index information table or query the existing index table information based on the received QP and whether the current frame is the first SN. As shown in Table 1, the address of the index information table is the applied row number, that is, the division number of the on-chip storage interval, and its information is the QP, first SN, and last SN of the current applied frame.
[0083] Table 1 Format of index information table
[0084]
[0085] If the current frame does not find the index information table and is not the first SN frame, it will be directly stored outside the chip. When the upper-layer application moves this data frame, it will query the index table information based on QP and the first SN. If it is not found, it will be marked that the data frame is stored outside the chip. In this way, data will not be lost and joint scheduling inside and outside the chip can be guaranteed.
[0086] If the current frame is the first SN frame, the QP of the current frame, the first SN, and the last SN calculated according to the size of the data packet will be written to the address of the index information table applied for, and the index information table will be set to valid, and the bit position indicating that the frame of the current SN has been received will be valid. As shown in Table 2, the address of the SN code table managed by each QP is the QP of the current storage frame. Each bit in the table entry corresponds to an SN. When the bit position is valid, it indicates that a data frame is written into the cache. When the data is submitted to the upper layer, the bit corresponding to the moved data frame will be cleared.
[0087] Table 2 Storage instruction table format
[0088]
[0089] If the current frame is not the first SN frame, and a valid index information table is found based on the QP and SN, the frame will be stored in the base address mapped by the index information table plus the offset address calculated by the SN and the first SN recorded in the index information table for accurate storage, and the storage bit corresponding to the SN code table under the current QP is recorded to be valid. As shown in Table 3, the on-chip storage address can be uniquely determined based on the index information table and the offset address determined by the current SN and the first SN.
[0090] Table 3 Storage address mapping relationship
[0091]
[0092] Example 2
[0093] The method for implementing the hardware-secured large packet data out-of-order reorganization and retransmission request provided in the embodiment of the present invention is a large packet out-of-order reorganization and precise retransmission request for data in the network based on FPGA or specially developed custom chips, supports reliable transmission of data connecting any two devices in the network, supports frame loss detection of data packets and sending retransmission requests to the other end.
[0094] The method for implementing the out-of-order reorganization and retransmission request of large data packets provided by the embodiment of the present invention with hardware guarantee includes the following steps:
[0095] Each time a data frame is received, the storage locations corresponding to the first SN and the last SN corresponding to the currently received frame are scanned;
[0096] Before the receiving counter times out, if the storage bits of the first SN and the last SN of the current frame are all valid, a time write event is initiated to the upper layer application to pass the collected data to the upper layer;
[0097] After the receiving counter times out, if the storage bits of the first SN and the last SN of the current frame are still not all received, a retransmission request will be made for the SN number with invalid bit position.
[0098] The implementation mechanism of precise retransmission request is that each time a data frame is stored, a storage indication table and an index information table will be updated, and the table formats are shown in Table 1 and Table 2. The address of the storage indication table is QP. Each queue maintains a bit code table of SN numbers. When the SN number corresponding to the current QP is stored, it is set to valid, indicating that the data has been written into the cache. When the data packet is collected and passed to the upper layer, it is set to invalid, indicating that the data has been read. The address of the index information table is the identification number of the free storage space applied for the current frame. Each identification number corresponds to a fixed storage space range. The content of the index information table is the base address corresponding to the identification number currently applied, as well as the first SN and last SN number corresponding to the current frame. Each time a new data frame is received, the storage bits of the SN code table under the corresponding QP address are scanned according to the first SN and the last SN stored in the index information table. If the bits within the range of the first and last SNs are valid, the timeout counter is cleared. Otherwise, after the receiving counter times out, the storage bits of the SN code table under the corresponding QP address are scanned according to the first SN and the last SN stored in the index information table, and an accurate retransmission request is made for the SN number whose storage bit mark is invalid, so as to avoid performance degradation caused by retransmission request of the entire data packet.
[0099] Example 3
[0100] The method for implementing the hardware-secured large packet data out-of-order reorganization and retransmission request provided in the embodiment of the present invention is a large packet out-of-order reorganization and precise retransmission request for data in the network based on FPGA or specially developed custom chips, supports reliable transmission of data connecting any two devices in the network, supports frame loss detection of data packets and sending retransmission requests to the other end.
[0101] The hardware-guaranteed reorganization of large data packets and the implementation of retransmission requests to indicate the fullness or emptyness of the on-chip storage space provided by the embodiment of the present invention include the following steps:
[0102] Each time a new data frame is received, the bit vector indicating the free space of the storage unit is scanned to see if all are valid. If it is found that the storage space corresponding to a bit is invalid, the row number (base address) corresponding to this bit is returned as the storage base address of the current frame.
[0103] If all the indication bits of the current storage space mapping are found to be valid after scanning, it means that the current storage space has been fully occupied and the current frame needs to be stored off-chip;
[0104] If the storage bits of the first SN and the last SN of the scanned frame are valid after the current frame is received, a write event is sent to the upper layer to move the collected data to the upper layer. At the same time, after the move is completed, the index table information and the storage indication table information mapped according to the QP and the first SN of the current frame are cleared, and the corresponding bit position of the storage unit idle indication bit vector is invalid, indicating that the current storage space has been released.
[0105] The updating and releasing of the on-chip storage space is based on the on-chip cache, which divides the address interval according to the PMTU in the pre-configured network and the maximum SN number consumed by a data request packet. As shown in Table 4, each divided storage interval corresponds to a storage bit to indicate whether the current storage space is occupied. When a new data frame is stored, the update interval storage bit is valid, indicating that this interval is occupied. At the same time, after the migration is completed, the index table information and storage indication table information mapped according to the QP and the first SN of the current frame are cleared, and the corresponding bit position of the storage unit idle indication bit vector is invalid, indicating that the current storage space has been released.
[0106] Table 4 Storage Unit Idle Indication Vector
[0107]
[0108] 2. Application Examples: In order to prove the creativity and technical value of the technical solution of the present invention, this section provides application examples of the technical solution of the claims on specific products or related technologies.
[0109] A specific application example of the present invention is: it is applied in a high-performance RDMA network card. When establishing a connection, the RDMA network card exchanges the memory window and the binding QP of the local host and the remote host through a private protocol, and assigns each binding with its own independent SN number. When transmitting large packets, it can be specified that a maximum of 8 SN numbers are requested each time, and each SN number, that is, the PMTU is 8KB, so that the maximum supported large packet is 64KB. When the receiving end receives the large packet data, it can be stored according to the QP and SN number, and an accurate retransmission request can be made according to the maintained information table. At the same time, the joint storage inside and outside the chip can accelerate the performance when there is storage space inside the chip, and ensure that no frame is lost when there is no space.
[0110] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.
[0111] 3. Evidence of the effects of the embodiments. The embodiments of the present invention have achieved some positive effects during the development or use process, and indeed have great advantages over the prior art. The following content is described in conjunction with the data, charts, etc. of the test process.
[0112] The present invention can perform out-of-order reorganization when there is storage space on the chip and reduce the extra delay consumed by off-chip storage and reading through joint simulation on the chip and off-chip, and perform accurate data retransmission requests by maintaining the index information table and storage indication table. The following is the simulation result of the on-chip reorder reorganization unit based on RDMA, as shown in the figure. Figure 5 As shown, when the RDMA packet type indicator opcode is used to identify the first SN, the frame length carried by the data frame header is extracted (in this project, the frame length is 0, which represents 64KB), and the tail SN can be calculated as 7 through the frame length and the first SN (the first SN is 0 at this time, and each SN is a maximum of 8KB). At the same time, the buf_select indicating that the chip storage is full is pulled high, indicating that there is still space left in the chip, so the index information table is updated after the requested row number (in this simulation, the 0th bit indicates that the row number is 0), and an ACK signal is returned, indicating that this SN has been correctly received.
[0113] like Figure 6 As shown, when the RDMA packet type indicator opcode is identified as a non-first SN, the ram_row_adder base address (that is, the first SN, the requested row number) will be queried according to the QP and SN, and the ram_row_offset offset address will be calculated as 1 through the first SN and the current SN, and an ACK signal will be returned, indicating that this SN has been correctly received.
[0114] like Figure 7 As shown, when all the data are collected, the upper layer queries the frame storage location through the verify signal and returns the query result. When the upper layer application completes the data movement, the frame storage information is cleared through the clr signal to release the current on-chip cache.
[0115] like Figure 8 As shown, the overall process of RDMA data is to first apply for the index information table through the first SN, then search the index information table through the non-SN to store the data, and finally notify the upper layer to read after detecting that all data are collected and clear the corresponding information after the reading is completed.
[0116] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.
Claims
1. A method for reorganizing out-of-order large-packet data and accurately requesting retransmission, characterized in that: The method for out-of-order reorganization and accurate retransmission request of large packet data includes: By using the on-chip cache for storing data frames and the index information table for storing data storage addresses and information, when the on-chip storage is full, the data frames are stored in the off-chip cache; by using the cooperation of the on-chip and off-chip caches and the index information table, frame loss detection and accurate retransmission requests for data frames are realized, supporting reliable transmission of data between any two devices in the network to establish a connection, supporting frame loss detection of data packets and sending retransmission requests to the other end; The method for out-of-order reorganization and accurate retransmission request of large packet data comprises the following steps: Step 1: Apply for the index information table address through the QP and the first SN number, record the QP and the first SN and the last SN number of the transmitted data packet, and record whether the storage bit corresponding to the SN code table under the current QP is valid; Step 2: query the index information table through QP and non-first SN, store the frame at the mapped storage address, and record whether the storage bit corresponding to the SN code table under the current QP is valid; Step 3: If the index information table is not found through the QP and SN number, it indicates that the frame is stored in the slice cache. The address of the off-chip cache is mapped with the QP and SN number and then stored. At the same time, it is recorded that the storage bit corresponding to the SN code table under the current QP is valid. For the reception of large packets, there is a configurable on-chip cache that supports the number of out-of-order reassembled data packets and an off-chip cache that is uniquely mapped by QP and SN. The interval division of the on-chip cache is dynamically configured and adjusted according to the storage resources of the current hardware; when a new data frame is received, it is determined whether to allocate a new index information table or query the existing index table information according to the received QP and whether the current frame is the first SN; the address of the index information table is the applied row number, the row number is the division number of the on-chip storage interval, and the information is the QP, first SN and last SN of the current applied frame; If the current frame does not find the index information table and is not the first SN frame, it is directly stored outside the chip. When the upper-layer application moves the data frame, it queries the index table information based on the QP and the first SN. If it is not found, it marks the data frame as being stored outside the chip. If the current frame is the first SN frame, the QP of the current frame, the first SN and the last SN calculated according to the size of the data packet are written into the address of the index information table applied for, and the index information table is set to be valid, and the bit position indicating that the frame of the current SN has been received is set to be valid; the address of the SN code table managed by each QP is the QP of the current stored frame, and each bit in the table entry corresponds to an SN. When the bit position is valid, it indicates that a data frame is written into the cache; when the data is submitted to the upper layer, the bit corresponding to the moved data frame will be cleared; If the current frame is not the first SN frame and a valid index information table is found based on the QP and SN, the frame will be accurately stored at the offset address calculated by adding the base address mapped by the index information table to the SN and the first SN recorded in the index information table. At the same time, the storage bit corresponding to the SN code table under the current QP is recorded to be valid. The on-chip storage address is uniquely determined based on the index information table and the offset address determined by the current SN and the first SN.
2. The method for reorganizing out-of-order large data packets and accurately requesting retransmission as claimed in claim 1, characterized in that: The method for out-of-order reorganization and accurate retransmission request of large packet data also includes: Each time a data frame is received, the storage locations corresponding to the first SN and the last SN corresponding to the currently received frame are scanned; Before the receiving counter times out, if the storage bits of the first SN and the last SN of the current frame are all valid, a time write event is initiated to the upper layer application to pass the collected data to the upper layer; After the receiving counter times out, if the storage bits of the first SN and the last SN of the current frame are still not all received, a retransmission request will be made for the SN number with invalid bit position.
3. The method for reorganizing out-of-order large data packets and requesting accurate retransmission as claimed in claim 2, characterized in that: The implementation mechanism for accurate retransmission request is to update a storage indication table and index information table every time a data frame is stored; The address of the storage indication table is QP. Each queue maintains a bit code table of SN numbers. When the SN number corresponding to the current QP is stored, it is set to valid, indicating that the data has been written into the cache; When the data packet is collected and passed to the upper layer, it is set to invalid, indicating that the data has been read; the address of the index information table is the identification number of the free storage space applied for the current frame, each identification number corresponds to a fixed storage space range, and the content of the index information table is the base address corresponding to the identification number currently applied, as well as the first SN and last SN corresponding to the current frame; each time a new data frame is received, the storage bit of the SN code table under the corresponding QP address is scanned according to the first SN and last SN stored in the index information table. If the bits within the range of the first and last SNs are valid, the timeout counter is cleared. Otherwise, after the receiving counter times out, the storage bit of the SN code table under the corresponding QP address is scanned according to the first SN and last SN stored in the index information table, and an accurate retransmission request is made for the SN number with an invalid storage bit identification.
4. The method for reorganizing out-of-order large data packets and requesting accurate retransmission as claimed in claim 1, characterized in that: The method for out-of-order reorganization and accurate retransmission request of large packet data also includes: Each time a new data frame is received, the bit vectors of the storage unit idle indication are scanned to see if they are all valid. If it is found that the storage space corresponding to a bit is invalid, the row number and / or base address corresponding to the bit is returned as the storage base address of the current frame. If all the indication bits of the current storage space mapping are found to be valid after scanning, indicating that the current storage space has been fully occupied, the current frame will be stored off-chip; If the storage bits of the first SN and the last SN of the scanned frame are valid after the current frame is received, a write event is sent to the upper layer to move the collected data to the upper layer, and after the move is completed, the index table information and the storage indication table information mapped according to the QP and the first SN of the current frame are cleared; at the same time, the corresponding bit position of the storage unit idle indication bit vector is invalid, indicating that the current storage space has been released.
5. The method for reorganizing out of order and accurately retransmitting large data packets as claimed in claim 4, characterized in that: The update and release of the on-chip storage space is based on the on-chip cache, which divides the address interval according to the pre-configured PMTU in the network and the maximum SN number consumed by a data request packet; each divided storage interval corresponds to a storage bit, which is used to identify whether the current storage space is occupied; When a new data frame is stored, the interval storage bit is updated to be valid, indicating that this interval has been occupied, and after the move is completed, the index table information mapped according to the QP and first SN of the current frame and the storage indication table information are cleared; at the same time, the corresponding bit position of the storage unit idle indication bit vector is invalid, indicating that the current storage space has been released.
6. A system for reordering and accurately requesting retransmission of large packets of data using the method for reordering and accurately requesting retransmission of large packets of data as claimed in any one of claims 1 to 5, characterized in that: The large packet data out-of-order reorganization and accurate retransmission request system includes an on-chip and off-chip joint storage structure and a storage indication table and an index information table, and also includes: The index information table address application module is used to apply for the index information table address through the QP and the first SN number, record the QP and the first SN and the last SN number of the transmitted data packet, and record whether the storage bit corresponding to the SN code table under the current QP is valid; An index information table query module is used to query the index information table through QP and non-first SN, store the frame at the mapped storage address, and record whether the storage bit corresponding to the SN code table under the current QP is valid; The data frame storage module is used to store the frame after mapping the address of the off-chip cache using the QP and SN number and recording whether the storage bit corresponding to the SN code table under the current QP is valid.
7. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the large packet data out-of-order reassembly and accurate retransmission request method according to any one of claims 1 to 5, including the following steps: Using the on-chip cache for storing data frames and the index information table for storing data storage addresses and information, when the on-chip storage is full, the data frames are stored in the off-chip cache; The chip's internal and external caches and index information tables are used in conjunction to implement frame loss detection and accurate retransmission requests for data frames. It supports reliable transmission of data between any two devices in the network, supports frame loss detection of data packets, and sends retransmission requests to the other end.
8. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the large packet data out-of-order reorganization and accurate retransmission request system as described in claim 6.