FPGA comparator resource allocation method and system for network telemetry rearrangement

Abstract

The application belongs to the field of in-band network telemetry, and provides an FPGA comparator resource allocation method and system for network telemetry rearrangement. The current out-of-order measurement indexes are counted based on the out-of-order telemetry data, including the out-of-order rate in the online sorting window, the out-of-order rate between the online sorting windows, and the weighted out-of-order distance. The historical out-of-order measurement index weight is adaptively adjusted according to the current FPGA reordering system performance deviation ratio to obtain the current out-of-order measurement index weight. The comparator resource allocation ratio of online sorting and offline sorting is determined based on the current out-of-order measurement index weight. The comparator resource optimal allocation ratio is obtained by adjusting the comparator resource allocation ratio according to the current FPGA reordering system performance deviation. The application can significantly improve the FPGA comparator resource utilization rate and the telemetry data reordering efficiency in the telemetry data out-of-order reordering process of the telemetry server in the in-band network telemetry system.

Description

Technical Field

[0001] This invention belongs to the field of in-band network telemetry technology, specifically relating to an FPGA comparator resource allocation method and system for network telemetry rearrangement. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In-band network telemetry (INT) is an emerging network measurement technology that embeds telemetry commands and information into service traffic to achieve end-to-end, hop-by-hop network status acquisition. In real-world large-scale network environments, telemetry tasks typically employ multi-path, multi-task orchestration, with telemetry data packets being transmitted concurrently through multiple paths. Due to differences in latency, congestion, and other factors along each path, even telemetry data packets from the same source flow may have different generation orders than their arrival order at the telemetry server, leading to severe out-of-order data. Out-of-order data not only affects the accuracy of timing analysis of network status by upper-layer applications but also causes data loss and statistical distortion. Network management applications based on in-band network telemetry can be divided into online and offline applications. Online applications (such as traffic engineering and fault detection) have extremely high requirements for the freshness of telemetry data, while offline applications (such as historical data analysis and gray fault analysis) have relatively lower requirements for freshness. However, regardless of the application type, out-of-order telemetry data will adversely affect its functionality and performance. Furthermore, corresponding to the types of telemetry applications, out-of-order reordering in in-band network telemetry is a reordering problem that combines online and offline sorting, and there is currently no effective solution.

[0004] Field-Programmable Gate Arrays (FPGAs) have been widely used in high-performance data processing scenarios due to their highly parallel and customizable hardware characteristics. Utilizing FPGAs for out-of-order reordering of telemetry data can fully leverage their parallel processing capabilities and low latency advantages. However, most existing methods only deploy online or offline sorting functions independently, lacking dedicated designs for the multi-stream, multi-task characteristics of in-band network telemetry, making it difficult to simultaneously meet the demands of real-time performance and large-scale data processing. The comparator unit within the FPGA is a key hardware unit for achieving efficient sorting of out-of-order data, and is the key reason why FPGAs significantly outperform general-purpose CPUs / GPUs in out-of-order reordering tasks in in-band network telemetry scenarios. The number and allocation of comparators directly affect the parallelism and processing efficiency of the FPGA. How to efficiently reorder out-of-order reported data from in-band network telemetry, meeting the ordered data requirements of upper-layer telemetry applications, within the limited FPGA comparator resources and addressing the collaborative organization needs of online and offline sorting in in-band network telemetry scenarios, is a crucial problem that current in-band network telemetry systems urgently need to overcome. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes an FPGA comparator resource allocation method and system for network telemetry reordering. This invention presents a comparator resource allocation method for simultaneously deploying online and offline sorting on a single FPGA accelerator card. Through a time reuse mechanism and parameter adaptive strategy, it achieves optimal allocation of comparator resources between online and offline reordering in multi-stream parallel processing. This satisfies the data out-of-order sorting requirements of high-speed real-time telemetry applications and large-scale offline telemetry applications, significantly improving FPGA accelerator card resource utilization and network telemetry data processing performance.

[0006] According to some embodiments, the first aspect of the present invention provides an FPGA comparator resource allocation method for network telemetry reordering, employing the following technical solution:

[0007] FPGA comparator resource allocation methods for network telemetry reordering include:

[0008] The out-of-order telemetry data in the FPGA reordering system is first sorted online, and then sorted offline to obtain completely ordered telemetry data.

[0009] The optimal allocation ratio of comparator resources for online and offline sorting in the FPGA reordering system is determined using current out-of-order telemetry data. Specifically:

[0010] Based on the current out-of-order telemetry data, the current out-of-order metrics are statistically analyzed, including the out-of-order rate within the online sorting window, the out-of-order rate between online sorting windows, and the weighted out-of-order distance.

[0011] The weights of historical out-of-order metrics are adaptively adjusted based on the current out-of-order metrics and the current performance deviation ratio of the FPGA reordering system to obtain the current out-of-order metrics weights.

[0012] Based on the current out-of-order metric weights and the current out-of-order metric, determine the comparator resource allocation ratio for online sorting and offline sorting;

[0013] Adjust the comparator resource allocation ratio according to the current performance deviation of the FPGA reordering system to obtain the optimal comparator resource allocation ratio for online sorting and offline sorting.

[0014] Furthermore, the out-of-order rate within the online sorting window refers to the ratio of the total number of out-of-order telemetry data packets to the total number of telemetry data packets within an online sorting observation window;

[0015] The out-of-order rate between online sorting windows refers to the average ratio of the number of out-of-order data packets across two adjacent online sorting windows to the total number of telemetry data packets between the two windows;

[0016] The weighted out-of-order distance refers to the distance offset between each out-of-order telemetry data packet and its correctly ordered position.

[0017] Furthermore, the step of adaptively adjusting the weights of historical out-of-order metrics based on the current out-of-order metric and the current performance deviation ratio of the FPGA reordering system to obtain the current out-of-order metric weight is as follows:

[0018] If the proportion of online sorting delay deviation in the current FPGA reordering system performance deviation ratio is greater than the delay deviation threshold, then the weight adjustment amount is updated according to the online sorting delay deviation.

[0019] If the offline sorting throughput deviation ratio in the current FPGA reordering system performance deviation ratio is greater than the throughput deviation threshold, then the weight adjustment amount is updated according to the offline sorting throughput deviation.

[0020] Based on the updated weight adjustment, the weights of the historical out-of-order metrics are smoothed, normalized, and weighted to obtain the current out-of-order metrics weights.

[0021] Furthermore, based on the current out-of-order metric weights and the current out-of-order metric, the comparator resource allocation ratio for online sorting and offline sorting is determined, specifically as follows:

[0022] Calculate the comparator resource demand intensity for online sorting and offline sorting based on the current out-of-order metric weights and the current out-of-order metric.

[0023] Calculate the comparator resource allocation ratio for online sorting and the comparator resource allocation ratio for offline sorting based on the comparator resource demand intensity for online sorting and offline sorting.

[0024] Furthermore, the resource demand intensity of the comparator for the online sorting is calculated as follows:

[0025] ;

[0026] The resource requirement intensity of the comparator for offline sorting is calculated as follows:

[0027] ;

[0028] in, It is the out-of-order rate within the online sorting window. It is the rate of disorder between online sorting windows. The normalized weighted out-of-order distance It is a weighted out-of-order distance. These are the weighting coefficients for the current disorder measurement indicators.

[0029] Furthermore, the comparator resource allocation ratio is adjusted based on the current performance deviation of the FPGA reordering system to obtain the optimal comparator resource allocation ratio for online and offline sorting, specifically as follows:

[0030] Based on the current performance deviation of the FPGA reordering system, a proportional control method is used to calculate the delay adjustment output and throughput adjustment output.

[0031] The actual resource allocation adjustment for online sorting and the actual resource allocation adjustment for offline sorting are calculated based on the delay adjustment output and the throughput adjustment output.

[0032] The comparator resource allocation ratio is updated based on the actual resource allocation adjustments for online and offline sorting to obtain the optimal comparator resource allocation ratio for both online and offline sorting.

[0033] According to some embodiments, the second aspect of the present invention provides an FPGA comparator resource allocation system for network telemetry reordering, employing the following technical solution:

[0034] An FPGA comparator resource allocation system for network telemetry reordering includes:

[0035] The out-of-order sorting module is configured to first sort the current out-of-order telemetry data in the FPGA reordering system online, and then sort it offline to obtain completely ordered telemetry data.

[0036] The resource allocation module is configured to determine the optimal allocation ratio of comparator resources for online and offline sorting in the FPGA reordering system using the current out-of-order telemetry data. Specifically:

[0037] Based on the current out-of-order telemetry data, the current out-of-order metrics are statistically analyzed, including the out-of-order rate within the online sorting window, the out-of-order rate between online sorting windows, and the weighted out-of-order distance.

[0038] The weights of historical out-of-order metrics are adaptively adjusted based on the current out-of-order metrics and the current performance deviation ratio of the FPGA reordering system to obtain the current out-of-order metrics weights.

[0039] Based on the current out-of-order metric weights and the current out-of-order metric, determine the comparator resource allocation ratio for online sorting and offline sorting;

[0040] Adjust the comparator resource allocation ratio according to the current performance deviation of the FPGA reordering system to obtain the optimal comparator resource allocation ratio for online sorting and offline sorting.

[0041] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium.

[0042] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the FPGA comparator resource allocation method for network telemetry rearrangement as described in the first embodiment above.

[0043] According to some embodiments, a fourth aspect of the present invention provides a computer device.

[0044] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the FPGA comparator resource allocation method for network telemetry rearrangement as described in the first embodiment above.

[0045] According to some embodiments, a fifth aspect of the present invention provides a computer program product or computer program.

[0046] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium, wherein a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in the FPGA comparator resource allocation method for network telemetry rearrangement as described in the first embodiment above.

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0048] This invention proposes an FPGA comparator resource allocation method for network telemetry reordering. Addressing the out-of-order telemetry data problem arising in multi-path, multi-task environments of in-band network telemetry, a resource allocation model based on multi-dimensional out-of-order metrics is designed. Adaptive weight adjustment algorithms, resource allocation control algorithms, and proportional fine-tuning control algorithms are proposed. These algorithms dynamically adjust the out-of-order metric weights based on FPGA reordering system performance feedback, allocate comparator resources according to the current weights and out-of-order metrics, and fine-tune the resource allocation results based on the deviation between actual and target performance. Furthermore, this invention proposes a time multiplexing method for multi-stream online sorting modules, further improving the utilization rate of FPGA comparator resources. This invention can significantly improve the FPGA comparator resource utilization and telemetry data reordering efficiency during the telemetry server's implementation of out-of-order telemetry data reordering in in-band network telemetry systems, meeting the real-time ordered data requirements of online telemetry applications and the large-scale ordered data requirements of offline telemetry applications. Attached Figure Description

[0049] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0050] Figure 1 This is a flowchart of an FPGA comparator resource allocation method for network telemetry rearrangement in an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of an application scenario in an embodiment of the present invention;

[0052] Figure 3 This is a flowchart of the data processing for online and offline sorting in an embodiment of the present invention;

[0053] Figure 4 This is a hardware system architecture diagram of the method implemented in the embodiments of the present invention;

[0054] Figure 5 This is a schematic diagram illustrating a single online reordering method in an embodiment of the present invention;

[0055] Figure 6 This is a schematic diagram of the online reordering time reuse method in an embodiment of the present invention;

[0056] Figure 7 This is a schematic diagram of an example method for offline reordering in an embodiment of the present invention;

[0057] Figure 8 This is a schematic diagram illustrating the effects of online and offline sorting in an embodiment of the present invention;

[0058] Figure 9This is a flowchart illustrating the process of determining the optimal allocation of comparator resources for online and offline sorting using current out-of-order telemetry data in an embodiment of the present invention.

[0059] Figure 10 This is a schematic diagram of an FPGA comparator resource allocation system for network telemetry rearrangement in an embodiment of the present invention. Detailed Implementation

[0060] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0061] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0062] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0063] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0064] Example 1

[0065] like Figure 1 As shown, this embodiment provides an FPGA comparator resource allocation method for network telemetry rearrangement. This embodiment uses the application of this method to a server as an example for illustration. It is understood that this method can also be applied to terminals, and can also be applied to systems including terminals, servers, and other components, and can be implemented through interaction between the terminal and the server. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. It can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communication, middleware services, domain name services, CDN security services, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein. In this embodiment, the method includes the following steps:

[0066] Step S1: First, perform online sorting on the current out-of-order telemetry data in the FPGA reordering system, and then perform offline sorting to obtain completely ordered telemetry data;

[0067] Step S2: Determine the optimal allocation ratio of comparator resources for online and offline sorting in the FPGA reordering system using the current out-of-order telemetry data, specifically:

[0068] Step S2.1: Based on the current out-of-order telemetry data, statistically analyze the current out-of-order metrics, including the out-of-order rate within the online sorting window, the out-of-order rate between online sorting windows, and the weighted out-of-order distance;

[0069] Step S2.2: Adaptively adjust the weights of historical out-of-order metrics based on the current out-of-order metrics and the current performance deviation ratio of the FPGA reordering system to obtain the weights of the current out-of-order metrics;

[0070] Step S2.3: Based on the current out-of-order metric weights and the current out-of-order metric, determine the comparator resource allocation ratio for online sorting and offline sorting;

[0071] Step S2.4: Adjust the comparator resource allocation ratio according to the current performance deviation of the FPGA reordering system to obtain the optimal comparator resource allocation ratio for online sorting and offline sorting.

[0072] Regarding step S1, such as Figure 2 The diagram illustrates a typical application scenario of this embodiment. In an in-band network telemetry multipath transmission scenario, terminal 2 and terminal 3 continuously send service data packets, embedding INT telemetry instructions within the data packets. Switching devices along the route write telemetry metadata indicators into the specified telemetry instruction fields. The two forwarding paths are topologically independent but intersect in the middle, forming a multipath concurrent environment. The delay of each hop link is as follows... Figure 1As shown, the link delay between INT node 2 and INT node 4 is 4ms, which is the bottleneck link causing out-of-order delivery in the network topology. Telemetry path 1 (blue route) passes through INT node 1, node 2, node 4, and node 5 in sequence, finally reaching the telemetry server on the right; telemetry path 2 (green route) passes through INT node 1, node 2, node 4, and node 5 in sequence, finally reaching the telemetry server on the right. Because telemetry data packet 2 generated by terminal 2 arrives at INT node 3 first via telemetry path 2, then merges with telemetry path 1 at INT node 4, and then is sent to the telemetry server via INT node 5. Due to the significant difference in the overall propagation delay between the two telemetry paths (the blue path has a longer total delay and includes a 4ms high-latency link segment), telemetry data packet 1 is very likely to arrive at the server later than telemetry data packet 2, resulting in an arrival order inconsistent with the original sending order, i.e., a typical out-of-order phenomenon, that is, the measurement timestamps of the two telemetry paths at INT node 1 cannot maintain consistency with the actual timestamps of arrival at the telemetry server.

[0073] The data processing flow for online and offline sorting in this embodiment is as follows: Figure 3 As shown, the top square represents the sequential number of the telemetry data packets when they arrive at the telemetry server. Among them, telemetry data packets numbered 3, 8, 20, and 26 are out of order, and the out-of-order distances are 2, 1, 1, and 3, respectively.

[0074] The FPGA accelerator card deploys two types of sorting function modules: online sorting and offline sorting. That is, several online sorting modules and offline sorting modules are deployed on the FPGA accelerator card. The online sorting module performs online sorting on the real-time received in-band network telemetry reports (out-of-order telemetry data packets) and stores some of the ordered sorting results in the host-side database. The offline sorting module reads some of the ordered telemetry data from the database, performs offline sorting, and stores all the ordered sorting results back to the host-side database.

[0075] The online sorting module performs high-speed insertion sort within a small sliding window, arranging several out-of-order telemetry data packets received in real time by the FPGA acceleration card into a partially ordered result, and writing it into the time-series database for low-latency reading by real-time telemetry applications.

[0076] The offline sorting module uses multi-level FIFO caching and merge logic to sort all telemetry results in the time series database, and then writes the globally ordered results back to the database.

[0077] like Figure 4As shown, the hardware system architecture implemented by the method in this embodiment, namely the FPGA reordering system, consists of a host side and an FPGA accelerator card side. The host side runs a host program, a database program, and a telemetry application. The host-side program schedules the reordering module and manages the data transmission between the CPU and the FPGA. The telemetry application uses telemetry data to perform different related network management functions.

[0078] In the online sorting section, the packet classifier in the FPGA segments the in-band network telemetry data from different network devices in multiple telemetry paths and classifies the telemetry data into the corresponding BRAM (Block Random Access Memory) according to the device ID.

[0079] The FPGA executes multiple online reordering modules in parallel to reorder multiple telemetry data streams received by the host-side in-band network telemetry server. After reordering, the telemetry data is written to a BRAM temporary buffer and returned to the host memory, where it is finally used by the host application to perform various related online network management functions.

[0080] In the offline sorting section, the host writes the partially ordered telemetry data, which has been sorted online, into the BRAM in the FPGA. The FPGA's offline reordering module reorders the telemetry data in the BRAM. After reordering, the telemetry data is written back to the BRAM and returned to the host memory. Finally, the host application uses the telemetry data to perform relevant offline network management functions.

[0081] like Figure 5 As shown, for a single online reordering method example and the required modules, Figure 6 Taking the first unit as an example, the insertion sorting process can be divided into determining the insertion position and moving elements. When a telemetry packet in the network arrives at packet 1 of the reordering module, the timestamp in the telemetry packet is compared with the timestamps of all packets in the reordering first unit simultaneously. When the timestamp of the new data packet is later than all existing data packets in the window, the FPGA reordering system inserts it at the end of the queue; when the timestamp is earlier than all existing data packets, it is inserted at the beginning of the queue; if the timestamp of the new data packet is between some existing data packets, the FPGA reordering system determines that it should be inserted before the first existing data whose timestamp is later than the data packet. The above judgment is completed in a single clock cycle through parallel comparison logic. Immediately afterwards, the FPGA reordering system completes the right shift of the corresponding data and the insertion of new data in the next cycle, thereby realizing that the entire sorting process is completed in two cycles.

[0082] Furthermore, this embodiment also provides a time-division multiplexing method for the online sorting module of multi-telemetry flows, such as... Figure 6As shown, online sorting modules corresponding to different telemetry flows are allowed to reuse the same set of FPGA comparator resources. Comparator switching is achieved through a dual-ended multiplexing module. Each sorting module is configured with an access interface and latch control logic to ensure that no access conflicts occur during switching. The scheduling period can be set to a fixed period or a fixed window. When two reordering modules time-multiplex a set of comparators, the time is divided into several time slices. In each time slice, only one reordering module uses the comparator resources. In the first time slice, reordering module 1 uses the comparator resources to perform insertion sort operations. In the second time slice, reordering module 2 uses the same comparator resources to perform insertion sort operations. The two reordering modules alternately use the comparator resources, thereby achieving time multiplexing. Through precise synchronization and scheduling mechanisms, it can be ensured that the two reordering modules can safely use the comparator resources in different time slices, thereby achieving the reuse of comparator resources among online sorting modules and improving the utilization rate of comparator resources.

[0083] like Figure 7 As shown, for the offline reordering method example and required modules, the FPGA accelerator card is equipped with the necessary hardware resources, including multi-level FIFO buffers, merge sorting logic (FLiMS), inputs, and memory. The host program configures the parameters of the offline sorting module to ensure the correctness of data flow and resource allocation. After the initial sorting is completed, data blocks are defined as 32 telemetry data points per block. The merge sorting input interface transfers each data block sequentially to a FIFO, while simultaneously transferring eight data blocks in parallel to eight independent FIFOs. Data blocks are read from memory into the FPGA's on-chip buffer FIFOs via DMA (Direct Memory Access) operations. Multi-level FIFO buffers are used to temporarily store data blocks transferred from memory, ensuring smooth data flow between different processing stages. Data blocks pass through each level of FIFO sequentially, ready to enter the merge sorting logic. The FLiMS module integrates multiple comparator units for merge operations; all instances in FLiMS perform ordered merging of multiple data streams in a hardware pipeline manner. FLiMS merges data blocks from two sorted blocks to generate a larger ordered data block. Multiple FLiMS instances sequentially undergo multi-level merging, ultimately combining all data blocks into a globally ordered dataset. After merging across multiple FLiMS instances, the resulting globally ordered dataset is written to the final database. This globally ordered data can be stored in memory or transferred back to the host's persistent storage via the host interface for real-time analysis or offline processing by upper-layer applications. A precise time synchronization mechanism ensures that each FLiMS instance safely uses shared comparator resources within different time slices, avoiding resource conflicts and data contention. The running status of FLiMS instances is dynamically adjusted based on data traffic and processing demands, optimizing resource utilization and processing efficiency.

[0084] Comparator switching is implemented through a dual-ended multiplexing module. Each sorting module is configured with an access interface and latch control logic to ensure that no access conflicts occur during switching. The scheduling period can be set to a fixed period or a fixed window.

[0085] like Figure 8 As shown, the entire sorting process is explained from a data perspective. After receiving out-of-order telemetry data packets, the in-band network telemetry server on the host side processes the data through the online sorting module to obtain partially ordered telemetry data, and then processes it through the offline sorting module to obtain fully ordered telemetry data. It should be noted that the online sorting module cannot handle out-of-order cases across sorting windows, so the telemetry data after online sorting is only partially ordered.

[0086] Regarding step S2, the FPGA reordering system calculates the degree of out-of-order nature of the out-of-order telemetry data packets in real time based on their timestamps, which guides the comparator resource allocation strategy. The specific method is as follows:

[0087] The FPGA reordering system introduces three out-of-order metrics, including the out-of-order rate within the online sorting window. Online sorting window disorder rate and weighted out-of-order distance The aforementioned metrics are calculated in real-time on the pipeline using counters and comparison logic within the FPGA. For example, It is obtained by recording the ratio of the number of out-of-order data packets to the total number of data packets.

[0088] Based on the current out-of-order metrics, the controller dynamically adjusts the comparator resource allocation strategy.

[0089] like and If the value remains below the set threshold, it indicates that the current network is basically ordered. The FPGA reordering system will allocate more comparator resources to the online sorting module to reduce sorting latency.

[0090] Conversely, if If the value exceeds the threshold, the FPGA reordering system will switch some comparators to the offline sorting module to handle large-scale out-of-order issues.

[0091] Step S2.1: Based on the current out-of-order telemetry data, statistically analyze the current out-of-order metrics, including the out-of-order rate within the online sorting window, the out-of-order rate between online sorting windows, and the weighted out-of-order distance;

[0092] This example proposes a multi-dimensional out-of-order measurement index system for in-band network telemetry, which comprehensively reflects the out-of-order characteristics of telemetry data, specifically including:

[0093] Online sorting window out-of-order rate ( (This refers to the ratio of the total number of out-of-order packets to the total number of packets within an online sorting observation window.)

[0094] (1);

[0095] (2);

[0096] in, It is an indicator function, when The value is 1 if the condition is true, and 0 otherwise. This represents the total number of telemetry data packets received within the online sorting observation window. The number of telemetry data packets in an out-of-order state within the observation window; For the first in the order of arrival The first telemetry data packet, also the first The position number of each telemetry data packet in the actual arrival sequence; For the first The position number that a telemetry data packet should occupy in an ideal ordered sequence.

[0097] Online sorting window disorder rate ( The average ratio of the number of out-of-order telemetry packets across two adjacent online sorting windows to the total number of telemetry packets in both windows.

[0098] (3);

[0099] in, Indicates the ideal position in the window But it actually landed in the window The number of telemetry data packets, and The first and the The total number of telemetry data packets within each window. The total number of windows for online sorting.

[0100] Weighted out-of-order distance ( ): The distance offset between each out-of-order packet and its correctly sorted position, i.e.

[0101] (4);

[0102] Measuring the degree of short-term disorder. Measuring the degree of global disorder. The offset distance is measured, and the above metrics are updated in real time on the FPGA data plane via a hardware pipeline.

[0103] Based on the aforementioned out-of-order metrics, this embodiment establishes an FPGA comparator resource allocation model, which is implemented through real-time monitoring. , and Three metrics are used to dynamically adjust the comparator resource allocation ratio for online and offline sorting.

[0104] First, the relevant FPGA reordering system model is represented as follows, defining the total FPGA comparator resources as... The number of online sorting comparators allocated is The number of offline sorting comparators allocated is ,in, , The utility of the FPGA reordering system is defined as follows:

[0105] (5);

[0106] The utility component function is defined as:

[0107] (6);

[0108] (7);

[0109] The constraints are:

[0110] (8);

[0111] (9);

[0112] in, For online sorting, a delay utility function can be used to delay... The evaluation metric is defined as follows: the smaller the delay, the higher the utility; its value range is... ; This is the offline sorting throughput utility function, which can handle throughput. The evaluation metric is based on throughput, where higher throughput equates to higher utility; its value range is [range missing]. ; and These are weighting coefficients used to manually adjust the relative utility of online and offline sorting. and It represents the target online sorting processing latency and target offline sorting throughput expected by the FPGA reordering system.

[0113] Based on the above model, this embodiment proposes an adaptive weight adjustment algorithm, a resource allocation control algorithm, and a proportional fine-tuning control algorithm based on performance feedback. Among them, the adaptive weight adjustment algorithm updates the out-of-order index by monitoring the performance deviation of the FPGA reordering system. , , The weights in resource allocation decisions enable the resource allocation strategy to adaptively adapt to changes in network state and ensure the stability of the FPGA reordering system. The resource allocation control algorithm is the main control loop of the entire comparator resource allocation process; its function is to allocate resources using the weights obtained from the adaptive weight adjustment algorithm. The proportional fine-tuning control algorithm fine-tunes the deviation based on the resource allocation control algorithm to prevent performance drift and avoid oscillations in the comparator resource allocation process. The specific process is as follows:

[0114] Step S2.2: Adaptively adjust the weights of historical out-of-order metrics based on the current out-of-order metric and the current performance deviation ratio of the FPGA reordering system to obtain the current out-of-order metric weights, specifically as follows:

[0115] 1) Based on the online sorting delay deviation ratio Offline sorting throughput deviation ratio Calculate the performance deviation ratio of the FPGA reordering system;

[0116] 2) Calculate the weight adjustment amount, specifically:

[0117] like Then update the weight adjustment amount. ;

[0118] like Then update the weight adjustment amount. ;

[0119] otherwise .

[0120] in, It is the delay deviation threshold. It is the throughput deviation threshold; This refers to the weight adjustment amount of the disorder rate indicator within the online sorting window. It is the normalized weighted out-of-order distance indicator weight adjustment amount. This is the weight adjustment amount for the online sorting window disorder rate indicator; It is an online sorting delay bias pair Adjustment coefficient, It is an online sorting delay bias pair Adjustment coefficient, It is an offline sorting throughput bias pair Adjustment coefficient.

[0121] 3) Adopt The method smooths out the weight adjustments.

[0122] in, As a smoothing factor, to prevent weights from changing too quickly, It is the weight of the historical disorder measurement index;

[0123] 4) Use Perform a weight normalization operation;

[0124] 5) Use Limit the range of weights to prevent them from becoming too large or too small after the update;

[0125] 6) Return the updated weights .

[0126] Among them, let and These represent the online sorting latency bias ratio and the offline sorting throughput bias ratio, respectively, expressed as:

[0127] (10);

[0128] (11);

[0129] in, and For actual latency and actual throughput, and For target delay and target throughput, and The weight adjustment amounts used for updating are all percentages of the performance deviation of the FPGA reordering system.

[0130] The adaptive weight adjustment algorithm takes the target performance as input. Actual performance Historical weight The algorithm output is the updated weight coefficients. .

[0131] Step S2.3: Based on the current out-of-order metric weights and the current out-of-order metric, determine the comparator resource allocation ratio for online sorting and offline sorting, specifically as follows:

[0132] Collect out-of-order metrics for network telemetry flows, including the out-of-order rate within the online sorting window. Online sorting window disorder rate and weighted out-of-order distance .

[0133] Based on the current weighting coefficients Calculate the comparator resource requirements for online sorting and offline sorting respectively.

[0134] The resource demand intensity of the comparator in online sorting is determined using the resource demand evaluation function of online sorting, as follows:

[0135] (12);

[0136] The resource requirement intensity of the comparator in offline sorting is determined using the resource requirement evaluation function for offline sorting, as follows:

[0137] (13);

[0138] in, For the normalized weighted out-of-order distance, since The numerical range depends on the size of the telemetry data packets, and it is not possible to directly compare the severity of out-of-order delivery under different data packet sizes. Therefore, a normalized weighted out-of-order distance metric is used. These are the weighting coefficients for the in-window disorder rate index, the normalized weighted out-of-order distance index, and the out-of-order rate index between online sorting windows, respectively, satisfying... and . and These represent the resource requirements of the online sorting comparator and the offline sorting comparator, respectively.

[0139] Based on the intensity of resource demand, calculate the resource allocation ratio for online sorting; based on the resource allocation ratio for online sorting, determine and allocate the number of comparator resources for online sorting and the number of comparator resources allocated to offline sorting.

[0140] Resource allocation ratio for online sorting The calculation is as follows:

[0141] (14);

[0142] in, It is a smoothing factor used to ensure the stability of the allocation ratio.

[0143] Number of comparator resources for online sorting ,as follows:

[0144] (15);

[0145] Number of comparator resources for offline sorting ,as follows:

[0146] (16);

[0147] in, It is a floor function.

[0148] 4) Repeat the above steps.

[0149] The resource allocation control algorithm is the core algorithm of this embodiment. Its function is to use the weights obtained by the adaptive weight adjustment algorithm. Allocate the number of comparator resources for online and offline sorting.

[0150] Step S2.4: Adjust the comparator resource allocation ratio according to the current performance deviation of the FPGA reordering system to obtain the optimal comparator resource allocation ratio for online sorting and offline sorting, specifically as follows:

[0151] 1) Estimated latency based on current online sorting Offline sorting and estimated throughput Compared to actual performance: and The performance error is calculated separately for the actual latency and actual throughput, as follows:

[0152] (17);

[0153] (18);

[0154] in, This is the online latency performance deviation value; It is the offline throughput performance deviation value. Both are FPGA reordering system performance deviations, used to update the resource adjustment fine-tuning amount of formula (21) and formula (22).

[0155] Online sorting can be viewed as streaming sorting, and the estimated processing latency for a given number of comparators is expressed as:

[0156] (19);

[0157] in, The base latency includes data reception and preprocessing time; Size of the online sorting window; This represents the actual number of comparators allocated to the online sorting process. This is the FPGA clock frequency. Each data packet is quickly inserted into the correct position via a parallel comparator upon arrival. The number of comparators determines the online sorting and parallel processing capability under multiple telemetry streams. Processing latency is directly proportional to the window size and inversely proportional to the number of comparators.

[0158] Offline sorting can be viewed as ordinary parallel comparison. The estimated sorting throughput for a given data size and number of comparators is expressed as:

[0159] (20);

[0160] in, The amount of data to be sorted; This is the number of sorting levels when using the merge sort algorithm; This represents the actual number of comparators allocated to offline sorting. The number of comparators determines the parallel processing capability of each layer. Throughput is directly proportional to the number of comparators and inversely proportional to the logarithm of the data volume.

[0161] 2) Based on the performance error, calculate the delay adjustment output using a proportional control method. and throughput adjustment output ,as follows:

[0162] (twenty one);

[0163] (twenty two);

[0164] in, This is the proportional gain coefficient.

[0165] 3) Calculate the actual resource allocation adjustment amount in online sorting. Offline sorting and actual allocation of actual resources adjustment ,as follows:

[0166] (twenty three);

[0167] (twenty four);

[0168] in, To achieve the maximum adjustment ratio, It is a floor function.

[0169] 4) Based on the calculations and right and Update.

[0170] In the above algorithm, This is the suggested resource adjustment amount in the latency direction for online sorting. This indicates that the actual latency is lower than the estimated target, and the comparator resources allocated to online sorting can be appropriately reduced; if This indicates that the actual latency is higher than the estimated target, and it is necessary to appropriately increase the comparator resources for online sorting.

[0171] This is the suggested resource adjustment amount in terms of throughput for offline sorting. This indicates that the actual throughput is higher than the estimated target, and the comparator resources can be appropriately reduced for offline sorting; if This indicates that the actual latency is lower than the estimated target, and it is necessary to appropriately increase the comparator resources for offline sorting. and It is the result calculated by the proportional fine-tuning control algorithm in the resource allocation control algorithm. and The amount of fine-tuning.

[0172] like Figure 9 As shown, the input-output relationship of the three sub-algorithms involved in step S2 of this embodiment is described, including the adaptive weight adjustment algorithm, the resource allocation control algorithm, and the proportional fine-tuning control algorithm.

[0173] The adaptive weight adjustment algorithm updates the out-of-order index by monitoring the performance deviation of the FPGA reordering system. , , The weights in the resource allocation decision enable the resource allocation strategy to adaptively adapt to changes in network state and ensure the stability of the FPGA reordering system. The resource allocation control algorithm is the main control loop of the entire comparator resource allocation process. Its function is to allocate resources using the weights obtained by the adaptive weight adjustment algorithm. The proportional fine-tuning control algorithm fine-tunes the deviation based on the resource allocation control algorithm to prevent performance drift and avoid oscillations in the comparator resource allocation process.

[0174] Example 2

[0175] like Figure 10 As shown, this embodiment provides an FPGA comparator resource allocation system for network telemetry reordering, including:

[0176] The out-of-order sorting module is configured to first sort the current out-of-order telemetry data in the FPGA reordering system online, and then sort it offline to obtain completely ordered telemetry data.

[0177] The resource allocation module is configured to determine the optimal allocation ratio of comparator resources for online and offline sorting in the FPGA reordering system using the current out-of-order telemetry data. Specifically:

[0178] Based on the current out-of-order telemetry data, the current out-of-order metrics are statistically analyzed, including the out-of-order rate within the online sorting window, the out-of-order rate between online sorting windows, and the weighted out-of-order distance.

[0179] The weights of historical out-of-order metrics are adaptively adjusted based on the current out-of-order metrics and the current performance deviation ratio of the FPGA reordering system to obtain the current out-of-order metrics weights.

[0180] Based on the current out-of-order metric weights and the current out-of-order metric, determine the comparator resource allocation ratio for online sorting and offline sorting;

[0181] Adjust the comparator resource allocation ratio according to the current performance deviation of the FPGA reordering system to obtain the optimal comparator resource allocation ratio for online sorting and offline sorting.

[0182] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0183] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0184] The proposed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0185] Example 3

[0186] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the FPGA comparator resource allocation method for network telemetry rearrangement as described in Embodiment 1 above.

[0187] Example 4

[0188] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the FPGA comparator resource allocation method for network telemetry rearrangement as described in Embodiment 1 above.

[0189] Example 5

[0190] This embodiment provides a computer program product or computer program, including computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in the FPGA comparator resource allocation method for network telemetry rearrangement described in Embodiment 1 above.

[0191] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0192] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0193] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0194] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0195] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0196] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for allocating FPGA comparator resources for network telemetry reordering, characterized in that, include: The out-of-order telemetry data in the FPGA reordering system is first sorted online, and then sorted offline to obtain completely ordered telemetry data. The optimal allocation ratio of comparator resources for online and offline sorting in the FPGA reordering system is determined using current out-of-order telemetry data. Specifically: Based on the currently received out-of-order telemetry data, the current out-of-order metrics are statistically analyzed, including the out-of-order rate within the online sorting window, the out-of-order rate between online sorting windows, and the weighted out-of-order distance. The out-of-order rate within the online sorting window refers to the ratio of the total number of out-of-order telemetry data packets to the total number of telemetry data packets within an online sorting observation window. The out-of-order rate between online sorting windows refers to the average ratio of the number of out-of-order data packets across two adjacent online sorting windows to the total number of telemetry data packets between the two windows; The weighted out-of-order distance refers to the distance offset between each out-of-order telemetry data packet and its correct sorting position; The weights of historical out-of-order metrics are adaptively adjusted based on the current out-of-order metric and the current performance deviation ratio of the FPGA reordering system to obtain the current out-of-order metric weights, specifically: If the proportion of online sorting delay deviation in the current FPGA reordering system performance deviation ratio is greater than the delay deviation threshold, then the weight adjustment amount is updated according to the online sorting delay deviation. If the offline sorting throughput deviation ratio in the current FPGA reordering system performance deviation ratio is greater than the throughput deviation threshold, then the weight adjustment amount is updated according to the offline sorting throughput deviation. Based on the updated weight adjustment, the weights of the historical out-of-order metrics are smoothed, normalized, and weighted to obtain the current out-of-order metrics weights. Based on the current weights and metrics of the out-of-order metric, the comparator resource allocation ratio for online and offline sorting is determined as follows: Calculate the comparator resource demand intensity for online sorting and offline sorting based on the current out-of-order metric weights and the current out-of-order metric. Calculate the comparator resource allocation ratio for online sorting and the comparator resource allocation ratio for offline sorting based on the comparator resource demand intensity for online sorting and offline sorting. The comparator resource allocation ratio is adjusted based on the current performance deviation of the FPGA reordering system to obtain the optimal comparator resource allocation ratio for online and offline sorting, specifically: Based on the current performance deviation of the FPGA reordering system, a proportional control method is used to calculate the delay adjustment output and throughput adjustment output. The actual resource allocation adjustment for online sorting and the actual resource allocation adjustment for offline sorting are calculated based on the delay adjustment output and the throughput adjustment output. The comparator resource allocation ratio is updated based on the actual resource allocation adjustments made during online sorting and offline sorting to obtain the optimal comparator resource allocation ratio.

2. The FPGA comparator resource allocation method for network telemetry reordering as described in claim 1, characterized in that, The resource requirement intensity of the comparator for the online sorting is calculated as follows: ； The resource requirement intensity of the comparator for offline sorting is calculated as follows: ； in, It is the rate of disorder within the online sorting window. It is the rate of disorder between online sorting windows. The normalized weighted out-of-order distance It is a weighted out-of-order distance. These are the weighting coefficients for the current disorder measurement indicators.

3. An FPGA comparator resource allocation system for network telemetry reordering, characterized in that, include: The out-of-order sorting module is configured to perform online or offline sorting of the currently out-of-order telemetry data in the FPGA reordering system. The resource allocation module is configured to determine the optimal allocation ratio of comparator resources for online and offline sorting in the FPGA reordering system using the current out-of-order telemetry data. Specifically: Based on the current out-of-order telemetry data, the current out-of-order metrics are statistically analyzed, including the out-of-order rate within the online sorting window, the out-of-order rate between online sorting windows, and the weighted out-of-order distance. The out-of-order rate within the online sorting window refers to the ratio of the total number of out-of-order telemetry data packets to the total number of telemetry data packets within an online sorting observation window. The out-of-order rate between online sorting windows refers to the average ratio of the number of out-of-order data packets across two adjacent online sorting windows to the total number of telemetry data packets between the two windows; The weighted out-of-order distance refers to the distance offset between each out-of-order telemetry data packet and its correct sorting position; The weights of historical out-of-order metrics are adaptively adjusted based on the current out-of-order metric and the current performance deviation ratio of the FPGA reordering system to obtain the current out-of-order metric weights, specifically: If the proportion of online sorting delay deviation in the current FPGA reordering system performance deviation ratio is greater than the delay deviation threshold, then the weight adjustment amount is updated according to the online sorting delay deviation. If the offline sorting throughput deviation ratio in the current FPGA reordering system performance deviation ratio is greater than the throughput deviation threshold, then the weight adjustment amount is updated according to the offline sorting throughput deviation. Based on the updated weight adjustment, the weights of the historical out-of-order metrics are smoothed, normalized, and weighted to obtain the current out-of-order metrics weights. Based on the current weights and metrics of the out-of-order metric, the comparator resource allocation ratio for online and offline sorting is determined as follows: Calculate the comparator resource demand intensity for online sorting and offline sorting based on the current out-of-order metric weights and the current out-of-order metric. Calculate the comparator resource allocation ratio for online sorting and the comparator resource allocation ratio for offline sorting based on the comparator resource demand intensity for online sorting and offline sorting. The comparator resource allocation ratio is adjusted based on the current performance deviation of the FPGA reordering system to obtain the optimal comparator resource allocation ratio for online and offline sorting, specifically: Based on the current performance deviation of the FPGA reordering system, a proportional control method is used to calculate the delay adjustment output and throughput adjustment output. The actual resource allocation adjustment for online sorting and the actual resource allocation adjustment for offline sorting are calculated based on the delay adjustment output and the throughput adjustment output. The comparator resource allocation ratio is updated based on the actual resource allocation adjustments made during online sorting and offline sorting to obtain the optimal comparator resource allocation ratio.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the steps in the FPGA comparator resource allocation method for network telemetry rearrangement as described in any one of claims 1-2.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the FPGA comparator resource allocation method for network telemetry rearrangement as described in any one of claims 1-2.

6. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps in the FPGA comparator resource allocation method for network telemetry rearrangement as described in any one of claims 1-2.

Images