A pre-filtered hybrid Katz centrality measurement system, electronic device, and storage medium

Abstract

This invention discloses a pre-filter-based hybrid Katz centrality measurement system, electronic device, and storage medium. The measurement system comprises six parts: data stream sampling, centrality normalization calculation, prefilter precision region maintenance, EdgeSketch compressed backend storage, time decay, and final centrality query. An electronic device and storage medium are also disclosed. Through hierarchical storage management and a time-aware incremental update mechanism, the trade-offs between computational efficiency, memory usage, and measurement accuracy are effectively balanced.

Description

Technical Field

[0001] This invention relates to graph network analysis and flow measurement techniques, particularly flow measurement systems and methods based on dynamic data flows, which are especially suitable for the accurate calculation of the Katz centrality of each node in large-scale dynamic graphs. Background Technology

[0002] With the rapid development of the Internet of Things (IoT) and 5G communication technologies, the number of smart devices connected to the internet has exploded, expanding from traditional computers and mobile phones to a diverse range of devices such as smart homes, industrial sensors, and vehicle-to-everything (V2X) terminals. Global network link speeds continue to improve, and network topologies are shifting from relatively fixed static patterns to dynamic and evolving complex forms. This trend has directly driven the exponential growth of network traffic data and complex graph data. This type of data contains key information such as the relationship patterns between nodes and propagation paths, making in-depth analysis of it of significant practical value. Modeling this highly dynamic, large-scale data as a streaming graph allows for real-time capture of the dynamic update characteristics of nodes and edges, accurately depicting the evolutionary patterns of the network. It has shown broad application prospects in many fields, including social network influence propagation, communication network resource optimization, financial risk node identification, and industrial internet fault location, and has become a research hotspot in data mining and network analysis in recent years. Among numerous graph analysis metrics, Katz centrality, as a classic and effective method for evaluating node importance, occupies an important position in complex network analysis due to its unique advantage of considering both direct and indirect relationships between nodes. Its core value lies not only in considering the number of direct connections between nodes, but also in assigning differentiated weights to paths of all lengths by introducing a decay factor, thus comprehensively reflecting the propagation ability and structural importance of nodes in the entire network. This provides core support for accurately identifying key nodes and analyzing network topology, and therefore has extremely high research and application value.

[0003] Katz centrality is a graph analysis metric that assesses the relative importance of a node in a network by quantifying the cumulative contribution of all paths between nodes (the contribution of long paths decreases exponentially through a decay factor). Accurate measurement of Katz centrality is crucial for practical scenarios such as network optimization, key node identification, and rational resource allocation: in social networks, it can precisely locate core opinion leaders, providing a basis for information dissemination strategy formulation; in communication networks, it can assist in identifying key routing nodes and optimizing data transmission paths to reduce latency; in financial networks, it can promptly identify core nodes of risk propagation and prevent systemic risks. Unfortunately, traditional Katz centrality computation schemes are mostly based on static graph models, making them ill-suited for the dynamic characteristics of streaming data in real-world scenarios. Nodes and edges in streaming graphs are updated frequently and continuously, requiring traditional methods to perform complete path traversal and matrix operations on the entire graph. This leads to computational complexity increasing exponentially with network size, failing to meet real-time analysis needs. Furthermore, to ensure accuracy, traditional methods require storing numerous intermediate results such as adjacency matrices and path matrices, resulting in extremely high memory consumption and making deployment difficult in memory-constrained scenarios like edge computing devices. Moreover, facing a continuous stream of data, traditional algorithms frequently need to recalculate or maintain complex intermediate states, further exacerbating time delays and severely impacting the timeliness of analysis results. Although a few studies in recent years have attempted to introduce classic sketch data structures such as Count-Min Sketch and Bloom Filter to reduce resource consumption through compressed storage and approximate computation, these solutions still have significant shortcomings. On the one hand, existing sketch structures do not fully consider the high dynamism and time dependence of streaming graphs, and are prone to accuracy drift when dealing with frequent updates to node edges. On the other hand, for complex computational scenarios such as TemporalKatz Centrality (TKC), which require the integration of time decay mechanisms and historical influence weights, existing solutions struggle to achieve a balance between efficiency, accuracy, and real-time performance, often facing the dilemma of "insufficient accuracy leads to invalid results, while pursuing accuracy results in low efficiency." Therefore, there is an urgent need to design a novel sketch data structure and computational algorithm that adapts to highly dynamic streaming scenarios and supports time decay mechanisms. Summary of the Invention

[0004] This invention provides a pre-filtered hybrid Katz centrality measurement framework, electronic device, and storage medium, which effectively balances the contradiction between computational efficiency, memory usage, and measurement accuracy through hierarchical storage management and a time-aware incremental update mechanism.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A pre-filtered hybrid Katz centrality measurement system includes:

[0007] Data stream sampling: Extract node identifiers from the arriving edge data and perform hash index calculations to determine whether to transmit the stream identifiers to on-chip memory for processing;

[0008] Centrality normalization calculation: The node centrality is normalized according to the time decay function;

[0009] Prefilter Precision Zone Maintenance: Through promotion and demotion strategies, it ensures that it always retains the latest and most accurate centrality values, and saves the accurate Katz centrality values ​​of the Top-K influential nodes in the network in real time;

[0010] EdgeSketch compressed backend storage: It maintains massive long-tail node information with controllable memory growth by using fingerprint compression, bucket structure and hot / cold entry differentiation technology.

[0011] Time decay update: based on a time function The node centrality is decayed and incrementally calculated to ensure the time-dynamic accuracy of the centrality measurement.

[0012] Final centrality query: At the end of the measurement period or during real-time query, the Katz centrality estimate of the node is returned based on the joint state of Prefilter and EdgeSketch. This provides a fast and accurate response for Top-K nodes, while for ordinary nodes, a reliable approximation is returned through robust minimum estimation. This achieves a balance between query efficiency and result reliability at the system level.

[0013] The aforementioned hybrid Katz centrality measurement system based on pre-filtering includes the following specific implementation process for data stream sampling:

[0014] First, for the incoming data packets, the switch will first extract relevant fields from the packet header according to the measurement requirements to form a flow identifier f. The flow identifier f uniquely identifies a network flow, ensuring that the data packets of each flow can be correctly associated in subsequent processing.

[0015] Secondly: The switch uses a combination of the flow identifier f and the timestamp t for hash mapping;

[0016] Specifically, through flow identifiers and timestamp Concatenation calculation of hash index The mapping formula is as follows:

[0017] (Formula 1)

[0018] in, It is a stream identifier-index mapping function. This indicates the concatenation of the stream identifier and the timestamp. It is an independent uniform hash function. This represents the maximum value that the hash function can take. This represents the modulo operator, used to calculate the hash index. The switch will determine whether the index is less than or equal to a preset threshold. ,when At that time, the flow identifier The data is transferred to on-chip memory via a high-speed bus for further processing.

[0019] The aforementioned hybrid Katz centrality measurement system based on pre-filtering, wherein the specific implementation process of the centrality normalization calculation includes:

[0020] First: In the input of a dynamic flow graph, each flow identifier represents an edge, and each edge is represented as... , where u is the source node, v is the target node, and t is the timestamp when the edge arrives;

[0021] Secondly, the system processes edge data sequentially according to the time series and performs a centrality incremental update process for each edge;

[0022] Specifically, define the time decay function. , where c is a configurable decay coefficient used to control the sensitivity of Katz centrality to time changes;

[0023] Upon arrival at each edge, first obtain the current timestamp t and calculate its normalized time decay value. .

[0024] The aforementioned hybrid Katz centrality measurement system based on pre-filtering maintains the normalized centrality values ​​of nodes internally to avoid frequent global decay calculations. Its definition is as follows:

[0025] (Formula 2)

[0026] in, Indicates the last update time of node u. The true Katz centrality value;

[0027] When a new edge Upon arrival, calculate the increment of node v based on the normalized centrality value of node u. The calculation formula is as follows:

[0028] (Formula 3)

[0029] in, This is the path decay factor, used to control the intensity of Katz centrality propagation.

[0030] The aforementioned hybrid Katz centrality measurement system based on pre-filtering, wherein the specific implementation process of maintaining the precise region of the prefilter includes:

[0031] First: Before the measurement begins, initialize the Prefilter data structure, which is a heap storage area with a fixed number of slots K, used to store the records of the top K nodes with the highest centrality;

[0032] Second: Each Prefilter slot contains the following field: Node Identifier Normalization centrality Last update timestamp and heap index The total size of each slot is 24 bytes;

[0033] Third: For any node After its centrality is updated, if its new estimated centrality is greater than the minimum value in the Prefilter heap... times, of which If the threshold for promotion is not met, a promotion operation is performed, inserting the node into the Prefilter and demoting the top node of the Prefilter heap back to the backend storage.

[0034] Fourth: The degradation operation occurs when the node has not been updated for a long time, which satisfies the condition. When triggered, it is removed from Prefilter and written to EdgeSketch. If Prefilter is full, the smallest node at the top of the heap is popped and written back to the backend EdgeSketch.

[0035] The aforementioned hybrid Katz centrality measurement system based on pre-filtering, specifically implementing the EdgeSketch compressed backend storage process, includes:

[0036] First: EdgeSketch consists of two independent arrays A1 and A2, each array contains w buckets, and each bucket can hold a maximum of B items;

[0037] Each entry is divided into two types: hot and cold;

[0038] Hot entries fully store the node ID and timestamp, while cold entries use a 16-bit fingerprint instead of the node ID to save memory.

[0039] The data structure definition for the hot entry is as follows: type: 1 byte; vertex_id: 8 bytes; tile_C: 4 bytes; last_ts: 4 bytes; padding 3 bytes, total 20 bytes;

[0040] The data structure for the cold entry is defined as follows: type: 1 byte; fp: 2 bytes; tilde_C: 4 bytes; age: 1 byte; padding 2 bytes, total 10 bytes;

[0041] Second: Whenever the side Upon arrival, the system calculates two hash indices. and Then check the storage buckets one by one. , Check whether the hot entry ID or cold entry fingerprint matches a vertex in the edge;

[0042] Specifically, system updates can be divided into the following three situations:

[0043] If a hot entry exists and the ID matches, then simply accumulate the results. And update the timestamp;

[0044] If a cold entry exists and the fingerprint matches, then accumulate. And update the age;

[0045] If no match is found and the bucket is not full, insert a new cold entry;

[0046] If the bucket is full, then execute the replacement strategy;

[0047] Specifically, the replacement strategy is implemented as follows:

[0048] Let the entry with the minimum centrality of the current bucket be When the new amount satisfy:

[0049] (Formula 4)

[0050] At that time, The entry is replaced with the new node; otherwise, for The entry undergoes a slight decay operation:

[0051] (Formula 5)

[0052] in For deterministic replacement threshold, This is the attenuation coefficient.

[0053] The aforementioned hybrid Katz centrality measurement system based on pre-filtering, wherein the time decay update process specifically includes:

[0054] First: When node centrality is updated, it is based on the node's last update timestamp. Calculate the time decay factor using the current timestamp t. And update as follows:

[0055] (Formula 6)

[0056] Second: Write the new normalized centrality and the current timestamp back to the corresponding structure.

[0057] The aforementioned hybrid Katz centrality measurement system based on pre-filtering, wherein the final centrality query implementation process includes:

[0058] At the end of the measurement period or during real-time query, first determine whether the node m to be queried is in the Prefilter:

[0059] If present, then return:

[0060] (Formula 7)

[0061] If not found, then search for a matching entry in both buckets of EdgeSketch and take the estimated value. , and restore the true centrality according to φ(t);

[0062] The specific implementation method is as follows:

[0063] (Formula 8)

[0064] in These are the estimated values ​​of node m in the two candidate buckets, respectively;

[0065] Finally, the system returns the centrality estimate of the nodes and can update the Top K node set in real time.

[0066] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method steps of a pre-filtered hybrid Katz centrality measurement system as described above.

[0067] A storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of a method for a pre-filtered hybrid Katz centrality measurement system as described above.

[0068] The beneficial effects of this invention are:

[0069] 1. The pre-filtering and hierarchical storage mechanism of the present invention can accurately identify and maintain the most influential Top-K nodes based on the real-time changes in node centrality in a dynamic data flow environment, and record this key information in a fixed and precise storage area. This hybrid structure achieves significant optimization in terms of memory overhead and computational accuracy.

[0070] 2. This invention also designs a promotion and demotion strategy based on competitive elimination to dynamically manage node members in the precise storage area and prioritize the retention of nodes with consistently high centrality within the limited storage space. For nodes with declining centrality or insufficient activity, this mechanism will promptly degrade them to the compressed storage area, thereby effectively improving the utilization efficiency of limited storage resources. In addition, the demotion decision is based not only on the centrality value but also on the node's most recent update time, achieving more intelligent and reasonable resource scheduling and further improving the overall measurement accuracy of the system.

[0071] 3. This invention combines the tail-biased distribution characteristic of node centrality, which is commonly followed in real networks, and designs a hierarchical measurement architecture of "precise + approximate," achieving higher accuracy and lower overhead in centrality estimation. Finally, when performing a centrality query, this invention only needs to jointly query the precise storage area and the compressed storage area to quickly return the result, thus significantly reducing the computational complexity of the query process and outperforming traditional streaming graph centrality measurement schemes. Attached Figure Description

[0072] Figure 1 This is a flowchart of the overall system structure of the present invention;

[0073] Figure 2 This is a flowchart of the data stream sampling section;

[0074] Figure 3 This is a flowchart of the centrality normalization calculation part;

[0075] Figure 4 This is a flowchart of the Prefilter precision area maintenance section;

[0076] Figure 5 This is a flowchart of the EdgeSketch compression backend storage section;

[0077] Figure 6 This is a flowchart of the time decay and incremental update part;

[0078] Figure 7 This is a flowchart of the final centrality query part. Detailed Implementation

[0079] The present invention will now be further described with reference to the accompanying drawings. Example 1

[0080] This invention provides a pre-filter-based hybrid Katz centrality measurement system, comprising six parts: data stream sampling, centrality normalization calculation, Prefilter precise region maintenance, EdgeSketch compressed backend storage, time decay, and final centrality query.

[0081] The data stream sampling section is used to extract node identifiers from the arriving edge data and perform hash index calculations to determine whether to transmit the stream identifiers to on-chip memory for processing. This controls the processing scale at the data source, significantly improves computational efficiency, and prevents memory overflow. At the same time, deterministic sampling ensures the continuous capture of key traffic patterns.

[0082] The specific implementation of the data stream sampling part is as follows:

[0083] Upon receiving a data packet, the switch first extracts relevant fields from the packet header according to measurement requirements, such as source address, destination address, protocol type, and timestamp, to form a flow identifier. Flow identifier It uniquely identifies a network flow, ensuring that packets from each flow can be correctly associated in subsequent processing.

[0084] Next, the switch uses a combination of flow identifier and timestamp for hash mapping. Specifically, through the flow identifier... and timestamp Concatenation calculation of hash index The mapping formula is as follows:

[0085] (Formula 1)

[0086] in, It is a stream identifier-index mapping function. This indicates the concatenation of the stream identifier and the timestamp. It is an independent uniform hash function. This represents the maximum value that the hash function can take. Represents the modulo operator. The hash index is calculated from this value. The switch will determine whether the index is less than or equal to a preset threshold. Only when At that time, the flow identifier Only then will it be transferred to the on-chip memory via a high-speed bus (such as PCIe) for further processing.

[0087] The specific implementation steps are as follows: Figure 2 As shown, it includes the following steps:

[0088] S21: Received data streams from the switch, where each stream contains a source node. Target node and timestamp ;

[0089] S22: Extract the flow identifier from the header of each flow. This identifier uniquely corresponds to a single stream;

[0090] S23: Calculate the stream identifier using (Formula 1) The hash index i;

[0091] S24: Calculated hash index With threshold In comparison, if If the result is positive, proceed to step S25; otherwise, proceed to step S26.

[0092] S25: Transfer the stream identifier to the on-chip memory via the PCIe bus for storage and processing, so that it can be used by subsequent modules;

[0093] S26: Complete message sampling and proceed to the centrality normalization calculation section.

[0094] The centrality normalization calculation part is used to normalize the node centrality according to the time decay function to eliminate the influence of time difference. This method transforms the complex global exponential decay calculation into an incremental update that can be linearly accumulated, which greatly reduces the computational complexity and is the key to supporting the high throughput of the system.

[0095] The specific implementation of the centrality normalization calculation is as follows:

[0096] In the input of a dynamic flow graph, each flow identifier represents an edge, and each edge is represented as... ,in As the source node, For the target node, This is the timestamp of when the edge was reached;

[0097] The system processes edge data sequentially according to the time series and performs a centrality incremental update process for each edge.

[0098] Specifically, define the time decay function. ,in This is a configurable decay factor used to control the sensitivity of Katz centrality to time variations.

[0099] Upon arrival at each edge, first obtain the current timestamp t and calculate its normalized time decay value. .

[0100] To avoid frequent global decay calculations, the system internally maintains the normalized centrality values ​​of nodes. Its definition is as follows:

[0101] (Formula 2)

[0102] in, Represents a node Last update time The true Katz centrality value.

[0103] When a new edge Upon arrival, according to the node Normalized centrality value calculation node Increment The calculation formula is as follows:

[0104] (Formula 3)

[0105] in, This is the path decay factor, used to control the intensity of Katz centrality propagation.

[0106] The flowchart for the centrality normalization calculation part is as follows: Figure 3 As shown, this part is used to calculate the normalized Katz centrality value of the node. And process the decay function, specifically including the following steps:

[0107] S31: Obtain node data stream from the sampling module ;

[0108] S32: Based on timestamp and current timestamp Calculate the time decay function This is used to adjust the change in centrality value over time;

[0109] S33: For each reached edge The source node is calculated using (Formula 2). Normalized Katz centrality value ;

[0110] S34: Calculate the target node according to (Formula 3) Incremental value ;

[0111] S35: Calculate the nodes normalized centrality value As an incremental output, it is used for subsequent module updates.

[0112] The Prefilter precision region maintenance section is used to save the precise Katz centrality values ​​of the Top-K influential nodes in the network in real time. Through promotion and demotion strategies, it ensures that the latest and most accurate centrality values ​​are always retained. Limited memory and computing resources are preferentially allocated to the most important nodes, thereby ensuring the absolute accuracy of the Top-K node results that are of most concern to the business under the condition of overall resource constraints.

[0113] The specific implementation of the Prefilter precision region maintenance part is as follows:

[0114] Before the measurement begins, initialize the Prefilter data structure, which has a fixed number of slots. The heap-style storage area is used to store the records of the top K most central nodes.

[0115] Each Prefilter slot contains the following fields: Node Identifier Normalization centrality Last update timestamp and heap index .

[0116] The total size of each slot is 24 bytes.

[0117] For any node After its centrality is updated, if its new estimated centrality is greater than the minimum value in the Prefilter heap... times ( If the threshold is reached, then the promotion operation is performed, the node is inserted into the Prefilter, and the top node of the Prefilter heap is demoted back to the backend storage;

[0118] The degradation operation occurs when the node has not been updated for a long time, which satisfies the condition. When triggered, it is removed from Prefilter and written to EdgeSketch.

[0119] If the Prefilter is full, pop the smallest node at the top of the heap and write it back to the backend EdgeSketch.

[0120] This process ensures that Prefilter always accurately preserves the centrality of the Top-K nodes.

[0121] The flowchart for the Prefilter precision area maintenance section is as follows: Figure 4 As shown, this section is used to maintain the Katz centrality of the Top-K nodes and ensure their accuracy, specifically including the following steps:

[0122] S41: Set the Prefilter precision area, which contains a fixed number of... Each slot stores the identifier of a node. Normalization centrality Last update timestamp and heap index ;

[0123] S42: For each node centrality update, when the node... The new normalized centrality value Exceeding the minimum value in Prefilter If the value exceeds the limit, proceed to step S43; otherwise, proceed to step S44.

[0124] S43: Check if the Prefilter is full. If the Prefilter is not full, proceed to step S431; otherwise, proceed to step S432.

[0125] S431: Move the node Promoted to Prefilter;

[0126] S432: Pop the smallest node from the Prefilter, write it back to the EdgeSketch backend storage, and then promote the node to the Prefilter;

[0127] S44: Update the corresponding node records in Prefilter and EdgeSketch.

[0128] The EdgeSketch compressed backend storage section is used to approximate the centrality information of ordinary nodes with low memory cost. Through fingerprint compression, bucket structure and hot and cold entry differentiation, it maintains massive long-tail node information with controllable memory growth, which is the foundation for realizing system scalability.

[0129] The specific implementation of the compression backend storage part of EdgeSketch is as follows:

[0130] EdgeSketch consists of two independent arrays, A1 and A2. Each array contains w buckets, and each bucket can hold a maximum of B entries.

[0131] Each entry is divided into two types: hot and cold.

[0132] Hot entries store the complete node ID and timestamp, while cold entries use a 16-bit fingerprint instead of the node ID to save memory.

[0133] The data structure definition for the hot entry is as follows: type: 1 byte; vertex_id: 8 bytes; tile_C: 4 bytes; last_ts: 4 bytes; padding 3 bytes, total 20 bytes;

[0134] The data structure for the cold entry is defined as follows: type: 1 byte; fp: 2 bytes; tilde_C: 4 bytes; age: 1 byte; padding 2 bytes, totaling 10 bytes.

[0135] Whenever the side Upon arrival, the system calculates two hash indices. and Then check the storage buckets one by one. , Check whether the hot entry ID or cold entry fingerprint matches a vertex in the edge. Specifically, the system update can be divided into the following three cases:

[0136] If a hot entry exists and the ID matches, then simply accumulate the results. And update the timestamp;

[0137] If a cold entry exists and the fingerprint matches, then accumulate. And update the age;

[0138] If no match is found and the bucket is not full, insert a new cold entry;

[0139] If the bucket is full, the replacement strategy is executed as follows:

[0140] The replacement strategy is implemented as follows:

[0141] Let the entry with the minimum centrality of the current bucket be When the new amount satisfy:

[0142] (Formula 4)

[0143] At that time, The entry is replaced with the new node; otherwise, for The entry undergoes a slight decay operation:

[0144] (Formula 5)

[0145] in For deterministic replacement threshold, This is the attenuation coefficient.

[0146] This mechanism ensures that hot nodes remain in the upper levels of the bucket, thus prioritizing the retention of highly central nodes.

[0147] The flowchart of EdgeSketch's compression backend storage section is as follows: Figure 5 As shown, this section is used to compress the Katz centrality of storing a large number of nodes under memory constraints, and specifically includes the following steps:

[0148] S51: EdgeSketch uses two independent arrays and Each array contains Each bucket can store a maximum of [number] containers. Items;

[0149] S52: For each stream of data, calculate its corresponding hash index. and ;

[0150] S53: Check the storage bucket , Check whether the hot entry ID or cold entry fingerprint matches a vertex in the edge;

[0151] S541: If a hot entry exists and the ID matches, then accumulate. Update the timestamp and proceed to execute S55;

[0152] S542: If a cold entry exists and the fingerprint matches, then accumulate. Then update the age and jump to execute S55;

[0153] S543: If no match is found and the bucket is not full, insert a new cold entry and proceed to S55.

[0154] S544: If the bucket is full, select whether to replace the existing entry according to the replacement strategy, and then proceed to S55.

[0155] S5441: Find the entry with the minimum centrality in the current bucket, and let this entry be... ;

[0156] S5442: Determine the new quantity Does it satisfy (Formula 4)? If it does, proceed to step S5443; otherwise, proceed to step S5444.

[0157] S5443: When the new quantity When (Formula 4) is satisfied, The entry is replaced with the new node, and execution jumps to S55;

[0158] S5444: If (Formula 4) is not satisfied, then proceed according to (Formula 5). Perform a slight attenuation operation and jump to execute S55;

[0159] S55: After each update, EdgeSketch ensures that hot nodes remain on the upper layer.

[0160] The time decay update part is used to update the time function. The node centrality is decayed and incrementally calculated, thus ensuring the time-dynamic accuracy of the centrality measure.

[0161] The specific implementation of the time decay update part is as follows:

[0162] When node centrality is updated, it is based on the node's last update timestamp. Calculate the time decay factor using the current timestamp t. And update as follows:

[0163] (Formula 6)

[0164] Then the new normalized centrality and the current timestamp are written back to the corresponding structure.

[0165] This mechanism ensures that Katz centrality decays over time while maintaining the stored values ​​in a linearly summable, normalized form.

[0166] The flowchart for the time decay and incremental update section is as follows: Figure 6 As shown, this section is used to calculate the Katz centrality of the updated node based on the time decay function and the increment, specifically including the following steps:

[0167] S61: Obtain the node's centrality data and timestamp from Prefilter or EdgeSketch;

[0168] S62: Based on the current timestamp Compared to the node's last update timestamp The difference between them, calculate the time decay factor ;

[0169] S63: Calculate the centrality increment of the node according to (Formula 3). ;

[0170] S64: Calculate the new normalized centrality of the node according to (Formula 6). ;

[0171] S65: Output the updated node centrality value and the current timestamp for use by other modules.

[0172] The final centrality query component returns the Katz centrality estimate of the node based on the joint state of Prefilter and EdgeSketch at the end of the measurement period or during real-time query. This fully leverages the advantages of the hierarchical architecture, enabling fast and accurate responses for Top-K nodes and returning reliable approximate values ​​for ordinary nodes through robust minimum estimation. This achieves a balance between query efficiency and result reliability at the system level.

[0173] The specific implementation of the final centrality query part is as follows:

[0174] At the end of the measurement period or during real-time query, first determine whether the node m to be queried is in the Prefilter:

[0175] If present, then return:

[0176] (Formula 7)

[0177] If not found, then search for a matching entry in both buckets of EdgeSketch and take the estimated value. And restore the true centrality according to φ(t).

[0178] The specific implementation method is as follows:

[0179] (Formula 8)

[0180] in , respectively, are the estimated values ​​of node m in the two candidate buckets.

[0181] Finally, the system returns the centrality estimate of the nodes and can update the Top K node set in real time.

[0182] The flowchart for the final centrality query part is as follows: Figure 7 As shown, this section is used to return the Katz centrality estimate of a node based on Prefilter and EdgeSketch when a query request is made. Specifically, it includes the following steps:

[0183] S71: Receive a node query request and obtain the identifier of the node to be queried;

[0184] S72: Check if the node is in Prefiler. If it is, proceed to step S721; otherwise, proceed to step S7221.

[0185] S721: If the node is in the Prefilter, based on the time decay factor... And calculate its centrality value using formula (7), then proceed to step S73;

[0186] S7221: If a node is not in the Prefilter, search for and estimate its centrality value from the two candidate buckets in EdgeSketch;

[0187] S7222: Based on time decay factor And (Formula 8) recovers the true Katz centrality value of the node;

[0188] S73: Returns the Katz centrality estimate of the node. Example 2

[0189] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method steps of a pre-filtered hybrid Katz centrality measurement system as described above. Example 3

[0190] The present invention also provides a storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of a method for a pre-filtered hybrid Katz centrality measurement system as described above.

[0191] In summary, a pre-filtered hybrid Katz centrality measurement framework, electronic device, and storage medium effectively balance the contradictions between computational efficiency, memory usage, and measurement accuracy through hierarchical storage management and a time-aware incremental update mechanism.

[0192] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A hybrid Katz centrality measurement system based on pre-filtering, characterized in that: include: Data stream sampling: Extract node identifiers from the arriving edge data and perform hash index calculations to determine whether to transmit the stream identifiers to on-chip memory for processing; Centrality normalization calculation: The node centrality is normalized according to the time decay function; Prefilter Precision Zone Maintenance: Through promotion and demotion strategies, it ensures that it always retains the latest and most accurate centrality values, and saves the accurate Katz centrality values ​​of the Top-K influential nodes in the network in real time; EdgeSketch compressed backend storage: It maintains massive long-tail node information with controllable memory growth by using fingerprint compression, bucket structure and hot / cold entry differentiation technology. Time decay update: based on the time decay function Perform decay correction and incremental calculation on node centrality; Final centrality query: At the end of the measurement period or during real-time query, the Katz centrality estimate of the node is returned based on the joint state of Prefilter and EdgeSketch. This provides a fast and accurate response for Top-K nodes, while for ordinary nodes, a reliable approximation is returned through robust minimum estimation. The specific implementation process of the EdgeSketch compressed backend storage includes: First: EdgeSketch consists of two independent arrays A1 and A2, each array contains w buckets, and each bucket can hold a maximum of B items; Each entry is divided into two types: hot and cold; Hot entries fully store the node ID and timestamp, while cold entries use a 16-bit fingerprint instead of the node ID to save memory. The data structure definition for the hot entry is as follows: type: 1 byte; vertex_id: 8 bytes; tile_C: 4 bytes; last_ts: 4 bytes; padding 3 bytes, total 20 bytes; The data structure for the cold entry is defined as follows: type: 1 byte; fp: 2 bytes; tilde_C: 4 bytes; age: 1 byte; padding 2 bytes, total 10 bytes; Second: Whenever the side Upon arrival, where u is the source node, v is the destination node, and t is the timestamp of arrival for each edge, the system calculates two hash indices. and , and Use two independent uniform hash functions, and then check the buckets sequentially. , Check whether the hot entry ID or cold entry fingerprint matches a vertex in the edge; Specifically, system updates can be divided into the following three situations: If a hot entry exists and the ID matches, then simply accumulate the results. And update the timestamp. This is the normalized centrality value of the matching node; If a cold entry exists and the fingerprint matches, then accumulate. And update the age; If no match is found and the bucket is not full, insert a new cold entry; If the bucket is full, then execute the replacement strategy; Specifically, the replacement strategy is implemented as follows: Let the entry with the minimum centrality of the current bucket be When the new amount satisfy: ； At that time, The entry is replaced with a new node, where for The normalized centrality value of the node; otherwise, for The entry undergoes a slight decay operation: ； in For deterministic replacement threshold, This is the attenuation coefficient.

2. The hybrid Katz centrality measurement system based on pre-filtering according to claim 1, characterized in that: The specific implementation process of the data stream sampling includes: First, for the incoming data packet, the switch will first extract relevant fields from the packet header according to the measurement requirements to form a flow identifier f, which uniquely identifies a network flow. Secondly: The switch uses a combination of the flow identifier f and the timestamp t for hash mapping; Specifically, the hash index i is calculated by concatenating the stream identifier f and the timestamp t, and the mapping formula is as follows: ； in, It is a stream identifier-index mapping function. This indicates the concatenation of the stream identifier and the timestamp. It is an independent uniform hash function. This represents the maximum value that the hash function can take. This represents the modulo operator. Using the calculated hash index i, the exchange determines whether the index is less than or equal to a preset threshold. ,when At that time, the stream identifier f is transmitted to the on-chip memory via the high-speed bus for subsequent processing.

3. The hybrid Katz centrality measurement system based on pre-filtering according to claim 2, characterized in that: The specific implementation process of the centrality normalization calculation includes: First: In the input of a dynamic flow graph, each flow identifier represents an edge, and each edge is represented as... Where u is the source node, v is the target node, and t is the timestamp of each edge arrival; Secondly, the system processes edge data sequentially according to the time series and performs a centrality incremental update process for each edge; Specifically, define the time decay function. , where c is a configurable decay coefficient and e is the base of the natural logarithm, used to control the sensitivity of Katz centrality to time changes; Upon arrival at each edge, first obtain the current timestamp t and calculate its normalized time decay value. .

4. The hybrid Katz centrality measurement system based on pre-filtering according to claim 3, characterized in that: To avoid frequent global decay calculations, the system internally maintains the normalized centrality values ​​of nodes. Its definition is as follows: ； in, Indicates the last update time of node u. The true Katz centrality value, Indicates time The time decay function value; When a new edge Upon arrival, calculate the increment of node v based on the normalized centrality value of node u. The calculation formula is as follows: ； in, This is the path decay factor, used to control the intensity of Katz centrality propagation.

5. A hybrid Katz centrality measurement system based on pre-filtering according to claim 3 or 4, characterized in that: The specific implementation process of maintaining the Prefilter precision region includes: First: Before the measurement begins, initialize the Prefilter data structure, which is a heap storage area with a fixed number of slots K, used to store the records of the top K nodes with the highest centrality; Second: Each Prefilter slot contains the following field: Node Identifier Normalization centrality Last update timestamp and heap index The total size of each slot is 24 bytes; Third: For any node v, after its centrality is updated, if its new estimated centrality is greater than the minimum value in the Prefilter heap... times, of which If the threshold for promotion is not met, a promotion operation is performed, inserting the node into the Prefilter and demoting the top node of the Prefilter heap back to the backend storage. Fourth: The degradation operation occurs when the node has not been updated for a long time, which satisfies the condition. When triggered, it is removed from Prefilter and written to EdgeSketch. If Prefilter is full, the smallest node at the top of the heap is popped and written back to the backend EdgeSketch.

6. The hybrid Katz centrality measurement system based on pre-filtering according to claim 5, characterized in that: The specific implementation process of the time decay update includes: First: When node centrality is updated, it is based on the node's last update timestamp. Calculate the time decay factor using the current timestamp t. And update as follows: ； in, Let this be the new centrality value of the node to be updated. The old centrality value of the node to be updated; Second: Write the new normalized centrality and the current timestamp back to the corresponding structure.

7. A hybrid Katz centrality measurement system based on pre-filtering according to claim 6, characterized in that: The specific implementation process of the final centrality query includes: At the end of the measurement period or during real-time query, first determine whether the node m to be queried is in the Prefilter: If present, then return: ； in, The centrality estimate at time t is... Let m be the normalized centrality value of node m. If not found, then search for a matching entry in both buckets of EdgeSketch and take the estimated value. , and according to Restoring true centrality; The specific implementation method is as follows: ； in , These are the estimated values ​​of node m in the two candidate buckets, respectively; Finally, the system returns the centrality estimate of the nodes and can update the Top K node set in real time.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the method steps of a pre-filtered hybrid Katz centrality measurement system as described in any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it comprises the method steps of a pre-filtered hybrid Katz centrality measurement system as described in any one of claims 1 to 7.

Images